Add more metrics to in_monitor_agent #2450
Conversation
lib/fluent/plugin/output.rb
Outdated
| } | ||
|
|
||
| if @buffer && @buffer.respond_to?(:statistics) | ||
| (@buffer.statistics && @buffer.statistics['buffer'] || {}).each do |k, v| |
There was a problem hiding this comment.
When @buffer.statistics return false/nil?
There was a problem hiding this comment.
looks unnecessary. thank you!
7e0941a
| stats['newest_timekey'] = m | ||
| end | ||
|
|
||
| { 'buffer' => stats } |
There was a problem hiding this comment.
What is the merit of {'buffer' => stats} rather than stats?
There was a problem hiding this comment.
To have consistent with the result of Output#statistics.
I think it better having { 'output' => stats } than stats since in_monitor_agent.rb can identify which the data come from when the components we want to collect stats( at https://github.com/fluent/fluentd/pull/2450/files#diff-0ad62dd2012fecaa6ed2232f8918c528R382) increase.
|
👋Semi-related question: do you think it would make sense to add |
they are used for monitoring Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
7e0941a to
63c006e
Compare
Looks neat for me. I added |
|
Fantastic, thank you. |
Is this useful? This value seems the total of flush time. How to use this value in monitoring? |
|
I think it is useful for getting the value which is like average flush_time in 5 mins. |
This is exactly what I've meant 👍 🎉 |
|
Okay, I understood. |
Good catch… Since flush_time_count is counter, I think that this value should increase monotonically. I think it's good to use Integer type (to calling done 628379e |
|
@ganmacs Do we need to multiply |
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
628379e to
8b5e34f
Compare
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
8b5e34f to
11105cf
Compare
lib/fluent/plugin/output.rb
Outdated
| @@ -1202,7 +1204,10 @@ def backup_chunk(chunk, using_secondary, delayed_commit) | |||
|
|
|||
| def check_slow_flush(start) | |||
| elapsed_time = Fluent::Clock.now - start | |||
| # millsec precision | |||
| @counters_monitor.synchronize { @flush_time_count += (elapsed_time * 1000).to_i } | |||
There was a problem hiding this comment.
How about (elapsed_time * 1000).to_i calculation outside of synchronize?
Signed-off-by: Yuta Iwama <ganmacs@gmail.com>
|
Thx! |
Which issue(s) this PR fixes:
no
What this PR does / why we need it:
I added metrics which output and buffer have to the response of in_monitor_agent plugin to fetch the fluentd's inner stats.
Docs Changes:
Need to revise examples https://docs.fluentd.org/input/monitor_agent#output-example
Release Note:
Same as the title