Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add calculate_rate for NetworkIn/Out, DiskRead/WriteBytes and DiskRead/WriteOps #13203

Merged
merged 4 commits into from
Aug 18, 2019
Merged

Add calculate_rate for NetworkIn/Out, DiskRead/WriteBytes and DiskRead/WriteOps #13203

merged 4 commits into from
Aug 18, 2019

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Aug 8, 2019

This PR is to try solve elastic/kibana#42687 in metricbeat side.
In ec2 metricset, adding a config parameter called calculate_rate to calculate rates for 8 metrics:

network.in.bytes_per_sec = network.in.bytes / 300 or network.in.bytes / 60
network.out.bytes_per_sec = network.out.bytes / 300 or network.out.bytes / 60
network.in.packets_per_sec = network.in.packets / 300 or network.in.bytes / 60
network.out.packets_per_sec = network.out.packets / 300 or network.out.bytes / 60
diskio.read.bytes_per_sec = diskio.read.bytes / 300 or diskio.read.bytes / 60
diskio.write.bytes_per_sec = diskio.write.bytes / 300 or diskio.write.bytes / 60
diskio.read.ops_per_sec = diskio.read.ops / 300 or diskio.read.ops / 60
diskio.write.ops_per_sec = diskio.write.ops / 300 or diskio.write.ops / 60

Right now both the original metrics and the rate metrics are reported by ec2 metricset, in the future, the original metrics will be removed.

@kaiyan-sheng
Copy link
Contributor Author

If detailed monitoring is enabled, then these metrics will be sent to Cloudwatch every 1 min instead of every 5min. The rate calculated here is for each data point, which has nothing to do with the collection period that user sets in Metricbeat aws module. Therefore this calculation doesn't seem to be able to replace by adding collection period into each event. @jsoriano What do you think?

Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments, and this will also need a changelog entry.

x-pack/metricbeat/module/aws/_meta/config.yml Outdated Show resolved Hide resolved
x-pack/metricbeat/module/aws/ec2/_meta/docs.asciidoc Outdated Show resolved Hide resolved
x-pack/metricbeat/module/aws/ec2/_meta/data.json Outdated Show resolved Hide resolved
x-pack/metricbeat/module/aws/ec2/ec2.go Show resolved Hide resolved
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It LGTM, but it needs a changelog entry.

x-pack/metricbeat/module/aws/ec2/ec2.go Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Metricbeat Metricbeat review Team:Integrations Label for the Integrations team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants