Cherry-pick #20171 to 7.x: Add host inventory metrics to ec2 metricset

[kaiyan-sheng]

Cherry-pick of PR #20171 to 7.x branch. Original message:

What does this PR do?

This PR is to add proposed host common fields into ec2 metricset:

• host.id
• host.name
• host.cpu.pct
• host.network.in.bytes
• host.network.in.packets
• host.network.out.bytes
• host.network.out.packets
• host.disk.read.bytes
• host.disk.write.bytes

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

1. Start Metricbeat AWS module: ./metricbeat modules enable aws
2. Edit modules.d/aws.yml to only include ec2 metricset:

- module: aws
  period: 5m
  credential_profile_name: elastic-beats
  metricsets:
    - ec2

3. Change add_host_metadata processor config in metricbeat.yml file:

processors:
  - add_host_metadata:
      replace_fields: false

4. Start metricbeat
5. You should see metrics from ec2 metricset and includes fields listed above.
   (host.name only equals to EC2 instance name when EC2 instance name is available.)

Event Example

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "aws": {
        "ec2": {
            "cpu": {
                "credit_balance": 144,
                "credit_usage": 0.05477,
                "surplus_credit_balance": 0,
                "surplus_credits_charged": 0,
                "total": {
                    "pct": 1.166484417714406
                }
            },
            "diskio": {
                "read": {
                    "bytes": 0,
                    "bytes_per_sec": 0,
                    "count": 0,
                    "count_per_sec": 0
                },
                "write": {
                    "bytes": 0,
                    "bytes_per_sec": 0,
                    "count": 0,
                    "count_per_sec": 0
                }
            },
            "instance": {
                "core": {
                    "count": 1
                },
                "image": {
                    "id": "ami-04bc3da8f14823e88"
                },
                "monitoring": {
                    "state": "disabled"
                },
                "private": {
                    "dns_name": "ip-172-31-9-119.us-west-1.compute.internal",
                    "ip": "172.31.9.119"
                },
                "public": {
                    "dns_name": "ec2-13-52-163-56.us-west-1.compute.amazonaws.com",
                    "ip": "13.52.163.56"
                },
                "state": {
                    "code": 16,
                    "name": "running"
                },
                "threads_per_core": 1
            },
            "network": {
                "in": {
                    "bytes": 6644.4,
                    "bytes_per_sec": 22.148,
                    "packets": 44.6,
                    "packets_per_sec": 0.14866666666666667
                },
                "out": {
                    "bytes": 6016.6,
                    "bytes_per_sec": 20.055333333333333,
                    "packets": 35.4,
                    "packets_per_sec": 0.118
                }
            },
            "status": {
                "check_failed": 0,
                "check_failed_instance": 0,
                "check_failed_system": 0
            }
        },
        "tags": {
            "Name": "mysql-test",
            "created-by": "ks"
        }
    },
    "cloud": {
        "account": {
            "id": "428152502467",
            "name": "elastic-beats"
        },
        "availability_zone": "us-west-1b",
        "instance": {
            "id": "i-0516ddaca5c1d231f",
            "name": "mysql-test"
        },
        "machine": {
            "type": "t2.micro"
        },
        "provider": "aws",
        "region": "us-west-1"
    },
    "event": {
        "dataset": "aws.ec2",
        "duration": 115000,
        "module": "aws"
    },
    "host": {
        "cpu": {
            "pct": 1.166484417714406
        },
        "disk": {
            "read": {
                "bytes": 0
            }
        },
        "diskio": {
            "write": {
                "bytes": 0
            }
        },
        "id": "i-0516ddaca5c1d231f",
        "name": "mysql-test",
        "network": {
            "in": {
                "bytes": 6644.4,
                "packets": 44.6
            },
            "out": {
                "bytes": 6016.6,
                "packets": 35.4
            }
        }
    },
    "metricset": {
        "name": "ec2",
        "period": 10000
    },
    "service": {
        "type": "aws"
    }
}

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #20673 opened]

• Start Time: 2020-08-18T18:23:57.272+0000

• Duration: 66 min 31 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3538
Skipped	797
Total	4335