Cloudwatch multi group

[jsirianni]

Description of Changes

We need to merge Go 1.17 first, in order to resolve a CI failure: #421

The primary goal is to support multiple AWS cloudwatch groups with a single configuration.

• added log_groups parameter, for defining a static list
• added log_group_prefix for dynamically detecting log groups
• added configureSession method for aws session re-use, instead of creating a session for each log group

log_group_name, log_groups, and log_group_prefix can all be used together. Their values are merged into log_groups.

What is missing?

There is no upper limit to number of log groups. If we think we need to, we could add the same logic that file_input uses for managing max number of open files. This logic could be added in a follow up PR, allowing us to deliver a working implementation quickly.

Please check that the PR fulfills these requirements

• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)
• Add a changelog entry (for non-trivial bug fixes / features)
• CI passes

[djaglowski]

Log Files	Logs / Second	CPU Avg (%)	CPU Avg Δ (%)	Memory Avg (MB)	Memory Avg Δ (MB)
1	1000	1.413788	-0.12070298	129.13173	+0.3659668
1	5000	4.620734	-0.5173459	135.06789	-3.0447235
1	10000	10.310679	+0.13789558	144.95111	+0.10412598
1	50000	50.030502	-5.0201187	175.28987	+3.1559753
1	100000	90.2087	-19.44905	238.01051	-10.789993
10	100	2.3276343	+0.60349524	134.7531	+1.3655701
10	500	6.017297	-0.017197132	142.10062	+3.8851013
10	1000	12.017414	-1.5864506	148.00566	+1.0576477
10	5000	57.796642	+2.4174461	184.17754	+5.3069763
10	10000	108.61167	-10.240814	229.93158	-6.9776306

[djaglowski]

Log Files	Logs / Second	CPU Avg (%)	CPU Avg Δ (%)	Memory Avg (MB)	Memory Avg Δ (MB)
1	1000	1.3793107	-0.15518022	128.10332	-0.66244507
1	5000	5.1207447	-0.017335415	135.5532	-2.5594025
1	10000	10.396961	+0.22417736	144.12985	-0.71713257
1	50000	51.879944	-3.1706772	172.01347	-0.12042236
1	100000	99.104805	-10.552948	238.10965	-10.690857
10	100	2.2070158	+0.48287666	133.49596	+0.108428955
10	500	7.6381836	+1.6036897	137.01212	-1.2033997
10	1000	10.793429	-2.8104353	145.1972	-1.7508087
10	5000	56.61498	+1.2357826	184.34698	+5.476425
10	10000	107.70534	-11.147148	223.13658	-13.772629

[djaglowski]

Log Files	Logs / Second	CPU Avg (%)	CPU Avg Δ (%)	Memory Avg (MB)	Memory Avg Δ (MB)
1	1000	1.5517452	-0.0344913	124.18965	-4.701782
1	5000	5.1897397	+0.20685291	137.06775	+0.431427
1	10000	9.8622	+0.4137783	142.05212	-1.4150085
1	50000	51.328148	+2.9469643	178.94492	+6.323288
1	100000	91.69764	+4.6788025	234.52762	+6.5305786
10	100	1.8621787	-0.4137007	132.22037	+1.9546051
10	500	6.5348544	+0.24166632	139.25511	+0.17564392
10	1000	11.086456	+0.10360241	142.8517	-3.9031525
10	5000	47.931305	-3.7928848	180.04445	-0.35128784
10	10000	98.73759	+0	215.47818	+0