[Agents]: Error logs for linux and mac agent on adding/renaming integrations and restarting agent.

[amolnater-qasource]

Kibana version: 8.3 BC2 Kibana cloud environment

Host OS and Browser version: Ubuntu 20 and MAC , All

Build details:

VERSION: 8.3.0
BUILD: 53231
COMMIT: 25476b531ba9f32292bde85508d342aa5e1c29eb
Artifact Link: https://staging.elastic.co/8.3.0-fcd43da7/downloads/beats/elastic-agent/elastic-agent-8.3.0-darwin-x86_64.tar.gz

Preconditions:

1. 8.3 BC2 Kibana cloud environment should be available.
2. Linux and MAC agent must be installed using policy having System integration.

Steps to reproduce:

1. Login to Kibana environment.
2. Update name for system integration
3. Observe error logs under Logs tab.

Impact:
No other impact on agent behaviour is observed.

Screenshot:

Expected Result:
Error logs should not be available on changing system integration name.

Logs:
elastic-agent-diagnostics-2022-06-07T11-47-45Z-00.zip

[amolnater-qasource]

@manishgupta-qasource Please review.

[manishgupta-qasource]

Secondary review for this ticket is Done

[amolnater-qasource]

Hi @jlind23
While testing on 8.3 BC5 Kibana cloud environment, we have observed this issue still reproducible.

• On adding endpoint security to the policy we have observed these error logs for MAC and linux agents.

Build details:
BUILD: 53457
COMMIT: ba70bdf9fe5d28e832c40b7dbdade1e582133af1

Logs:
elastic-agent-diagnostics-2022-06-20T06-19-07Z-00.zip

Screenshot:

Thanks

[jlind23]

@kvch @fearful-symmetry could you help us out here as the panic seems to occur in the Elastic Agent lib.

[fearful-symmetry]

So, the actual panic is happening here: https://github.com/elastic/beats/blob/667266f141e63c32b42f97e6f5d16be05638b0e6/metricbeat/beater/metricbeat.go#L276

The part in the agent lib is just the helper function that's calling that Close() on a sigint. As far as I can tell, aside from the code being migrated to elastic-agent-lib, I don't see any sign that this code has changed recently. I'm guessing this is a change somewhere else impacting how metricbeat is being shut down by agent itself. Is this present on 8.2?

[amolnater-qasource]

Hi @fearful-symmetry
We have revalidated this issue on 8.2.3 released kibana cloud build and found it not reproducible there.

• No error logs were observed on adding Endpoint Security or renaming system integration for Linux and MAC agent on 8.2.3.

Screenshot:

Build details:
BUILD: 52239
COMMIT: b6ed351db30239b669b6f4950d70d1d1bd9253c5

Please let us know if anything else is required from our end.
Thanks

[amolnater-qasource]

Hi @jlind23
We have revalidated this issue on latest 8.4 BC6 Kibana cloud environment and found this issue still reproducible.

Observations:

• Error logs are available under Logs tab for MAC and Linux agent on adding Endpoint Security or an agent restart.

Build details:
BUILD: 55395
COMMIT: 1922ab0b6f8dc2906ef10f3fb65fc4d0e0076f90

Logs:
MAC:
elastic-agent-diagnostics-MAC.zip
Linux:
elastic-agent-diagnostics-Linux1.zip
elastic-agent-diagnostics-Linux2.zip

Please let us know if anything else is required from our end.
Thanks

[jlind23]

@pierrehilbert could you please triage it?

[pierrehilbert]

It seems to be related to:
elastic/beats#32608 (comment)

[jlind23]

@amolnater-qasource is this still a problem you are facing? As Pierre said this should have been fixed already: #522 (comment)

[jlind23]

Closing for now and I'll let you reopen if needed

[amolnater-qasource]

Hi Team,
We have revalidated this issue on latest 8.8.0 SNAPSHOT and found it fixed now.

Observations:

• No above reported error logs are observed for linux agent on adding/renaming integrations and restarting agent.

Build details:
VERSION: 8.8.0-SNAPSHOT
BUILD: 61970
COMMIT: 5d5e5d3c913fe7a23fb53b79acaaff75c90d2c88

Screenshot:

Hence we are marking this issue as QA:Validated.

Thanks