Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CM] Send error from beats back to Kibana #9382

Closed
6 of 8 tasks
ph opened this issue Dec 4, 2018 · 0 comments · Fixed by #9865
Closed
6 of 8 tasks

[CM] Send error from beats back to Kibana #9382

ph opened this issue Dec 4, 2018 · 0 comments · Fixed by #9865
Assignees
@ph
Labels
libbeat Management

Comments

@ph
Copy link
Contributor

ph commented Dec 4, 2018 •
edited
Loading

In the current UI form we only display a boolean flag next to the beats that defines if the beats has the configuration applied or not. Beats is a complex agent and this is not enough to provide feedback to the user, we want to introduce in Kibana a STATE field instead that will represent the following concept:

  • Gray: PENDING, no configuration are currently link to that beat.
  • Green: RUNNING, current configuration are correctly applied to the beat.
  • Yellow: IN_PROGRESS, a new config is scheduled to be applied to a beat.
  • Red: FAILED, the configs were not applied to the beats successfully.

Beats requirements/tasks:

  • Aggregate error and keep track from which config block the error is originating
  • Serialize the errors into a JSON structure
  • Push the events back to Kibana
  • Error handling, make sure the error for the current config is send back to kibana, so keep track if the error were reported back to kibana.
  • Each set of config block need a UUID or a checksum.
  • Make sure we keep call back to kibana at a minimun.
  • Its a beta feature, do we need to think about the upgrade path from 6.5.x to 6.6

Nice to have:

  • Serialize field error from ucfg to be machine readable.

CC @mattapperson @monicasarbu
@ph ph self-assigned this Dec 4, 2018
@ph ph mentioned this issue Jan 4, 2019
Merged
4 tasks
ph added a commit to ph/beats that referenced this issue Feb 4, 2019
@ph
Allow Central management to send errors or events back to the Kibana …
1641b86 
…backend

Motivation: Currently CM allow send on the next tick a ConfigOK => true if the previous config was
successfully applied.

This PR do the following thing:

1. Add a AuthClient that wraps a Client but make sure that all calls are authenticated using the defined
UUID and the AccessToken.

2. Add an EventsReporter as part of the `api` package that will send asynchrously api.Event to the CM
backend. The reporter is started and closed at the same time of the Manager, on close events are flushed
to the backend.

3. Events are send in batch of a 1000 to the Kibana endpoint. (configurable)

5. Any errors from `BlackList` or `Reload` are send as Event.

6. Introduce Manager's State with the following: STARTING, IN_PROGRESS, RUNNING, FAILED, STOPPED and will
be send to the event endpoints.

7. Events are send by default every 30 sec. (configurable)

PATH: /api/beats/{UUID}/events exist
Verb used: "POST"

Request example:
```json
[
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STARTING"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: IN_PROGRESS"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: FAILED"}},
{"type": "ERROR", "event": { "type": "CONFIG", "message": "a bad error", "uuid": "xxxxxxxxxxxxx"}} # uuid could be omnitted
{"type": "ERROR", "event": { "type": "CONFIG", "message": "another bad error", "uuid": "xxxxxxxxxxxxx"}}
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STOPPED"}},
]
```

Expected response:

```json
{
"response": [
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
]
}
```
- [ ] Add the UUID from the API, waiting on api implementation to do that / review.

Fixes: elastic#9382
@ph ph closed this as completed in #9865 Feb 4, 2019
ph added a commit that referenced this issue Feb 4, 2019
@ph
[CM] Allow Central management to send errors or events back to the Ki…
16fb12e 
…bana backend (#9865)

* Allow Central management to send errors or events back to the Kibana backend

Motivation: Currently CM allow send on the next tick a ConfigOK => true if the previous config was
successfully applied.

This PR do the following thing:

1. Add a AuthClient that wraps a Client but make sure that all calls are authenticated using the defined
UUID and the AccessToken.

2. Add an EventsReporter as part of the `api` package that will send asynchrously api.Event to the CM
backend. The reporter is started and closed at the same time of the Manager, on close events are flushed
to the backend.

3. Events are send in batch of a 1000 to the Kibana endpoint. (configurable)

5. Any errors from `BlackList` or `Reload` are send as Event.

6. Introduce Manager's State with the following: STARTING, IN_PROGRESS, RUNNING, FAILED, STOPPED and will
be send to the event endpoints.

7. Events are send by default every 30 sec. (configurable)

PATH: /api/beats/{UUID}/events exist
Verb used: "POST"

Request example:
```json
[
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STARTING"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: IN_PROGRESS"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: FAILED"}},
{"type": "ERROR", "event": { "type": "CONFIG", "message": "a bad error", "uuid": "xxxxxxxxxxxxx"}} # uuid could be omnitted
{"type": "ERROR", "event": { "type": "CONFIG", "message": "another bad error", "uuid": "xxxxxxxxxxxxx"}}
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STOPPED"}},
]
```

Expected response:

```json
{
"response": [
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
]
}
```


Fixes: #9382
ph added a commit to ph/beats that referenced this issue Feb 4, 2019
@ph
[CM] Allow Central management to send errors or events back to the Ki…
2800b04 
…bana backend (elastic#9865)

* Allow Central management to send errors or events back to the Kibana backend

Motivation: Currently CM allow send on the next tick a ConfigOK => true if the previous config was
successfully applied.

This PR do the following thing:

1. Add a AuthClient that wraps a Client but make sure that all calls are authenticated using the defined
UUID and the AccessToken.

2. Add an EventsReporter as part of the `api` package that will send asynchrously api.Event to the CM
backend. The reporter is started and closed at the same time of the Manager, on close events are flushed
to the backend.

3. Events are send in batch of a 1000 to the Kibana endpoint. (configurable)

5. Any errors from `BlackList` or `Reload` are send as Event.

6. Introduce Manager's State with the following: STARTING, IN_PROGRESS, RUNNING, FAILED, STOPPED and will
be send to the event endpoints.

7. Events are send by default every 30 sec. (configurable)

PATH: /api/beats/{UUID}/events exist
Verb used: "POST"

Request example:
```json
[
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STARTING"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: IN_PROGRESS"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: FAILED"}},
{"type": "ERROR", "event": { "type": "CONFIG", "message": "a bad error", "uuid": "xxxxxxxxxxxxx"}} # uuid could be omnitted
{"type": "ERROR", "event": { "type": "CONFIG", "message": "another bad error", "uuid": "xxxxxxxxxxxxx"}}
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STOPPED"}},
]
```

Expected response:

```json
{
"response": [
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
]
}
```


Fixes: elastic#9382
(cherry picked from commit 16fb12e)
@ph ph mentioned this issue Feb 4, 2019
Merged
ph added a commit that referenced this issue Feb 5, 2019
@ph
Cherry-pick #9865 to 6.x: [CM] Allow Central management to send error…
bd2b994 
…s or events back to the Kibana backend (#10548)

* [CM] Allow Central management to send errors or events back to the Kibana backend (#9865)

* Allow Central management to send errors or events back to the Kibana backend

Motivation: Currently CM allow send on the next tick a ConfigOK => true if the previous config was
successfully applied.

This PR do the following thing:

1. Add a AuthClient that wraps a Client but make sure that all calls are authenticated using the defined
UUID and the AccessToken.

2. Add an EventsReporter as part of the `api` package that will send asynchrously api.Event to the CM
backend. The reporter is started and closed at the same time of the Manager, on close events are flushed
to the backend.

3. Events are send in batch of a 1000 to the Kibana endpoint. (configurable)

5. Any errors from `BlackList` or `Reload` are send as Event.

6. Introduce Manager's State with the following: STARTING, IN_PROGRESS, RUNNING, FAILED, STOPPED and will
be send to the event endpoints.

7. Events are send by default every 30 sec. (configurable)

PATH: /api/beats/{UUID}/events exist
Verb used: "POST"

Request example:
```json
[
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STARTING"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: IN_PROGRESS"}},
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: FAILED"}},
{"type": "ERROR", "event": { "type": "CONFIG", "message": "a bad error", "uuid": "xxxxxxxxxxxxx"}} # uuid could be omnitted
{"type": "ERROR", "event": { "type": "CONFIG", "message": "another bad error", "uuid": "xxxxxxxxxxxxx"}}
{"type": "STATE", "event": { "type": "STARTING", "message": "State change: STOPPED"}},
]
```

Expected response:

```json
{
"response": [
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
  { "success": true }
]
}
```


Fixes: #9382
(cherry picked from commit 16fb12e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ph ph

Labels
libbeat Management
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[CM] Allow Central management to send errors or events back to the Kibana backend ph/beats
1 participant
@ph