[7.9] Adding aggregations for endpoint events (#72705)

[jonathan-buttner]

Backports the following commits to 7.9:

• Adding aggregations for endpoint events ([Security Solution] Adding aggregations for endpoint events #72705)

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request
• Commit: 7ef0545
• Flaky suites:
  • xpack-kibana-ciGroup6

---

Test Failures

Kibana Pipeline / kibana-xpack-agent / X-Pack API Integration Tests.x-pack/test/api_integration/apis/ml/modules/get_module·ts.apis Machine Learning modules get_module "before all" hook for "lists all modules"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://github.com/elastic/kibana/issues/72381

[00:00:00]       │
[00:00:00]         └-: apis
[00:00:00]           └-> "before all" hook
[00:09:54]           └-: Machine Learning
[00:09:54]             └-> "before all" hook
[00:09:54]             └-> "before all" hook
[00:09:54]               │ debg creating role ml_source
[00:09:54]               │ info [o.e.x.s.a.r.TransportPutRoleAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added role [ml_source]
[00:09:54]               │ debg creating role ml_dest
[00:09:54]               │ info [o.e.x.s.a.r.TransportPutRoleAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added role [ml_dest]
[00:09:54]               │ debg creating role ml_dest_readonly
[00:09:54]               │ info [o.e.x.s.a.r.TransportPutRoleAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added role [ml_dest_readonly]
[00:09:54]               │ debg creating role ml_ui_extras
[00:09:54]               │ info [o.e.x.s.a.r.TransportPutRoleAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added role [ml_ui_extras]
[00:09:54]               │ debg creating user ml_poweruser
[00:09:54]               │ info [o.e.x.s.a.u.TransportPutUserAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added user [ml_poweruser]
[00:09:54]               │ debg created user ml_poweruser
[00:09:54]               │ debg creating user ml_viewer
[00:09:55]               │ info [o.e.x.s.a.u.TransportPutUserAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added user [ml_viewer]
[00:09:55]               │ debg created user ml_viewer
[00:09:55]               │ debg creating user ml_unauthorized
[00:09:55]               │ info [o.e.x.s.a.u.TransportPutUserAction] [kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308] added user [ml_unauthorized]
[00:09:55]               │ debg created user ml_unauthorized
[00:09:55]             └-: modules
[00:09:55]               └-> "before all" hook
[00:09:55]               └-: get_module
[00:09:55]                 └-> "before all" hook
[00:09:55]                 └-> "before all" hook
[00:09:55]                   │ debg applying update to kibana config: {"dateFormat:tz":"UTC"}
[00:09:55]                   └- ✖ fail: apis Machine Learning modules get_module "before all" hook for "lists all modules"
[00:09:55]                   │      Error: Request failed with status code 409
[00:09:55]                   │       at createError (/dev/shm/workspace/kibana/node_modules/axios/lib/core/createError.js:16:15)
[00:09:55]                   │       at settle (/dev/shm/workspace/kibana/node_modules/axios/lib/core/settle.js:17:12)
[00:09:55]                   │       at IncomingMessage.handleStreamEnd (/dev/shm/workspace/kibana/node_modules/axios/lib/adapters/http.js:236:11)
[00:09:55]                   │       at endReadableNT (_stream_readable.js:1145:12)
[00:09:55]                   │       at process._tickCallback (internal/process/next_tick.js:63:19)
[00:09:55]                   │ 
[00:09:55]                   │ 

Stack Trace

{ Error: Request failed with status code 409
    at createError (/dev/shm/workspace/kibana/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/dev/shm/workspace/kibana/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/dev/shm/workspace/kibana/node_modules/axios/lib/adapters/http.js:236:11)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  config:
   { url: 'http://elastic:changeme@localhost:6171/api/kibana/settings',
     method: 'post',
     data: '{"changes":{"dateFormat:tz":"UTC"}}',
     headers:
      { Accept: 'application/json, text/plain, */*',
        'Content-Type': 'application/json;charset=utf-8',
        'kbn-xsrf': 'kbn-client',
        'User-Agent': 'axios/0.19.2',
        'Content-Length': 35 },
     transformRequest: [ [Function: transformRequest] ],
     transformResponse: [ [Function: transformResponse] ],
     timeout: 0,
     adapter: [Function: httpAdapter],
     xsrfCookieName: 'XSRF-TOKEN',
     xsrfHeaderName: 'X-XSRF-TOKEN',
     maxContentLength: -1,
     validateStatus: [Function: validateStatus],
     httpsAgent: null },
  request:
   ClientRequest {
     _events:
      [Object: null prototype] {
        socket: [Function],
        abort: [Function],
        aborted: [Function],
        error: [Function],
        timeout: [Function],
        prefinish: [Function: requestOnPrefinish] },
     _eventsCount: 6,
     _maxListeners: undefined,
     output: [],
     outputEncodings: [],
     outputCallbacks: [],
     outputSize: 0,
     writable: true,
     _last: true,
     chunkedEncoding: false,
     shouldKeepAlive: false,
     useChunkedEncodingByDefault: true,
     sendDate: false,
     _removedConnection: false,
     _removedContLen: false,
     _removedTE: false,
     _contentLength: null,
     _hasBody: true,
     _trailer: '',
     finished: true,
     _headerSent: true,
     socket:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: 'localhost',
        _readableState: [ReadableState],
        readable: false,
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: null,
        _httpMessage: [Circular],
        write: [Function: writeAfterFIN],
        [Symbol(asyncId)]: 3295184,
        [Symbol(lastWriteQueueSize)]: 0,
        [Symbol(timeout)]: null,
        [Symbol(kBytesRead)]: 760,
        [Symbol(kBytesWritten)]: 318 },
     connection:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: 'localhost',
        _readableState: [ReadableState],
        readable: false,
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: null,
        _httpMessage: [Circular],
        write: [Function: writeAfterFIN],
        [Symbol(asyncId)]: 3295184,
        [Symbol(lastWriteQueueSize)]: 0,
        [Symbol(timeout)]: null,
        [Symbol(kBytesRead)]: 760,
        [Symbol(kBytesWritten)]: 318 },
     _header:
      'POST /api/kibana/settings HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nContent-Type: application/json;charset=utf-8\r\nkbn-xsrf: kbn-client\r\nUser-Agent: axios/0.19.2\r\nContent-Length: 35\r\nHost: localhost:6171\r\nAuthorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==\r\nConnection: close\r\n\r\n',
     _onPendingData: [Function: noopPendingOutput],
     agent:
      Agent {
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        defaultPort: 80,
        protocol: 'http:',
        options: [Object],
        requests: {},
        sockets: [Object],
        freeSockets: {},
        keepAliveMsecs: 1000,
        keepAlive: false,
        maxSockets: Infinity,
        maxFreeSockets: 256 },
     socketPath: undefined,
     timeout: undefined,
     method: 'POST',
     insecureHTTPParser: undefined,
     path: '/api/kibana/settings',
     _ended: true,
     res:
      IncomingMessage {
        _readableState: [ReadableState],
        readable: false,
        _events: [Object],
        _eventsCount: 3,
        _maxListeners: undefined,
        socket: [Socket],
        connection: [Socket],
        httpVersionMajor: 1,
        httpVersionMinor: 1,
        httpVersion: '1.1',
        complete: true,
        headers: [Object],
        rawHeaders: [Array],
        trailers: {},
        rawTrailers: [],
        aborted: false,
        upgrade: false,
        url: '',
        method: null,
        statusCode: 409,
        statusMessage: 'Conflict',
        client: [Socket],
        _consuming: false,
        _dumped: false,
        req: [Circular],
        responseUrl: 'http://elastic:changeme@localhost:6171/api/kibana/settings',
        redirects: [] },
     aborted: undefined,
     timeoutCb: null,
     upgradeOrConnect: false,
     parser: null,
     maxHeadersCount: null,
     _redirectable:
      Writable {
        _writableState: [WritableState],
        writable: true,
        _events: [Object],
        _eventsCount: 2,
        _maxListeners: undefined,
        _options: [Object],
        _redirectCount: 0,
        _redirects: [],
        _requestBodyLength: 35,
        _requestBodyBuffers: [],
        _onNativeResponse: [Function],
        _currentRequest: [Circular],
        _currentUrl: 'http://elastic:changeme@localhost:6171/api/kibana/settings' },
     [Symbol(isCorked)]: false,
     [Symbol(outHeadersKey)]:
      [Object: null prototype] {
        accept: [Array],
        'content-type': [Array],
        'kbn-xsrf': [Array],
        'user-agent': [Array],
        'content-length': [Array],
        host: [Array],
        authorization: [Array] } },
  response:
   { status: 409,
     statusText: 'Conflict',
     headers:
      { 'kbn-name': 'kibana-ci-immutable-ubuntu-18-tests-xl-1595432065224599308',
        'kbn-license-sig':
         '4e11ab66cb255b1c9c173ae1918a0e18a5e4d1d227d645f1bcbc6c8670a888eb',
        'kbn-xpack-sig': '654c1fa306389e96062643c895caf5ec',
        'content-type': 'application/json; charset=utf-8',
        'cache-control': 'private, no-cache, no-store, must-revalidate',
        'content-length': '348',
        date: 'Wed, 22 Jul 2020 16:36:38 GMT',
        connection: 'close' },
     config:
      { url: 'http://elastic:changeme@localhost:6171/api/kibana/settings',
        method: 'post',
        data: '{"changes":{"dateFormat:tz":"UTC"}}',
        headers: [Object],
        transformRequest: [Array],
        transformResponse: [Array],
        timeout: 0,
        adapter: [Function: httpAdapter],
        xsrfCookieName: 'XSRF-TOKEN',
        xsrfHeaderName: 'X-XSRF-TOKEN',
        maxContentLength: -1,
        validateStatus: [Function: validateStatus],
        httpsAgent: null },
     request:
      ClientRequest {
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        output: [],
        outputEncodings: [],
        outputCallbacks: [],
        outputSize: 0,
        writable: true,
        _last: true,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: true,
        sendDate: false,
        _removedConnection: false,
        _removedContLen: false,
        _removedTE: false,
        _contentLength: null,
        _hasBody: true,
        _trailer: '',
        finished: true,
        _headerSent: true,
        socket: [Socket],
        connection: [Socket],
        _header:
         'POST /api/kibana/settings HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nContent-Type: application/json;charset=utf-8\r\nkbn-xsrf: kbn-client\r\nUser-Agent: axios/0.19.2\r\nContent-Length: 35\r\nHost: localhost:6171\r\nAuthorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==\r\nConnection: close\r\n\r\n',
        _onPendingData: [Function: noopPendingOutput],
        agent: [Agent],
        socketPath: undefined,
        timeout: undefined,
        method: 'POST',
        insecureHTTPParser: undefined,
        path: '/api/kibana/settings',
        _ended: true,
        res: [IncomingMessage],
        aborted: undefined,
        timeoutCb: null,
        upgradeOrConnect: false,
        parser: null,
        maxHeadersCount: null,
        _redirectable: [Writable],
        [Symbol(isCorked)]: false,
        [Symbol(outHeadersKey)]: [Object] },
     data:
      { statusCode: 409,
        error: 'Conflict',
        message:
         '[config:7.9.0-SNAPSHOT]: version conflict, document already exists (current version [3]): [version_conflict_engine_exception] [config:7.9.0-SNAPSHOT]: version conflict, document already exists (current version [3]), with { index_uuid="K6Xdvu5BRyyp3l0YmyGdeA" & shard="0" & index=".kibana_2" }' } },
  isAxiosError: true,
  toJSON: [Function] }

---

Build metrics

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream