[Usage collection] Expose the new es client and SO client

[TinaHeiligers]

Summary
Handles step 3 in #74840 (comment).

As part of migrating over to the new Elasticsearch client, we provided the new client as an additional parameter to the fetch method in collectors and pass the client down as an additional collection config parameter.

To use the new client forces plugin owners to provide the old client as well, which is not ideal.

This work refactors the way in which we provide the clients from individual arguments to an object, allowing plugins to select the client they wish to use.

This is a breaking change for Kibana developers and requires refactoring collectors in all the plugins that currently use callCluster.

Checklist

Delete any items that are not applicable to this PR.

• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios

For maintainers

• This was checked for breaking API changes and was labeled appropriately
  [skip-ci]

[kibanamachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request
• Commit: 6f71de6
• Pipeline Steps (look for red circles / failed steps)
• Interpreting CI Failures

Failed CI Steps

• Check Types
• Check Doc API Changes
• X-Pack Jest Unit Tests
• Execute xpack-kibana-ciGroup6
• Execute xpack-kibana-ciGroup6
• API Integration Tests
• Jest Unit Tests

---

Test Failures

Jest Tests.src/plugins/usage_collection/server/collector.CollectorSet registers a collector set and runs lifecycle events should log debug status of fetching from the collector

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: collectors.values(...) is not a function or its return value is not iterable
    at CollectorSet.bulkFetch (/dev/shm/workspace/parallel/5/kibana/src/plugins/usage_collection/server/collector/collector_set.ts:130:22)
    at Object.it (/dev/shm/workspace/parallel/5/kibana/src/plugins/usage_collection/server/collector/collector_set.test.ts:89:39)
    at Object.asyncJestTest (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)
    at resolve (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:45:12)
    at new Promise (<anonymous>)
    at mapper (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:28:19)
    at promise.then (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:75:41)
    at process._tickCallback (internal/process/next_tick.js:68:7)

---

Jest Tests.src/plugins/usage_collection/server/collector.CollectorSet registers a collector set and runs lifecycle events should gracefully handle a collector fetch method throwing an error

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

Error: expect(received).toStrictEqual(expected) // deep equality

Expected: []
Received: undefined
    at Object.it (/dev/shm/workspace/parallel/5/kibana/src/plugins/usage_collection/server/collector/collector_set.test.ts:119:22)
    at process._tickCallback (internal/process/next_tick.js:68:7)

---

Jest Tests.src/plugins/usage_collection/server/collector.CollectorSet registers a collector set and runs lifecycle events should not break if isReady is not a function

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: collectors.values(...) is not a function or its return value is not iterable
    at CollectorSet.bulkFetch (/dev/shm/workspace/parallel/5/kibana/src/plugins/usage_collection/server/collector/collector_set.ts:130:22)
    at Object.it (/dev/shm/workspace/parallel/5/kibana/src/plugins/usage_collection/server/collector/collector_set.test.ts:132:39)
    at Object.asyncJestTest (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:100:37)
    at resolve (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:45:12)
    at new Promise (<anonymous>)
    at mapper (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:28:19)
    at promise.then (/dev/shm/workspace/kibana/node_modules/jest-jasmine2/build/queueRunner.js:75:41)
    at process._tickCallback (internal/process/next_tick.js:68:7)

---

and 6 more failures, only showing the first 3.

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[TinaHeiligers]

Closing to revisit using a different strategy.