[Pull-based Ingestion] Introduce the new pull-based ingestion engine, APIs, and Kafka plugin

[yupeng9]

Description

This PR implements the basics of the pull-based ingestion described in this RFC, including:

1. The APIs for the pull-based ingestion source
2. A Kafka plugin that implements the ingestion source API
3. A new IngestionEngine that pulls data from the ingestion sources

Currently WIP, and there are a few improvements to make and test coverage to increase

Related Issues

Resolves #16927 #16929 #16928

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 16dd9d0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[Bukhtawar]

Curious how would the FGAC security model work, espl with security plugin which intercepts transport actions to validate if authorised users can perform bulk actions on certain indices. Is the intent to handle permissions at a Kafka "partition level"
Another aspect is maintaining Kafka checkpoints durably, I'm yet to read that part but would be good to understand how are we handling fail overs and recoveries

[andrross]

@mch2 @msfroh @yupeng9 I just pushed a rebase to pull in the commit that fixes the failure related to spotless in the precommit check. Hopefully everything should pass now...

[github-actions]

❕ Gradle check result for acb627e: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.