[2.x]Changes in DataSourceService and DataSourceMetadataStorage interface

[vamsimanohar]

Description

Definitions

• Datasource : Datasource class holds basic datasource info(name, connector), storage engine and other objects created specific to a datasource.
• DatasourceMetadata: DatasourceMetadata class carries all the metadata required for a datasource.
• DatasourceService: This class acts as interface for datasources and datasource metadata management.
• DatasourceMetadataStorageService: This class acts as interface for datasource metadata storage.

usecases

• CRUD APIs for datasources.
• Analyzer class
  • Fully Qualified Name Resolution requires names of all datasources. [Queries GetAllDatasourceMetadata]
  • DataSourceTable class requires metadata(name, connector) of all datasources for show datasources command. [Queries GetAllDatasourceMetadata]
  • Requires storage engine to implement and execute queries. [Queries GetAllDatasourceMetadata]

code flow for each usecase.

Datasource bootstrapping during plugin startup [Internal usecase for storage engine creation]

Create datasource [REST API]

Get datasource along with storage engine and other artifacts [Internal use case only]

Get datasource [REST API]

Delete datasource [REST API]

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #1391 (61343b8) into 2.x (a32bcbe) will increase coverage by 0.00%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff            @@
##                2.x    #1391   +/-   ##
=========================================
  Coverage     98.38%   98.38%           
- Complexity     3698     3699    +1     
=========================================
  Files           345      345           
  Lines          9113     9118    +5     
  Branches        585      585           
=========================================
+ Hits           8966     8971    +5     
  Misses          142      142           
  Partials          5        5           

Flag	Coverage Δ
sql-engine	98.38% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...ain/java/org/opensearch/sql/analysis/Analyzer.java	100.00% <100.00%> (ø)
...ensearch/sql/datasource/DataSourceServiceImpl.java	100.00% <100.00%> (ø)
...anner/physical/datasource/DataSourceTableScan.java	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[penghuo]

thanks for the change!
Could you merge to main first? Then backport to 2.x.

[dai-chen]

Thanks for the refactoring! Could you add the detailed design in PR description to a dev doc under https://github.com/opensearch-project/sql/blob/main/docs/dev/index.md#data-sources ?

[vamsimanohar]

#1414
Closing in favour of this backport PR.