-
Notifications
You must be signed in to change notification settings - Fork 534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[INLONG-10704][Sort] Sort Add Oceanbase Support #10700
Conversation
...-common/src/main/java/org/apache/inlong/sort/protocol/node/extract/OceanBaseExtractNode.java
Outdated
Show resolved
Hide resolved
...ore/src/test/java/org/apache/inlong/sort/parser/MySqlExtractNodeToOceanBaseLoadNodeTest.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This factory should named OceanBaseDialectFactory?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it was found in the test that MySQL and Oceanbase both need this dialect to find the corresponding factory that conforms to the JDBC protocol
...nnectors/jdbc/src/main/java/org/apache/inlong/sort/jdbc/dialect/jdbc/JdbcDialectFactory.java
Show resolved
Hide resolved
...sort-connectors/jdbc/src/main/java/org/apache/inlong/sort/jdbc/dialect/jdbc/JdbcDialect.java
Outdated
Show resolved
Hide resolved
...sort-connectors/jdbc/src/main/java/org/apache/inlong/sort/jdbc/dialect/jdbc/JdbcDialect.java
Show resolved
Hide resolved
...sort-connectors/jdbc/src/main/java/org/apache/inlong/sort/jdbc/dialect/jdbc/JdbcDialect.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add test result with real data from any source to oceanbase
What kind of test data is needed and where to put it |
You can refer to inlong-sort/sort-end-to-end-tests/sort-end-to-end-tests-v1.15/src/test/java/org/apache/inlong/sort/tests/Mysql2StarRocksTest.java or package your code to manager container. Then start a job to verify the oceanbase connector work properly.If all right, plz |
LGTM~ |
Fixes #10704
Motivation
Added support for Oceanbase data sources to adapt to the Oceanbase ecosystem. close #10704
Modifications
It mainly adds oceanbase data source support and uses the oceanbase-client driver to connect.
It mainly adds oceanbase data source support and uses the oceanbase-client driver to connect. And write oceanbase's ExtractNode logic and LoadNode logic. The specific implementation logic is the same as mysql.Added support for oceanbase on the front-end page. Since the jdbc-inlong processing logic in the flink15 connection lacks the jdbc dialect, an error is reported. This is modified here.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework/code cleanup without any test coverage.
This change is already covered by existing tests, such as:
![image](https://private-user-images.githubusercontent.com/93303124/350850166-5fc8d69b-4536-4f55-9eb7-4bce1cdeb089.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODkzMTEsIm5iZiI6MTczOTE4OTAxMSwicGF0aCI6Ii85MzMwMzEyNC8zNTA4NTAxNjYtNWZjOGQ2OWItNDUzNi00ZjU1LTllYjctNGJjZTFjZGViMDg5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEyMDMzMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTY0ZDg3NzVlMzA5NjFhZGM0ZWE4Yzk5MzI3M2UxMDAxZjVmOWM2NzY1NGU0ZjJlN2VhYTEwYzY3YmI5MDRiY2MmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.mwIAgaUhpvF1KtRcwG7bUkX21NvM479fq5mQe8sPO3o)
![image](https://private-user-images.githubusercontent.com/93303124/350850266-23a29bd4-c33e-408c-9cfa-4bf778b1a9c3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODkzMTEsIm5iZiI6MTczOTE4OTAxMSwicGF0aCI6Ii85MzMwMzEyNC8zNTA4NTAyNjYtMjNhMjliZDQtYzMzZS00MDhjLTljZmEtNGJmNzc4YjFhOWMzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEyMDMzMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVmZGQyMDM1OGEyYzY4ODgxMDJhZjc2Y2JjNzc2OGMyMzUwNWM5OWFmMGQzZjVmYmY5OWYwOWJhMjRjYzdiOTMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.VqyZHhB_PMaJ-JX4kj7jU3xN_x2KP2AT8SFHkmxIpW4)
![image](https://private-user-images.githubusercontent.com/93303124/350850297-7fcb9917-6054-4783-9aa7-06305887d853.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODkzMTEsIm5iZiI6MTczOTE4OTAxMSwicGF0aCI6Ii85MzMwMzEyNC8zNTA4NTAyOTctN2ZjYjk5MTctNjA1NC00NzgzLTlhYTctMDYzMDU4ODdkODUzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEyMDMzMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVlMzcwYzIwMWRlNDI2MmYxYmM5NzIyZmZhZDZmOWFiMjY0Yzc0ZmVmMGRiZDJiNjQ5NjRkOWFlYjg0NmI4NjYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.VD4KMd7r3SCxrLgvquPLMrQnFESdbW9QDOWPzkTv-wc)
![image](https://private-user-images.githubusercontent.com/93303124/350850363-89468952-232a-4cc8-a1a9-aee0df308c86.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkxODkzMTEsIm5iZiI6MTczOTE4OTAxMSwicGF0aCI6Ii85MzMwMzEyNC8zNTA4NTAzNjMtODk0Njg5NTItMjMyYS00Y2M4LWExYTktYWVlMGRmMzA4Yzg2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjEwVDEyMDMzMVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTlhNDc4ZDY5ZjcxYjVkNTNjNWIyZTEyZGFhYzU2ZjJkNmMwYmNiMWMwZGZhNWE4MjIyNjY2ZWE0ODQ4ZjVmMGUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.Nc13dY5oYkIWsrgSQh-nKyffudgnii5v1r9BC6VqLas)
(please describe tests)
Here's how the page test works
This change added tests and can be verified as follows:
Please use the following test classes for testing org.apache.inlong.sort.parser.MySqlExtractNodeToOceanBaseLoadNodeTest
(example:)
Documentation