Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenSearch migration for JDBC driver #3

Merged
merged 6 commits into from
Apr 14, 2021
Merged

OpenSearch migration for JDBC driver #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
702ee44
Rename elasticsearch
dai-chen Apr 8, 2021
6789dce
Change jdbc connection url in comparison IT
dai-chen Apr 8, 2021
2299594
Bump plugin version to 1.15
dai-chen Apr 8, 2021
1669165
Rename elasticsearch
dai-chen Apr 8, 2021
cbcb0db
Rename more
dai-chen Apr 12, 2021
e4454d2
Rename more
dai-chen Apr 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...st/src/test/java/com/amazon/opendistroforelasticsearch/sql/correctness/CorrectnessIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ private DBConnection getESConnection(TestConfig config) {
} else {
client = RestClient.builder(HttpHost.create(esHost)).build();
}
return new ESConnection("jdbc:elasticsearch://" + esHost, client);
return new ESConnection("jdbc:opensearch://" + esHost, client);
}

/**
Expand Down
2 changes: 1 addition & 1 deletion ...st/java/com/amazon/opendistroforelasticsearch/sql/correctness/tests/ESConnectionTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ public class ESConnectionTest {

@Before
public void setUp() throws IOException {
conn = new ESConnection("jdbc:elasticsearch://localhost:12345", client);
conn = new ESConnection("jdbc:opensearch://localhost:12345", client);

Response response = mock(Response.class);
when(client.performRequest(any(Request.class))).thenReturn(response);
Expand Down
2 changes: 1 addition & 1 deletion ...test/src/test/java/com/amazon/opendistroforelasticsearch/sql/sql/CorrectnessTestBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ protected void verify(String... queries) {
*/
private DBConnection getESConnection() {
String esHost = client().getNodes().get(0).getHost().toString();
return new ESConnection("jdbc:elasticsearch://" + esHost, client());
return new ESConnection("jdbc:opensearch://" + esHost, client());
}

/**
Expand Down
68 changes: 34 additions & 34 deletions sql-jdbc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ Java application that needs to use it.

If using with JDBC compatible BI tools, refer to the tool documentation on configuring a new JDBC driver. Typically,
all that's required is to make the tool aware of the location of the driver jar and then use it to setup database (i.e
Elasticsearch) connections.
OpenSearch) connections.

### Connection URL and other settings

To setup a connection, the driver requires a JDBC connection URL. The connection URL is of the form:
```
jdbc:elasticsearch://[scheme://][host][:port][/context-path]?[property-key=value]&[property-key2=value2]..&[property-keyN=valueN]
jdbc:opensearch://[scheme://][host][:port][/context-path]?[property-key=value]&[property-key2=value2]..&[property-keyN=valueN]
```


Expand Down Expand Up @@ -58,11 +58,11 @@ To setup a connection, the driver requires a JDBC connection URL. The connection
| ------------- |-------------| -----|---------|
| user | Connection username. mandatory if `auth` property selects a authentication scheme that mandates a username value | any string | `null` |
| password | Connection password. mandatory if `auth` property selects a authentication scheme that mandates a password value | any string | `null` |
| fetchSize | Cursor page size | positive integer value. Max value is limited by `index.max_result_window` Elasticsearch setting | `0` (for non-paginated response) |
| fetchSize | Cursor page size | positive integer value. Max value is limited by `index.max_result_window` OpenSearch setting | `0` (for non-paginated response) |
| logOutput | location where driver logs should be emitted | a valid file path | `null` (logs are disabled) |
| logLevel | severity level for which driver logs should be emitted | in order from highest(least logging) to lowest(most logging): OFF, FATAL, ERROR, WARN, INFO, DEBUG, TRACE, ALL | OFF (logs are disabled) |
| auth | authentication mechanism to use | `NONE` (no auth), `BASIC` (HTTP Basic), `AWS_SIGV4` (AWS SIGV4) | `basic` if username and/or password is specified, `NONE` otherwise |
| awsCredentialsProvider | The AWS credential provider to be used when authentication mechanism is `AWS_SIGV4` (AWS SIGV4). If not set, the driver will use DefaultAWSCredentialsProviderChain to sign the request. Note that the driver renamed the namespaces of its dependencies, so the value has to be an instance of com.amazonaws.opendistro.elasticsearch.sql.jdbc.shadow.com.amazonaws.auth.AWSCredentialsProvider| Instance of an AWSCredentialProvider | DefaultAWSCredentialsProviderChain |
| awsCredentialsProvider | The AWS credential provider to be used when authentication mechanism is `AWS_SIGV4` (AWS SIGV4). If not set, the driver will use DefaultAWSCredentialsProviderChain to sign the request. Note that the driver renamed the namespaces of its dependencies, so the value has to be an instance of com.amazonaws.opendistro.opensearch.sql.jdbc.shadow.com.amazonaws.auth.AWSCredentialsProvider| Instance of an AWSCredentialProvider | DefaultAWSCredentialsProviderChain |
| region | if authentication type is `aws_sigv4`, then this is the region value to use when signing requests. Only needed if the driver can not determine the region for the host endpoint. The driver will detect the region if the host endpoint matches a known url pattern. | a valid AWS region value e.g. us-east-1 | `null` (auto-detected if possible from the host endpoint) |
| requestCompression | whether to indicate acceptance of compressed (gzip) responses when making server requests | `true` or `false` | `false` |
| useSSL | whether to establish the connection over SSL/TLS | `true` or `false` | `false` if scheme is `http`, `true` if scheme is `https` |
Expand All @@ -89,7 +89,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://localhost:9200";
String url = "jdbc:opensearch://localhost:9200";

Connection con = DriverManager.getConnection(url);
Statement st = con.createStatement();
Expand All @@ -108,7 +108,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";

Connection con = DriverManager.getConnection(url);
Statement st = con.createStatement();
Expand All @@ -127,7 +127,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://remote-host-name";
String url = "jdbc:opensearch://remote-host-name";

Properties properties = new Properties();
properties.put("useSSL", "true");
Expand All @@ -149,7 +149,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";
String user = "username";
String password = "password";

Expand All @@ -170,7 +170,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://remote-host-name";
String url = "jdbc:opensearch://remote-host-name";

Properties properties = new Properties();
properties.put("useSSL", "true");
Expand All @@ -194,7 +194,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://remote-host-name";
String url = "jdbc:opensearch://remote-host-name";

Properties properties = new Properties();
properties.put("useSSL", "true");
Expand Down Expand Up @@ -224,7 +224,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name?auth=aws_sigv4";
String url = "jdbc:opensearch://https://remote-host-name?auth=aws_sigv4";

Connection con = DriverManager.getConnection(url);
Statement st = con.createStatement();
Expand All @@ -242,7 +242,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";

Properties properties = new Properties();
properties.put("auth", "aws_sigv4");
Expand All @@ -264,7 +264,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";

Properties properties = new Properties();
properties.put("awsCredentialsProvider", new EnvironmentVariableCredentialsProvider());
Expand All @@ -286,7 +286,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name?auth=aws_sigv4&region=us-west-1";
String url = "jdbc:opensearch://https://remote-host-name?auth=aws_sigv4&region=us-west-1";

Connection con = DriverManager.getConnection(url);
Statement st = con.createStatement();
Expand All @@ -305,7 +305,7 @@ import java.sql.DriverManager;
import java.sql.Statement;
.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";

Properties properties = new Properties();
properties.put("auth", "aws_sigv4");
Expand All @@ -321,7 +321,7 @@ con.close();
```
### Connecting using the DataSource interface

The driver also provides a javax.sql.DataSource implementation via the `com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource` class that can be used to obtain a connection. Here are some typical code samples:
The driver also provides a javax.sql.DataSource implementation via the `com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource` class that can be used to obtain a connection. Here are some typical code samples:


* Connect to localhost on port 9200 with no authentication over a plain connection
Expand All @@ -331,13 +331,13 @@ import java.sql.Connection;
import java.sql.Statement;
import javax.sql.DataSource;

import com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource;
import com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource;

.
.
String url = "jdbc:elasticsearch://localhost:9200";
String url = "jdbc:opensearch://localhost:9200";

ElasticsearchDataSource ds = new ElasticsearchDataSource();
OpenSearchDataSource ds = new OpenSearchDataSource();
ds.setUrl(url);

Connection con = ds.getConnection(url);
Expand All @@ -356,13 +356,13 @@ import java.sql.Connection;
import java.sql.Statement;
import javax.sql.DataSource;

import com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource;
import com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource;

.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";

ElasticsearchDataSource ds = new ElasticsearchDataSource();
OpenSearchDataSource ds = new OpenSearchDataSource();
ds.setUrl(url);

Connection con = ds.getConnection(url);
Expand All @@ -381,13 +381,13 @@ import java.sql.Connection;
import java.sql.Statement;
import javax.sql.DataSource;

import com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource;
import com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource;

.
.
String url = "jdbc:elasticsearch://https://remote-host-name";
String url = "jdbc:opensearch://https://remote-host-name";

ElasticsearchDataSource ds = new ElasticsearchDataSource();
OpenSearchDataSource ds = new OpenSearchDataSource();
ds.setUrl(url);

Connection con = ds.getConnection(url, "user", "password");
Expand All @@ -407,13 +407,13 @@ import java.sql.Connection;
import java.sql.Statement;
import javax.sql.DataSource;

import com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource;
import com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource;

.
.
String url = "jdbc:elasticsearch://https://remote-host-name?auth=aws_sigv4";
String url = "jdbc:opensearch://https://remote-host-name?auth=aws_sigv4";

ElasticsearchDataSource ds = new ElasticsearchDataSource();
OpenSearchDataSource ds = new OpenSearchDataSource();
ds.setUrl(url);

Connection con = ds.getConnection(url);
Expand All @@ -432,13 +432,13 @@ import java.sql.Connection;
import java.sql.Statement;
import javax.sql.DataSource;

import com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource;
import com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource;

.
.
String url = "jdbc:elasticsearch://https://remote-host-name?auth=aws_sigv4&region=us-west-1";
String url = "jdbc:opensearch://https://remote-host-name?auth=aws_sigv4&region=us-west-1";

ElasticsearchDataSource ds = new ElasticsearchDataSource();
OpenSearchDataSource ds = new OpenSearchDataSource();
ds.setUrl(url);
ds.setAwsCredentialProvider(new EnvironmentVariableCredentialsProvider());

Expand All @@ -458,13 +458,13 @@ import java.sql.Connection;
import java.sql.Statement;
import javax.sql.DataSource;

import com.amazon.opendistroforelasticsearch.jdbc.ElasticsearchDataSource;
import com.amazon.opendistroforelasticsearch.jdbc.OpenSearchDataSource;

.
.
String url = "jdbc:elasticsearch://https://remote-host-name?auth=aws_sigv4&region=us-west-1";
String url = "jdbc:opensearch://https://remote-host-name?auth=aws_sigv4&region=us-west-1";

ElasticsearchDataSource ds = new ElasticsearchDataSource();
OpenSearchDataSource ds = new OpenSearchDataSource();
ds.setUrl(url);

Connection con = ds.getConnection(url);
Expand Down
4 changes: 2 additions & 2 deletions sql-jdbc/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ plugins {
group 'com.amazon.opendistroforelasticsearch.client'

// keep version in sync with version in Driver source
version '1.13.0.0'
version '1.15.0.0'

boolean snapshot = "true".equals(System.getProperty("build.snapshot", "true"));
if (snapshot) {
Expand Down Expand Up @@ -71,7 +71,7 @@ tasks.withType(JavaCompile) {
}

static def getShadowPath(String path) {
return 'com.amazonaws.opendistro.elasticsearch.sql.jdbc.shadow.' + path
return 'com.amazonaws.opendistro.opensearch.sql.jdbc.shadow.' + path
}

shadowJar {
Expand Down
10 changes: 5 additions & 5 deletions sql-jdbc/docs/tableau.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ Place the `opendistro-sql-jdbc-x.x.x.x.jar` file in the folder for your operatin

### Create TDC file

TDC file is required to add customization for the data connection. For reference, see the following sample `elasticsearch.tdc` file.
TDC file is required to add customization for the data connection. For reference, see the following sample `opensearch.tdc` file.
```
<?xml version='1.0' encoding='utf-16' ?>
<connection-customization class='genericjdbc' enabled='true' version='2019.3'>
<vendor name='genericjdbc' />
<driver name='elasticsearch' />
<driver name='opensearch' />
<customizations>
<customization name='CAP_CREATE_TEMP_TABLES' value='no'/>
<customization name='CAP_SUPPRESS_DISCOVERY_QUERIES' value='yes' />
Expand All @@ -39,7 +39,7 @@ TDC file is required to add customization for the data connection. For reference
</connection-customization>
```
* Using a text editor, add `<connection-customization>` section.
* Name the file `elasticsearch.tdc` and save it to `My Tableau Repository\Datasources`.
* Name the file `opensearch.tdc` and save it to `My Tableau Repository\Datasources`.
* Restart Tableau to apply the change.

For futher details check [using a .tdc file with Tableau](https://kb.tableau.com/articles/howto/using-a-tdc-file-with-tableau-server)
Expand All @@ -49,11 +49,11 @@ For futher details check [using a .tdc file with Tableau](https://kb.tableau.com
You will need:
* [JDBC connection string](https://github.com/opendistro-for-elasticsearch/sql/blob/master/sql-jdbc/README.md#connection-url-and-other-settings) to enter in the URL field when you connect.

Sample connection string for connecting to localhost: `jdbc:elasticsearch://localhost:9200`.
Sample connection string for connecting to localhost: `jdbc:opensearch://localhost:9200`.

* Credentials for signing in to the server (user name and password).
* (Optional) JDBC properties file to customize the driver behavior. For more details check [Customize JDBC Connections Using a Properties File](https://community.tableau.com/docs/DOC-17978)
* Create a properties file called `elasticsearch.properties`.
* Create a properties file called `opensearch.properties`.
* Save the file to the `My Tableau Repository\Datasources` directory.


Expand Down
2 changes: 1 addition & 1 deletion sql-jdbc/src/main/java/com/amazon/opendistroforelasticsearch/jdbc/ConnectionImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
import java.util.Properties;
import java.util.concurrent.Executor;

public class ConnectionImpl implements ElasticsearchConnection, JdbcWrapper, LoggingSource {
public class ConnectionImpl implements OpenSearchConnection, JdbcWrapper, LoggingSource {

private String url;
private String user;
Expand Down
Loading