-
Notifications
You must be signed in to change notification settings - Fork 25k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support compatibility test for JDBC driver #56722
Comments
Pinging @elastic/es-core-infra (:Core/Infra/Build) |
Pinging @elastic/es-ql (:Query Languages/SQL) |
It's possible that this somewhat overlaps with what @jakelandis is doing with REST compatibility, as this is another type of forwards compatibility, although implemented a bit differently. Again, in that case (correct me if I'm wrong) newer branches grab the artifact (in this case the rest specs) from the older branch and run them against the current version. |
I think we should continue with backwards compatibility testing, for a few reasons:
|
FYI I've opened: #56872, to "isolate" the jdbc integration tests and avoid hidden bugs because of possible unnecessary pulled in dependencies, and to prepare for the support compatibility testing. |
I also agree with keeping the bwc scheme. So for example: for 7.8 we can check for each 7.7.x and 7.8.x released versions of ES nodes with the [7.7.0 -> es node version - 1]. We'd like to have something soon as new features are blocked by this (for example date_nanos support PR awaits on this). |
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for #56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project.
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for elastic#56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project. (cherry picked from commit c09f4a0)
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for elastic#56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project. (cherry picked from commit c09f4a0)
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for #56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project. (cherry picked from commit c09f4a0)
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for elastic#56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project. (cherry picked from commit c09f4a0)
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for #56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project. (cherry picked from commit c09f4a0)
Move the JDBC functionality integration tests from `:sql:qa` to a separate module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the integration tests and they only depend to the `:sql:jdbc` module, thus removing the danger of accidentally pulling in some dependency that may hide bugs. Moreover this is a preparation for #56722, so that we can run those tests between different JDBC and ES node versions and ensure forward compatibility. Move the rest of existing tests inside a new `:sql:qa:server` project, so that the `:sql:qa` becomes the parent project for both and one can run all the integration tests by using this parent project. (cherry picked from commit c09f4a0)
@mark-vieira @rjernst Can we please have an ETA for this? The JDBC tests are already split into a separate Thank you. |
I don't have a concrete ETA for you but I've thrown this on the top of the list for upcoming work. |
@matriv I presume that the set of tests that we want to perform compatibility testing with are everything under The tests in |
@mark-vieira Correct! only the |
One thing I ran into here is that it's going to be problematic to use the shadowjar as the test artifact here. To do so, we have to exclude any dependencies from the test classpath that are bundled in the JDBC driver fatjar (things like core libs, etc). The problem is, then we use the libraries from the BWC jar when testing, which causes errors like this: We are relying on the JDBC driver jar to bring in that class, but it doesn't exist in |
I'm struggling to find a good way forward here given the way these tests are currently architected. The problem is our
|
I'll add a third item:
|
Ok item (3) likely won't work since these tests rely on |
Thanks for looking into this @mark-vieira. Both (1) and (2) work for me - I have a slight preference towards (2) since, despite not being a fan of shadowing packages, medium term this might prove beneficial as embedding JDBC drivers in other Java apps that use ES, will lead to the same problem that you are facing - classpath issues. The concern that I have though is this feature will be available only moving forward so past releases (in this case 7.7) would miss out on this. However I'd rather have us acting quickly through (2) and then look into ways to address 7.7 later down the road if we really have to. So 👍 for going with (2). |
Correct, with this option we effectively cannot do compatibility testing with existing releases. We would have to start with |
Even though this is not ideal, we can assume that is safe enough to start with |
To clarify, I've not investigated what it would take to implement solution (1). I suspect someone more familiar with the tests would be better suited to do that. It might be worth looking into in parallel with this. Course (2) is not necessarily an easy win, package relocation might not easily work, as it doesn't play nicely with things like reflection. We may find out that we have to make code changes in the JDBC driver or some of it's dependencies to make that work, at which point refactoring the tests might be an easier road. |
Have you tried changing the packages and encountered failures? The JDBC driver is not using reflection nor does it depend on the package name; maybe the |
I have not yet, I'll be giving this a try soon. |
Ok, with some tinkering I was able to get a green build of the current integration test suite against a shadowjar with relocated https://gradle-enterprise.elastic.co/s/3cutzwlvii6lq/tests I'll move on to testing this against similarly configured BWC branch as well. |
Here we are folks. This is a successful run of building the 7.8.1 JDBC driver (with relocated shadowed dependencies) and testing against a 7.9 Elasticsearch cluster. https://gradle-enterprise.elastic.co/s/sqjzlft75tx42/tests Since this approach seems to be ok I'll move forward with getting the necessary changes made in the 7.8 branch and forward. I'd also highly suggest we expand the existing test suite to also validate the current JDBC driver version is what we expect. I'm going to inject a |
Alright with #60430 merged we now have this in place. We didn't make this in to 7.8 so we are beginning compatibility testing with 7.9. As mentioned above, I would highly recommend putting some assertions in place to confirm the JDBC driver being loaded is the version that's expected in case we muck up the build wiring somehow. |
Forwards compatibility was added to our JDBC driver starting in version 7.7.0 (see #56223). The compatibility guarantee being that for any given version of the driver, it should support (mostly) communication for future versions of Elasticsearch of the same major version. For example, version 7.7.0 of the driver should be able to talk to Elasticearch nodes of version 7.7.1, 7.8.0, 7.8.1, 7.9.0 and so on...
Currently, there is no automated testing in place to ensure we maintain forwards compatibility. It's also not completely clear where, or how these tests should run. Currently, the build only supports backwards compatibility. That is, testing branch code against assets or artifacts from earlier versions or branches. Building or resolving artifacts to test against from future branches currently isn't supported, since, for example, the
7.x
branch has no knowledge of the code inmaster
.We'll need to sort out a few things here, such as which direction we test in. Do we do backwards testing, where in which future branches grab the older JDBC drivers and run the tests, or forwards testing, where older branches grab the new Elasticsearch distribution and run the tests? Much of this is implementation detail, but it has some semantic impact, as with the former method, a failure in a given branch generally points to an incompatible change made in a different branch (but not necessarily). Going the latter route would require more effort, since we'd need to add support to build unreleased artifacts from future branches, which we currently don't do.
cc @matriv
The text was updated successfully, but these errors were encountered: