Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix illegal reflective access warning from protostuff #373

Merged
merged 1 commit into from
Feb 8, 2022

Conversation

kaituo
Copy link
Collaborator

@kaituo kaituo commented Feb 7, 2022

Signed-off-by: Kaituo Li kaituo@amazon.com

Description

When using prostuff 1.7.4 under java9+ we get an Illegal reflective access warning due to protostuff 1.7.4's incompatibility with java9+. The issue is solved in protostuff/protostuff@b6ad96f and will be available in 1.8.0. Since protostuff 1.8.0 is not released, this PR built 1.8.0 packages and made AD depend on the local packages directly. With the change, the illegal reflective access is gone.

Testing done:

  1. Tested both single-stream and HCAD real-time detectors.
  2. Tested backward compatibility: start a 1.2.0 cluster with protostuff 1.7.4, start detectors and wait for model checkpoints to be generated. Reinstall AD plugin with locally packaged protostuff 1.8.0. Verified tha old model can come up and detectors can produce results.

Issues Resolved

#263
#319

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kaituo kaituo requested review from a team, ylwu-amzn and amitgalitz February 7, 2022 22:38
@kaituo kaituo force-pushed the protostuff branch 2 times, most recently from 424609b to 9891c64 Compare February 7, 2022 22:45
When using prostuff 1.7.4 under java9+ we get an Illegal reflective access warning due to protostuff 1.7.4's incompatibility with java9+.  The issue is solved in protostuff/protostuff@b6ad96f  and will be available in 1.8.0. Since  protostuff 1.8.0 is not released, this PR built 1.8.0 packages and made AD depend on the local packages directly. With the change, the illegal reflective access is gone.

Testing done:
1. Tested both single-stream and HCAD real-time detectors.
2. Tested backward compatibility: start a 1.2.0 cluster with protostuff 1.7.4, start detectors and wait for model checkpoints to be generated.  Reinstall AD plugin with locally packaged protostuff 1.8.0. Verified tha old model can come up and detectors can produce results.

Signed-off-by: Kaituo Li <kaituo@amazon.com>
@@ -582,8 +582,10 @@ dependencies {
compile files('lib/randomcutforest-core-2.0.1.jar')

// used for serializing/deserializing rcf models.
compile group: 'io.protostuff', name: 'protostuff-core', version: '1.7.4'
compile group: 'io.protostuff', name: 'protostuff-runtime', version: '1.7.4'
compile files('lib/protostuff-api-1.8.0-SNAPSHOT.jar')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know when protostuff team plan to publish 1.8.0 to maven? We can move to public maven once it's ready.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't. asked in their github repo, noboy replied.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find a link for the ask, opened protostuff/protostuff#327.

Copy link
Collaborator

@ylwu-amzn ylwu-amzn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the change. Just a minor comment.

Copy link
Member

@amitgalitz amitgalitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for making this change

@amitgalitz
Copy link
Member

Just realized this package was compiled under JRE 11. Meaning code can no longer be run after compiling AD with anything below (JRE 8). Is this an issue?

opensearch-trigger-bot bot pushed a commit that referenced this pull request Mar 2, 2022
When using prostuff 1.7.4 under java9+ we get an Illegal reflective access warning due to protostuff 1.7.4's incompatibility with java9+.  The issue is solved in protostuff/protostuff@b6ad96f  and will be available in 1.8.0. Since  protostuff 1.8.0 is not released, this PR built 1.8.0 packages and made AD depend on the local packages directly. With the change, the illegal reflective access is gone.

Testing done:
1. Tested both single-stream and HCAD real-time detectors.
2. Tested backward compatibility: start a 1.2.0 cluster with protostuff 1.7.4, start detectors and wait for model checkpoints to be generated.  Reinstall AD plugin with locally packaged protostuff 1.8.0. Verified tha old model can come up and detectors can produce results.

Signed-off-by: Kaituo Li <kaituo@amazon.com>
(cherry picked from commit 9315233)
ohltyler pushed a commit that referenced this pull request Mar 2, 2022
When using prostuff 1.7.4 under java9+ we get an Illegal reflective access warning due to protostuff 1.7.4's incompatibility with java9+.  The issue is solved in protostuff/protostuff@b6ad96f  and will be available in 1.8.0. Since  protostuff 1.8.0 is not released, this PR built 1.8.0 packages and made AD depend on the local packages directly. With the change, the illegal reflective access is gone.

Testing done:
1. Tested both single-stream and HCAD real-time detectors.
2. Tested backward compatibility: start a 1.2.0 cluster with protostuff 1.7.4, start detectors and wait for model checkpoints to be generated.  Reinstall AD plugin with locally packaged protostuff 1.8.0. Verified tha old model can come up and detectors can produce results.

Signed-off-by: Kaituo Li <kaituo@amazon.com>
(cherry picked from commit 9315233)
@ohltyler ohltyler mentioned this pull request Mar 9, 2022
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants