Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OTelProtoCodec for InstrumentationLibrary to InstrumentationScope rename #2114

Merged
merged 3 commits into from
Jan 4, 2023

Conversation

engechas
Copy link
Collaborator

@engechas engechas commented Jan 3, 2023

Description

The InstrumentationLibrary model was renamed to InstrumentationScope as part of this PR: open-telemetry/opentelemetry-proto#362

This PR updates the OTelProtoDecoder and OTelProtoEncoder to use the new InstrumentationScope model while continuing to support incoming requests using InstrumentationLibrary per the specification defined in the above PR.

Check List

  • New functionality includes testing.
  • [N/A] New functionality has been documented.
    • [N/A] New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
Signed-off-by: Chase Engelbrecht <engechas@amazon.com>
@engechas engechas requested a review from a team as a code owner January 3, 2023 17:29
@codecov-commenter
Copy link

Codecov Report

Merging #2114 (9000505) into main (49acfb9) will increase coverage by 0.19%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #2114      +/-   ##
============================================
+ Coverage     93.76%   93.95%   +0.19%     
- Complexity     1630     1633       +3     
============================================
  Files           204      204              
  Lines          4729     4729              
  Branches        378      378              
============================================
+ Hits           4434     4443       +9     
+ Misses          207      196      -11     
- Partials         88       90       +2     
Impacted Files Coverage Δ
...opensearch/dataprepper/pipeline/ProcessWorker.java 88.46% <0.00%> (+17.30%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@engechas
Copy link
Collaborator Author

engechas commented Jan 3, 2023

@dlvenable @chenqi0805 Could you guys take a look at this PR and verify that the behavior is correct for how DataPrepper should handle the OTel model change?

Copy link
Collaborator

@chenqi0805 chenqi0805 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me. Have you tried on the examples/dev/trace-analytics-sample-app by bumping otel-collector?

@@ -77,6 +78,9 @@ public class OTelProtoCodecTest {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private static final Random RANDOM = new Random();
private static final String TEST_REQUEST_JSON_FILE = "test-request.json";
private static final String TEST_REQUEST_INSTRUMENTATION_LIBRARY_JSON_FILE = "test-request-instrumentation-library.json";
private static final String TEST_REQUEST_BOTH_SPAN_TYPES_JSON_FILE = "test-request-both-span-types.json";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this mixture a possible payload scenario from otel-collector? If not, TEST_REQUEST_JSON_FILE(with proper renaming) and TEST_REQUEST_INSTRUMENTATION_LIBRARY_JSON_FILE are sufficient for coverage.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to OTEL's comment, clients can double publish

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried on the examples/dev/trace-analytics-sample-app

I tested by spinning up a trace pipeline manually with stdout as the sink and verified the data made it through

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used tracegen to push load with Otel collector: image: otel/opentelemetry-collector-contrib:latest

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can confirm that the dev/trace-analytics-sample-app is working now (with additional change to the Data Prepper Version constant). That example also works with a recent OTEL collector version (0.68.0).

Copy link
Collaborator

@chenqi0805 chenqi0805 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build failure in JDK 17 is from a different place.

@dlvenable dlvenable linked an issue Jan 4, 2023 that may be closed by this pull request
Copy link
Member

@dlvenable dlvenable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for fixing this! I created #2115 to track the change for our upcoming 2.1 release.

@engechas engechas merged commit 988014a into opensearch-project:main Jan 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support OTel scope_logs
5 participants