Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add meta_struct support on traces and spans #4334

Merged
merged 15 commits into from
Mar 10, 2025
Merged

Add meta_struct support on traces and spans #4334

merged 15 commits into from
Mar 10, 2025

Conversation

vpellan
Copy link
Contributor

@vpellan vpellan commented Jan 31, 2025

What does this PR do?

This PR adds meta_struct support to traces and spans. meta_struct is a way to send complex structures to the backend. It is sent as a Map<String, ByteArray> with the ByteArray being the structure encoded in messagepack.

Motivation:

meta_struct is required for stack trace collection, which is required for exploit prevention. But this can also be used by other products.

Change log entry

None.

Additional Notes:

How to test the change?

@github-actions github-actions bot added appsec Application Security monitoring product tracing labels Jan 31, 2025
@vpellan vpellan force-pushed the vpellan/meta-struct branch 2 times, most recently from b3dc5c2 to 0836e11 Compare January 31, 2025 12:48
@datadog-datadog-prod-us1
Copy link
Contributor

datadog-datadog-prod-us1 bot commented Jan 31, 2025 •
edited
Loading

Datadog Report

Branch report: vpellan/meta-struct
Commit report: a70a07e
Test service: dd-trace-rb

✅ 0 Failed, 20629 Passed, 1370 Skipped, 3m 16.82s Total Time

@vpellan vpellan removed the appsec Application Security monitoring product label Jan 31, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jan 31, 2025 •
edited
Loading

Benchmarks

Benchmark execution time: 2025-03-10 09:35:04

Comparing candidate commit a70a07e in PR branch vpellan/meta-struct with baseline commit f7b904d in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 31 metrics, 2 unstable metrics.

@codecov-commenter
Copy link

codecov-commenter commented Jan 31, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 93.22034% with 4 lines in your changes missing coverage. Please review.

Project coverage is 97.69%. Comparing base (f7b904d) to head (a70a07e).

Files with missing lines Patch % Lines
lib/datadog/tracing/span_operation.rb 25.00% 3 Missing ⚠️
lib/datadog/tracing/metadata/metastruct.rb 93.33% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #4334      +/-   ##
==========================================
- Coverage   97.70%   97.69%   -0.01%     
==========================================
  Files        1376     1379       +3     
  Lines       83824    83881      +57     
  Branches     4251     4251              
==========================================
+ Hits        81897    81949      +52     
- Misses       1927     1932       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vpellan vpellan force-pushed the vpellan/meta-struct branch from faf7819 to fcda7ca Compare February 3, 2025 14:27
@vpellan vpellan marked this pull request as ready for review February 3, 2025 14:30
@vpellan vpellan requested a review from a team as a code owner February 3, 2025 14:30
Strech
Strech reviewed Feb 3, 2025
View reviewed changes
Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work 👏🏼

I think we can run a round of light improvements over the naming and interfaces, rest looks solid!

p-datadog
p-datadog previously requested changes Feb 3, 2025
View reviewed changes
Copy link
Member

@p-datadog p-datadog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes due to mismatch between no change log entry claimed in PR description and the new API being marked public.

@vpellan vpellan force-pushed the vpellan/meta-struct branch from 0c00cb0 to 7d76a8e Compare February 4, 2025 17:29
@vpellan vpellan force-pushed the vpellan/meta-struct branch 2 times, most recently from 9b62558 to 52d5a77 Compare February 5, 2025 09:22
@p-datadog p-datadog dismissed their stale review February 5, 2025 15:21

repaired

Strech
Strech reviewed Feb 5, 2025
View reviewed changes
Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on the tracing team, but I've left a few notes following our discussion about the metastruct implementation

@vpellan vpellan force-pushed the vpellan/meta-struct branch 2 times, most recently from a771a6d to 65dd37b Compare February 6, 2025 12:31
Strech
Strech approved these changes Feb 6, 2025
View reviewed changes
Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing move, I think it's good to go! Solid work 👏🏼

P.S I left few suggestions, but they are not blockers

@vpellan vpellan force-pushed the vpellan/meta-struct branch 2 times, most recently from 771634f to a3ce0bb Compare February 20, 2025 12:58
@vpellan vpellan requested a review from a team as a code owner February 21, 2025 14:17
p-datadog
p-datadog reviewed Mar 3, 2025
View reviewed changes
y9v
y9v previously requested changes Mar 5, 2025
View reviewed changes
@y9v y9v force-pushed the vpellan/meta-struct branch from 52c0097 to dc96af1 Compare March 6, 2025 11:27
@y9v y9v requested a review from Strech March 6, 2025 11:28
@y9v y9v force-pushed the vpellan/meta-struct branch 2 times, most recently from 1fe22fc to 2e8289c Compare March 6, 2025 12:34
@y9v y9v dismissed their stale review March 6, 2025 12:37

I updated the code

@y9v y9v requested a review from p-datadog March 6, 2025 14:00
Strech
Strech approved these changes Mar 6, 2025
View reviewed changes
Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome 👍🏼

P.S I left few non-blocking test improvements.

vpellan and others added 15 commits March 10, 2025 10:11
@vpellan @y9v
Add meta_struct support on traces and spans
f172b0b
@vpellan @y9v
Add get_meta_struct and meta_struct_size methods to MetaStruct class …
25255bb 
…and TraceOperation
@vpellan @y9v
Add meta_struct specs
7322956
@vpellan @y9v
Rename meta_struct into metastruct
276261e
@vpellan @y9v
Remove metastruct_size (no usage)
7959119 
Split set_metastruct into deep_merge_metastruct! and metastruct=
@vpellan @y9v
Add metastruct type
7c1d042
@vpellan @y9v
Rename get_metastruct to get_metastruct_field
6012be1
@vpellan @y9v
Add set_metastruct_field method
521f1b0
@vpellan @y9v
Change Metastruct from module to class
be3a798
@vpellan @y9v
Add .dig to metastruct
a3e30b3
@vpellan @y9v
Remove metastruct private method
bca011e
@vpellan @y9v
- Move merger as a constant
6e4d4ef 
- Make metastruct extends Forwardable and forward methods to @Metastruct
-add self.empty for when creating an empty metastruct (used in SpanOperation
@y9v
Add MetaspanTagging module and Metastruct data class
0f11e1b
@y9v
Improve metrastruct tagging specs
0adbbb6
@y9v
Improve MetastructTagging module spec
a70a07e
@y9v y9v force-pushed the vpellan/meta-struct branch from d580c5e to a70a07e Compare March 10, 2025 09:11
@y9v y9v merged commit 65b2901 into master Mar 10, 2025
482 checks passed
@y9v y9v deleted the vpellan/meta-struct branch March 10, 2025 10:15
@github-actions github-actions bot added this to the 2.13.0 milestone Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Strech Strech Strech approved these changes

@y9v y9v y9v left review comments

@datadog-datadog-prod-us1 datadog-datadog-prod-us1[bot] datadog-datadog-prod-us1[bot] left review comments

@marcotc marcotc marcotc approved these changes

@p-datadog p-datadog Awaiting requested review from p-datadog

Assignees
No one assigned
Labels
tracing
Projects
None yet
Milestone
2.13.0
Development

Successfully merging this pull request may close these issues.
6 participants
@vpellan @codecov-commenter @marcotc @Strech @y9v @p-datadog