Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Span/Trace ID to be byte array #2001

Merged
merged 6 commits into from
Oct 26, 2020

Conversation

bogdandrutu
Copy link
Member

@bogdandrutu bogdandrutu commented Oct 22, 2020

A lot of boilerplate changes, the major ones:

  • Return error and reject requests with trace/span ids that are not 16/8 byte arrays
  • For OC copy only the first 8/16 bytes if larger than that, if shorter pad with 0 at the end.

Before:

goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/consumer/pdata
BenchmarkTracesFromOtlp
BenchmarkTracesFromOtlp-16    	     622	   1885796 ns/op	 1105052 B/op	   40944 allocs/op
PASS

After:

goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/consumer/pdata
BenchmarkTracesFromOtlp
BenchmarkTracesFromOtlp-16    	     686	   1759476 ns/op	  981565 B/op	   35113 allocs/op
PASS

@bogdandrutu bogdandrutu requested a review from a team October 22, 2020 22:39
@codecov
Copy link

codecov bot commented Oct 22, 2020

Codecov Report

Merging #2001 into master will increase coverage by 0.06%.
The diff coverage is 96.15%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2001      +/-   ##
==========================================
+ Coverage   91.49%   91.55%   +0.06%     
==========================================
  Files         282      282              
  Lines       16628    16631       +3     
==========================================
+ Hits        15214    15227      +13     
+ Misses        978      973       -5     
+ Partials      436      431       -5     
Impacted Files Coverage Δ
consumer/pdata/generated_metrics.go 100.00% <ø> (ø)
...mplingprocessor/tailsamplingprocessor/processor.go 77.06% <50.00%> (ø)
translator/trace/zipkin/traces_to_zipkinv2.go 93.84% <81.81%> (+0.88%) ⬆️
consumer/pdata/spanid.go 100.00% <100.00%> (ø)
consumer/pdata/traceid.go 100.00% <100.00%> (ø)
internal/data/testdata/log.go 98.13% <100.00%> (ø)
internal/goldendataset/generator_commons.go 88.88% <100.00%> (ø)
internal/goldendataset/span_generator.go 98.86% <100.00%> (ø)
...babilisticsamplerprocessor/probabilisticsampler.go 91.58% <100.00%> (+0.07%) ⬆️
translator/internaldata/oc_to_traces.go 85.63% <100.00%> (+0.16%) ⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e1f85d4...a8c9316. Read the comment docs.


// NewSpanID creates a SpanID from a byte slice.
func NewSpanID(bytes []byte) SpanID {
func NewSpanID(bytes [8]byte) SpanID {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it makes sense to define type alias for [8]byte (or [spanIDSize]byte), name it something like SpanIDBytes and use everywhere (and similar for traceid).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be done in a followup PR.

}

// Unmarshal inflates this trace ID from binary representation. Called by Protobuf serialization.
func (t *SpanID) Unmarshal(data []byte) error {
return (*bytesID)(t).Unmarshal(data)
func (sid *SpanID) Unmarshal(data []byte) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to document somewhere that either 0 length or exactly 8 byte length span id is allowed. Previously any length was allowed. Same for trace id.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the proto definition:
https://github.com/open-telemetry/opentelemetry-proto/blob/master/opentelemetry/proto/trace/v1/trace.proto#L58

I think we have that documented there, do you think duplicating the comment is ok?

}

// IsValid returns true if id contains at leas one non-zero byte.
func (sid SpanID) IsValid() bool {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call this IsNonZero instead? I am not quite sure "valid" is the right term. What do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on w3c and https://github.com/open-telemetry/opentelemetry-proto/blob/master/opentelemetry/proto/trace/v1/trace.proto#L58 the all zeros is called invalid, so I wanted to keep it consistent.

internal/data/opentelemetry-proto-gen/common/v1/spanid.go Outdated Show resolved Hide resolved
}

// Compare the ids. See bytes.Compare for expected return values.
func (t SpanID) Compare(that SpanID) int {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Compare function unnecessary? I think it was called by Gogoproto marshaling/unmarshaling.

Copy link
Member Author

@bogdandrutu bogdandrutu Oct 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. All tests are passing.
  2. I did a grep in the generated files, no Compare found, so I expect that it is not used.

internal/data/opentelemetry-proto-gen/common/v1/spanid.go Outdated Show resolved Hide resolved
translator/internaldata/oc_to_traces.go Outdated Show resolved Hide resolved
return false
// Transforms the byte slice trace ID into a [16]byte internal pdata.TraceID.
// If larger input then it is truncated to 16 bytes.
func traceIDToInternal(traceID []byte) pdata.TraceID {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it would be useful to have error handling for the cases when input byte slice is not the expected length. We should probably accept everything in the [0..16] range and reject anything longer. It likely "never" happens but who knows what we see in the input. Do you think it is fine to silently truncate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is fine to silently truncate, we have it documented in both protos otlp and opencensus that the size should be 16.

@tigrannajaryan
Copy link
Member

This should reduce the number of allocations of slices used for traceid/spanid. Can you run before/after benchmark, I am curious if we can see any performance difference.

bogdandrutu and others added 5 commits October 26, 2020 12:02
A lot of boilerplate changes, the major ones:
* Return error and reject requests with trace/span ids that are not 16/8 byte arrays
* For OC copy only the first 8/16 bytes if larger than that, if shorter pad with 0 at the end.

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Before:
```
goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/consumer/pdata
BenchmarkTracesFromOtlp
BenchmarkTracesFromOtlp-16    	     622	   1885796 ns/op	 1105052 B/op	   40944 allocs/op
PASS
```

After:
```
goos: darwin
goarch: amd64
pkg: go.opentelemetry.io/collector/consumer/pdata
BenchmarkTracesFromOtlp
BenchmarkTracesFromOtlp-16    	     686	   1759476 ns/op	  981565 B/op	   35113 allocs/op
PASS
```

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>
Copy link
Member

@tigrannajaryan tigrannajaryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement.

@bogdandrutu bogdandrutu merged commit 7de4bf9 into open-telemetry:master Oct 26, 2020
@bogdandrutu bogdandrutu deleted the bytearray branch October 26, 2020 19:38
MovieStoreGuy pushed a commit to atlassian-forks/opentelemetry-collector that referenced this pull request Nov 11, 2021
…lemetry#2001)

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>

Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants