Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[probabilistic sampling processor] encoded sampling probability (support OTEP 235) #31894

Merged
merged 99 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
e822a9b
Add t-value sampler draft
jmacd May 12, 2023
1bc6017
copy/import tracestate parser package
jmacd May 15, 2023
d1fd891
test ot tracestate
jmacd May 16, 2023
85e4472
tidy
jmacd May 16, 2023
bb75f8a
renames
jmacd May 16, 2023
6a57b77
testing two parsers w/ generic code
jmacd May 17, 2023
7fa8130
integrated
jmacd May 17, 2023
36230e7
Comments
jmacd May 17, 2023
7bae35c
revert two files
jmacd May 17, 2023
9010a67
Update with r, s, and t-value. Now using regexps and strings.IndexBy…
jmacd Jun 1, 2023
0e27e40
fix sampler build
jmacd Jun 1, 2023
efcdc3d
add support for s-value for non-consistent mode
jmacd Jun 1, 2023
939c758
WIP
jmacd Jul 10, 2023
b9a1e56
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Aug 2, 2023
a31266c
use new proposed syntax see https://github.com/open-telemetry/opentel…
jmacd Aug 2, 2023
690cd64
update tracestate libs for new encoding
jmacd Aug 2, 2023
c8baf29
wip working on probabilistic sampler with two new modes: downsampler …
jmacd Aug 2, 2023
7f47e4a
unsigned implement split
jmacd Aug 3, 2023
422e0b2
two implementations
jmacd Aug 3, 2023
787b9fd
wip
jmacd Sep 5, 2023
ed36f03
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Sep 6, 2023
d795210
Updates for OTEP 235
jmacd Sep 6, 2023
09000f7
wip TODO
jmacd Sep 6, 2023
a4d467b
versions.yaml
jmacd Sep 6, 2023
e373b9b
Add proportional sampler mode; comment on TODOs; create SamplerMode t…
jmacd Sep 7, 2023
fe6a085
back from internal
jmacd Oct 4, 2023
396efb1
wip
jmacd Oct 4, 2023
36de5dd
fix existing tests
jmacd Oct 6, 2023
f1aa0ad
:wip:
jmacd Oct 12, 2023
700734e
Update for rejection threshold
jmacd Nov 15, 2023
ae50bdd
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Nov 15, 2023
a94b8e7
fix preexisting tests
jmacd Nov 16, 2023
4edcbcb
basic yes/no t-value sampling test
jmacd Nov 16, 2023
53bf119
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Nov 29, 2023
3cdb957
add version for sampling pkg
jmacd Nov 29, 2023
e506847
more testing
jmacd Dec 7, 2023
2cddfeb
add probability to threshold with precision option
jmacd Dec 8, 2023
f69d6ee
ProbabilityToThresholdWithPrecision
jmacd Dec 8, 2023
cc02934
test coverage for equalizing and proportional
jmacd Dec 8, 2023
1eecc4a
config test
jmacd Dec 8, 2023
2159107
comments and notes
jmacd Dec 8, 2023
e0898a6
update README
jmacd Dec 8, 2023
d0991ed
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Jan 10, 2024
a002774
Remove sampling pkg, it is now upstream
jmacd Feb 14, 2024
3a49922
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Feb 28, 2024
fca0184
build w/ new sampling pkg
jmacd Feb 28, 2024
f11e0a4
more test coverage
jmacd Feb 29, 2024
3f495a6
more config tests
jmacd Feb 29, 2024
581095c
test precision underflow
jmacd Mar 1, 2024
7b8fd31
test warning logs
jmacd Mar 1, 2024
1a6be4f
hash_seed fixes
jmacd Mar 1, 2024
712cf17
wip
jmacd Mar 4, 2024
34c0d3b
aip
jmacd Mar 5, 2024
7742668
wip-refactoring
jmacd Mar 13, 2024
8d60168
refactor wip
jmacd Mar 14, 2024
3779caa
cleanup refactor
jmacd Mar 14, 2024
c261ac1
wip
jmacd Mar 14, 2024
34469e4
moving code
jmacd Mar 15, 2024
8dabf47
fix tests; round up small probs to avoid errors
jmacd Mar 15, 2024
d44afb5
preserve legacy behavior
jmacd Mar 15, 2024
1cf9991
logs handled sampling priority differently
jmacd Mar 15, 2024
365d35d
still two errors
jmacd Mar 18, 2024
12a3047
builds
jmacd Mar 19, 2024
8655f42
needs testing
jmacd Mar 19, 2024
468e6c6
fixing tests
jmacd Mar 21, 2024
23b4423
cleanup
jmacd Mar 21, 2024
07841e5
remove strict feature
jmacd Mar 21, 2024
6936bc4
tests fixed
jmacd Mar 21, 2024
c132f4c
wip
jmacd Mar 22, 2024
bd13ac9
typo
jmacd Mar 22, 2024
aa33b1c
more logs tests
jmacd Mar 22, 2024
06556dc
Add more comments
jmacd Mar 22, 2024
a4940e6
update README
jmacd Mar 22, 2024
4f616e9
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Mar 22, 2024
b4ca3aa
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Mar 25, 2024
fdd26ac
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Mar 25, 2024
794d1a1
wip update
jmacd May 30, 2024
a305a7f
undo comment changes
jmacd May 30, 2024
98433af
test all modes logs missing randomness
jmacd May 30, 2024
3aa4608
more missing rando
jmacd May 30, 2024
a0bc49e
smaller diff
jmacd May 30, 2024
d0aea21
comment carrier
jmacd May 30, 2024
7b81625
chlog
jmacd May 30, 2024
fe4dd37
simplify ctcom
jmacd May 30, 2024
a244866
lint
jmacd May 30, 2024
89331bc
combine README updates
jmacd May 30, 2024
04d65c4
tidy
jmacd May 30, 2024
9cb1586
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd May 30, 2024
a98db61
Add sampler mode use-cases
jmacd Jun 10, 2024
d33660b
rephrase tracestate; logs do not use tracestate
jmacd Jun 10, 2024
c67350d
explain sampling precision
jmacd Jun 10, 2024
b0a9516
move misplaced text
jmacd Jun 10, 2024
95ecbae
remove multierr
jmacd Jun 10, 2024
cbcc853
Apply suggestions from code review
jmacd Jun 11, 2024
ad32651
only debug and info
jmacd Jun 11, 2024
6b71ea8
adjust test for debug-level logs
jmacd Jun 11, 2024
61abf1f
revert change of default mode
jmacd Jun 11, 2024
0664ea1
Merge branch 'main' of github.com:open-telemetry/opentelemetry-collec…
jmacd Jun 12, 2024
1926afb
Merge branch 'main' into jmacd/tvaluesampler
jmacd Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add more comments
  • Loading branch information
jmacd committed Mar 22, 2024
commit 06556dc658708d8fda91c997dbfeb50dccbadccb
6 changes: 3 additions & 3 deletions pkg/sampling/oteltracestate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ func TestOpenTelemetryTraceStateTValueUpdate(t *testing.T) {
require.NotEqual(t, "", otts.RValue())

th, _ := TValueToThreshold("3")
require.NoError(t, otts.UpdateTValueWithSampling(th, "3"))
require.NoError(t, otts.UpdateTValueWithSampling(th))

require.Equal(t, "3", otts.TValue())
tv, hasTv := otts.TValueThreshold()
Expand All @@ -126,7 +126,7 @@ func TestOpenTelemetryTraceStateRTUpdate(t *testing.T) {
require.True(t, otts.HasAnyValue())

th, _ := TValueToThreshold("3")
require.NoError(t, otts.UpdateTValueWithSampling(th, "3"))
require.NoError(t, otts.UpdateTValueWithSampling(th))
otts.SetRValue(must(RValueToRandomness("00000000000003")))

const updated = "rv:00000000000003;th:3;a:b"
Expand Down Expand Up @@ -329,7 +329,7 @@ func TestUpdateTValueWithSampling(t *testing.T) {
newTh, err := ProbabilityToThreshold(test.prob)
require.NoError(t, err)

upErr := otts.UpdateTValueWithSampling(newTh, newTh.TValue())
upErr := otts.UpdateTValueWithSampling(newTh)

require.Equal(t, test.updateErr, upErr)

Expand Down
6 changes: 5 additions & 1 deletion pkg/sampling/randomness.go
Original file line number Diff line number Diff line change
Expand Up @@ -97,11 +97,15 @@ func (rnd Randomness) RValue() string {
return strconv.FormatUint(numRandomnessValues+rnd.unsigned, hexBase)[1:]
}

// @@@
// Unsigned returns the unsigned representation of the random value.
// Items of data are sampled when Threshold.Unsigned() is <= Randomness.Unsigned(.
func (rnd Randomness) Unsigned() uint64 {
return rnd.unsigned
}

// UnsignedToRandomness constructs a randomness using 56 random bits
// of unsigned number. If the input is out of range, the maximum value
// will be returned with an error.
func UnsignedToRandomness(x uint64) (Randomness, error) {
if x >= MaxAdjustedCount {
return AllProbabilitiesRandomness, ErrRValueSize
Expand Down
8 changes: 5 additions & 3 deletions pkg/sampling/threshold.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,8 @@ func TValueToThreshold(s string) (Threshold, error) {
}

// UnsignedToThreshold constructs a threshold expressed in terms
// defined by Unsigned(), the number of rejections.
// defined by number of rejections out of MaxAdjustedCount, which
// equals the number of randomness values.
func UnsignedToThreshold(unsigned uint64) (Threshold, error) {
if unsigned >= MaxAdjustedCount {
return NeverSampleThreshold, ErrTValueSize
Expand Down Expand Up @@ -114,9 +115,10 @@ func (th Threshold) TValue() string {
}

// ShouldSample returns true when the span passes this sampler's
// consistent sampling decision.
// consistent sampling decision. The sampling decision can be
// expressed as a T <= R.
func (th Threshold) ShouldSample(rnd Randomness) bool {
return rnd.unsigned >= th.unsigned
return th.unsigned <= rnd.unsigned
}

// Unsigned expresses the number of Randomness values (out of
Expand Down
2 changes: 1 addition & 1 deletion pkg/sampling/w3ctracestate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func ExampleW3CTraceState_Serialize() {
// value, since in some code paths the Threshold will have
// just been parsed from a T-value, and in other code paths
// the T-value will be precalculated.
err = w3c.OTelValue().UpdateTValueWithSampling(th, th.TValue())
err = w3c.OTelValue().UpdateTValueWithSampling(th)
if err != nil {
panic(err)
}
Expand Down