Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sampler and span processor prototype implementations for consistent sampling #226

Merged
merged 54 commits into from
Apr 20, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
7116c5e
sampler and exporter implementations for consistent sampling
oertl Feb 4, 2022
d8dd6e6
Merge branch 'main' into consistent-sampling
oertl Feb 7, 2022
6b4ba10
improved dependencies (in particular, removed dependency on guava)
oertl Feb 14, 2022
5ca2625
Merge remote-tracking branch 'origin' into consistent-sampling
oertl Feb 14, 2022
038f803
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sam…
oertl Feb 14, 2022
a686b3a
reverted some changes
oertl Feb 14, 2022
08bbc40
removed wrong immutable annotation
oertl Feb 15, 2022
9258ef4
added javadoc
oertl Feb 15, 2022
000ffea
avoid else statements when returning
oertl Feb 15, 2022
f7911c3
factory methods for consistent samplers, avoid exposure of implementa…
oertl Feb 15, 2022
b3459f9
added javadoc for AND and OR sampler composition
oertl Feb 15, 2022
0bbdb73
Merge remote-tracking branch 'origin' into consistent-sampling
oertl Feb 15, 2022
185f579
replaced use of synchronized by atomic reference
oertl Feb 15, 2022
abb7e71
simplified thread local initialization
oertl Feb 15, 2022
dd3df4e
removed consistent reservoir sampling
oertl Feb 16, 2022
90a2960
Merge branch 'open-telemetry:main' into consistent-sampling
oertl Mar 9, 2022
6f731b6
improved comment
oertl Mar 9, 2022
d40c0f1
removed unnecessary clipping of sampling probability
oertl Mar 10, 2022
775ca52
added javadoc explaining maths of implementation
oertl Mar 10, 2022
2ffe08f
Merge branch 'open-telemetry:main' into consistent-sampling
oertl Mar 10, 2022
a82eee6
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sta…
oertl Apr 9, 2022
6e8a38a
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sta…
oertl Apr 9, 2022
ae9670b
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sta…
oertl Apr 9, 2022
83c21f4
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sta…
oertl Apr 9, 2022
35d0373
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sam…
oertl Apr 19, 2022
348a240
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sam…
oertl Apr 19, 2022
0afaa2f
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sam…
oertl Apr 19, 2022
f63bb0b
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sam…
oertl Apr 19, 2022
aaffdd7
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sam…
oertl Apr 19, 2022
56eec39
added component owner for consistent sampling
oertl Apr 19, 2022
667e1f7
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sta…
oertl Apr 19, 2022
7910939
Merge branch 'consistent-sampling' of github.com:dynatrace-oss-contri…
oertl Apr 19, 2022
3fae630
removed nonnull annotation
oertl Apr 19, 2022
5040a74
renamed variable s -> pair
oertl Apr 19, 2022
e0041c0
renamed char parameter r -> c
oertl Apr 19, 2022
767e52f
renamed method isLowerCaseNum -> isDigit
oertl Apr 19, 2022
c18dcfe
use empty list instead of null for otherKeyValuePairs
oertl Apr 19, 2022
be68ff2
simplified isValueByte method
oertl Apr 19, 2022
30e42e3
Update consistent-sampling/src/main/java/io/opentelemetry/contrib/sta…
oertl Apr 19, 2022
d209258
Merge branch 'consistent-sampling' of github.com:dynatrace-oss-contri…
oertl Apr 19, 2022
7f58aa1
renamed variable sepPos -> separatorPos
oertl Apr 19, 2022
938382a
replaced 0. and 1. by 0.0 and 1.0
oertl Apr 19, 2022
630b157
improved readability as suggested by @trask
oertl Apr 19, 2022
1dca6f4
removed unused methods from RandomUtil
oertl Apr 19, 2022
09cc3a0
added javadoc
oertl Apr 19, 2022
15a8d50
renamed targetSpansPerNanosLimit -> targetSpansPerNanosecondLimit
oertl Apr 19, 2022
f70454a
throw IllegalArgumentException instead of returning NaN + added comments
oertl Apr 19, 2022
8d3a731
renamed tsStartPos -> startPos and eqPos -> colonPos
oertl Apr 19, 2022
3b7f045
improved readability of invariant check
oertl Apr 19, 2022
878c236
added some more test cases
oertl Apr 19, 2022
6fc45cd
fixed typo
oertl Apr 19, 2022
06cbeea
removed unused method
oertl Apr 19, 2022
edac8a9
refactored random generator
oertl Apr 19, 2022
da42018
made OtelTraceState and RandomGenerator package private and moved the…
oertl Apr 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
factory methods for consistent samplers, avoid exposure of implementa…
…tions
  • Loading branch information
oertl committed Feb 15, 2022
commit f7911c35c4c502b205493b02e10e5e441d47a00b
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,15 @@
import javax.annotation.concurrent.Immutable;

@Immutable
public final class ConsistentAlwaysOffSampler extends ConsistentSampler {
final class ConsistentAlwaysOffSampler extends ConsistentSampler {

private ConsistentAlwaysOffSampler() {}

private static final ConsistentSampler INSTANCE = new ConsistentAlwaysOffSampler();

static ConsistentSampler getInstance() {
return INSTANCE;
}

@Override
protected int getP(int parentP, boolean isRoot) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,15 @@
import javax.annotation.concurrent.Immutable;

@Immutable
public class ConsistentAlwaysOnSampler extends ConsistentSampler {
final class ConsistentAlwaysOnSampler extends ConsistentSampler {

private ConsistentAlwaysOnSampler() {}

private static final ConsistentSampler INSTANCE = new ConsistentAlwaysOnSampler();

static ConsistentSampler getInstance() {
return INSTANCE;
}

@Override
protected int getP(int parentP, boolean isRoot) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,13 @@
* <p>This sampler samples if both samplers would sample.
*/
@Immutable
public final class ConsistentComposedAndSampler extends ConsistentSampler {
final class ConsistentComposedAndSampler extends ConsistentSampler {

private final ConsistentSampler sampler1;
private final ConsistentSampler sampler2;
private final String description;

public static ConsistentComposedAndSampler create(
ConsistentSampler sampler1, ConsistentSampler sampler2) {
return new ConsistentComposedAndSampler(sampler1, sampler2);
}

private ConsistentComposedAndSampler(ConsistentSampler sampler1, ConsistentSampler sampler2) {
ConsistentComposedAndSampler(ConsistentSampler sampler1, ConsistentSampler sampler2) {
this.sampler1 = requireNonNull(sampler1);
this.sampler2 = requireNonNull(sampler2);
this.description =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,13 @@
* <p>This sampler samples if any of the two samplers would sample.
*/
@Immutable
public final class ConsistentComposedOrSampler extends ConsistentSampler {
final class ConsistentComposedOrSampler extends ConsistentSampler {

private final ConsistentSampler sampler1;
private final ConsistentSampler sampler2;
private final String description;

public static ConsistentComposedOrSampler create(
ConsistentSampler sampler1, ConsistentSampler sampler2) {
return new ConsistentComposedOrSampler(sampler1, sampler2);
}

private ConsistentComposedOrSampler(ConsistentSampler sampler1, ConsistentSampler sampler2) {
ConsistentComposedOrSampler(ConsistentSampler sampler1, ConsistentSampler sampler2) {
this.sampler1 = requireNonNull(sampler1);
this.sampler2 = requireNonNull(sampler2);
this.description =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
* spans).
*/
@Immutable
public final class ConsistentParentBasedSampler extends ConsistentSampler {
final class ConsistentParentBasedSampler extends ConsistentSampler {

private final ConsistentSampler rootSampler;

Expand All @@ -28,7 +28,7 @@ public final class ConsistentParentBasedSampler extends ConsistentSampler {
*
* @param rootSampler the root sampler
*/
public ConsistentParentBasedSampler(ConsistentSampler rootSampler) {
ConsistentParentBasedSampler(ConsistentSampler rootSampler) {
this(rootSampler, DefaultRandomGenerator.get());
}

Expand All @@ -39,7 +39,7 @@ public ConsistentParentBasedSampler(ConsistentSampler rootSampler) {
* @param rootSampler the root sampler
* @param threadSafeRandomGenerator a thread-safe random generator
*/
public ConsistentParentBasedSampler(
ConsistentParentBasedSampler(
ConsistentSampler rootSampler, RandomGenerator threadSafeRandomGenerator) {
super(threadSafeRandomGenerator);
this.rootSampler = requireNonNull(rootSampler);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

/** A consistent sampler that samples with a fixed probability. */
@Immutable
public class ConsistentProbabilityBasedSampler extends ConsistentSampler {
final class ConsistentProbabilityBasedSampler extends ConsistentSampler {

private final int lowerPValue;
private final int upperPValue;
Expand All @@ -23,7 +23,7 @@ public class ConsistentProbabilityBasedSampler extends ConsistentSampler {
*
* @param samplingProbability the sampling probability
*/
public ConsistentProbabilityBasedSampler(double samplingProbability) {
ConsistentProbabilityBasedSampler(double samplingProbability) {
this(samplingProbability, DefaultRandomGenerator.get());
}

Expand All @@ -33,7 +33,7 @@ public ConsistentProbabilityBasedSampler(double samplingProbability) {
* @param samplingProbability the sampling probability
* @param threadSafeRandomGenerator a thread-safe random generator
*/
public ConsistentProbabilityBasedSampler(
ConsistentProbabilityBasedSampler(
double samplingProbability, RandomGenerator threadSafeRandomGenerator) {
super(threadSafeRandomGenerator);
if (samplingProbability < 0.0 || samplingProbability > 1.0) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
* J. "Forecasting data published at irregular time intervals using an extension of Holt's method."
* Management science 32.4 (1986): 499-510.) to estimate the current rate of spans.
*/
public class ConsistentRateLimitingSampler extends ConsistentSampler {
final class ConsistentRateLimitingSampler extends ConsistentSampler {

private final String description;
private final LongSupplier nanoTimeSupplier;
Expand All @@ -44,8 +44,7 @@ public class ConsistentRateLimitingSampler extends ConsistentSampler {
* @param adaptationTimeSeconds the typical time to adapt to a new load (time constant used for
* exponential smoothing)
*/
public ConsistentRateLimitingSampler(
double targetSpansPerSecondLimit, double adaptationTimeSeconds) {
ConsistentRateLimitingSampler(double targetSpansPerSecondLimit, double adaptationTimeSeconds) {
this(
targetSpansPerSecondLimit,
adaptationTimeSeconds,
Expand All @@ -62,7 +61,7 @@ public ConsistentRateLimitingSampler(
* @param threadSafeRandomGenerator a thread-safe random generator
* @param nanoTimeSupplier a supplier for the current nano time
*/
public ConsistentRateLimitingSampler(
ConsistentRateLimitingSampler(
double targetSpansPerSecondLimit,
double adaptationTimeSeconds,
RandomGenerator threadSafeRandomGenerator,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,122 @@
import io.opentelemetry.sdk.trace.samplers.SamplingDecision;
import io.opentelemetry.sdk.trace.samplers.SamplingResult;
import java.util.List;
import java.util.function.LongSupplier;
import javax.annotation.Nonnull;

/** Abstract base class for consistent samplers. */
abstract class ConsistentSampler implements Sampler {
public abstract class ConsistentSampler implements Sampler {

/**
* Returns a {@link ConsistentSampler} that samples all spans.
*
* @return a sampler
*/
public static final ConsistentSampler alwaysOn() {
return ConsistentAlwaysOnSampler.getInstance();
}

/**
* Returns a {@link ConsistentSampler} that does not sample any span.
*
* @return a sampler
*/
public static final ConsistentSampler alwaysOff() {
return ConsistentAlwaysOffSampler.getInstance();
}

/**
* Returns a {@link ConsistentSampler} that samples each span with a fixed probability.
*
* @param samplingProbability the sampling probability
* @return a sampler
*/
public static final ConsistentSampler probabilityBased(double samplingProbability) {
return new ConsistentProbabilityBasedSampler(samplingProbability);
}

/**
* Returns a {@link ConsistentSampler} that samples each span with a fixed probability.
*
* @param samplingProbability the sampling probability
* @param threadSafeRandomGenerator a thread-safe random generator
* @return a sampler
*/
public static final ConsistentSampler probabilityBased(
double samplingProbability, RandomGenerator threadSafeRandomGenerator) {
return new ConsistentProbabilityBasedSampler(samplingProbability, threadSafeRandomGenerator);
}

/**
* Returns a new {@link ConsistentSampler} that respects the sampling decision of the parent span
* or falls-back to the given sampler if it is a root span.
*
* @param rootSampler the root sampler
*/
public static final ConsistentSampler parentBased(@Nonnull ConsistentSampler rootSampler) {
return new ConsistentParentBasedSampler(rootSampler);
}

/**
* Returns a new {@link ConsistentSampler} that respects the sampling decision of the parent span
* or falls-back to the given sampler if it is a root span.
*
* @param rootSampler the root sampler
* @param threadSafeRandomGenerator a thread-safe random generator
*/
public static final ConsistentSampler parentBased(
ConsistentSampler rootSampler, RandomGenerator threadSafeRandomGenerator) {
return new ConsistentParentBasedSampler(rootSampler, threadSafeRandomGenerator);
}

/**
* Returns a new {@link ConsistentSampler} that attempts to adjust the sampling probability
* dynamically to meet the target span rate.
*
* @param targetSpansPerSecondLimit the desired spans per second limit
* @param adaptationTimeSeconds the typical time to adapt to a new load (time constant used for
* exponential smoothing)
*/
public static final ConsistentSampler rateLimited(
double targetSpansPerSecondLimit, double adaptationTimeSeconds) {
return new ConsistentRateLimitingSampler(targetSpansPerSecondLimit, adaptationTimeSeconds);
}

/**
* Returns a new {@link ConsistentSampler} that attempts to adjust the sampling probability
* dynamically to meet the target span rate.
*
* @param targetSpansPerSecondLimit the desired spans per second limit
* @param adaptationTimeSeconds the typical time to adapt to a new load (time constant used for
* exponential smoothing)
* @param threadSafeRandomGenerator a thread-safe random generator
* @param nanoTimeSupplier a supplier for the current nano time
*/
public static final ConsistentSampler rateLimited(
double targetSpansPerSecondLimit,
double adaptationTimeSeconds,
RandomGenerator threadSafeRandomGenerator,
LongSupplier nanoTimeSupplier) {
return new ConsistentRateLimitingSampler(
targetSpansPerSecondLimit,
adaptationTimeSeconds,
threadSafeRandomGenerator,
nanoTimeSupplier);
}

public ConsistentSampler and(ConsistentSampler otherConsistentSampler) {
if (otherConsistentSampler == this) {
return this;
}
return new ConsistentComposedAndSampler(this, otherConsistentSampler);
}

public ConsistentSampler or(ConsistentSampler otherConsistentSampler) {
if (otherConsistentSampler == this) {
return this;
}
return new ConsistentComposedOrSampler(this, otherConsistentSampler);
}

protected final RandomGenerator threadSafeRandomGenerator;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@
import io.opentelemetry.api.internal.GuardedBy;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.contrib.samplers.ConsistentAlwaysOnSampler;
import io.opentelemetry.contrib.samplers.ConsistentProbabilityBasedSampler;
import io.opentelemetry.contrib.samplers.ConsistentSampler;
import io.opentelemetry.contrib.state.OtelTraceState;
import io.opentelemetry.contrib.util.RandomGenerator;
import io.opentelemetry.sdk.common.CompletableResultCode;
Expand Down Expand Up @@ -729,7 +728,7 @@ void exportDifferentConsistentlySampledSpans() {

sdkTracerProvider =
SdkTracerProvider.builder()
.setSampler(new ConsistentAlwaysOnSampler())
.setSampler(ConsistentSampler.alwaysOn())
.addSpanProcessor(spanProcessor)
.build();

Expand Down Expand Up @@ -791,8 +790,7 @@ private void testConsistentSampling(
sdkTracerProvider =
SdkTracerProvider.builder()
.setSampler(
new ConsistentProbabilityBasedSampler(
samplingProbability, threadSafeRandomGenerator2))
ConsistentSampler.probabilityBased(samplingProbability, threadSafeRandomGenerator2))
.addSpanProcessor(spanProcessor)
.build();

Expand Down Expand Up @@ -1013,8 +1011,7 @@ private StatisticalSummary calculateStatisticalSummary(
sdkTracerProvider =
SdkTracerProvider.builder()
.setSampler(
new ConsistentProbabilityBasedSampler(
samplingProbability, threadSafeRandomGenerator2))
ConsistentSampler.probabilityBased(samplingProbability, threadSafeRandomGenerator2))
.addSpanProcessor(spanProcessor)
.build();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ public void init() {
private void test(SplittableRandom rng, double samplingProbability) {
int numSpans = 1000000;

Sampler sampler = new ConsistentProbabilityBasedSampler(samplingProbability, rng::nextLong);
Sampler sampler = ConsistentSampler.probabilityBased(samplingProbability, rng::nextLong);

Map<Integer, Long> observedPvalues = new HashMap<>();
for (long i = 0; i < numSpans; ++i) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ void testConstantRate() {
double adaptationTimeSeconds = 5;
SplittableRandom random = new SplittableRandom(0L);

ConsistentRateLimitingSampler sampler =
new ConsistentRateLimitingSampler(
ConsistentSampler sampler =
ConsistentSampler.rateLimited(
targetSpansPerSecondLimit, adaptationTimeSeconds, random::nextLong, nanoTimeSupplier);

long nanosBetweenSpans = TimeUnit.MICROSECONDS.toNanos(100);
Expand Down Expand Up @@ -95,8 +95,8 @@ void testRateIncrease() {
double adaptationTimeSeconds = 5;
SplittableRandom random = new SplittableRandom(0L);

ConsistentRateLimitingSampler sampler =
new ConsistentRateLimitingSampler(
ConsistentSampler sampler =
ConsistentSampler.rateLimited(
targetSpansPerSecondLimit, adaptationTimeSeconds, random::nextLong, nanoTimeSupplier);

long nanosBetweenSpans1 = TimeUnit.MICROSECONDS.toNanos(100);
Expand Down Expand Up @@ -151,8 +151,8 @@ void testRateDecrease() {
double adaptationTimeSeconds = 5;
SplittableRandom random = new SplittableRandom(0L);

ConsistentRateLimitingSampler sampler =
new ConsistentRateLimitingSampler(
ConsistentSampler sampler =
ConsistentSampler.rateLimited(
targetSpansPerSecondLimit, adaptationTimeSeconds, random::nextLong, nanoTimeSupplier);

long nanosBetweenSpans1 = TimeUnit.MICROSECONDS.toNanos(10);
Expand Down