Getting Started • Getting Involved • Getting In Touch

OpenTelemetry Instrumentation for Java

This project provides a Java agent JAR that can be attached to any Java 8+ application and dynamically injects bytecode to capture telemetry from a number of popular libraries and frameworks. You can export the telemetry data in a variety of formats. You can also configure the agent and exporter via command line arguments or environment variables. The net result is the ability to gather telemetry data from a Java application without code changes.

Getting Started

Download the latest version.

This package includes the instrumentation agent as well as instrumentations for all supported libraries and all available data exporters. The package provides a completely automatic, out-of-the-box experience.

Enable the instrumentation agent using the -javaagent flag to the JVM.

java -javaagent:path/to/opentelemetry-javaagent-all.jar \
     -jar myapp.jar

By default, the OpenTelemetry Java agent uses OTLP exporter configured to send data to OpenTelemetry collector at http://localhost:4317.

Configuration parameters are passed as Java system properties (-D flags) or as environment variables. See below for a full list of environment variables. For example:

java -javaagent:path/to/opentelemetry-javaagent-all.jar \
     -Dotel.trace.exporter=zipkin \
     -jar myapp.jar

Specify the external exporter JAR file using the otel.exporter.jar system property:

java -javaagent:path/to/opentelemetry-javaagent-all.jar \
     -Dotel.exporter.jar=path/to/external-exporter.jar \
     -jar myapp.jar

Learn how to add custom instrumentation in the Manually Instrumenting section.

Configuration parameters (subject to change!)

Note: These parameter names are very likely to change over time, so please check back here when trying out a new version! Please report any bugs or unexpected behavior you find.

Exporters

The following configuration properties are common to all exporters:

System property	Environment variable	Purpose
otel.trace.exporter	OTEL_TRACE_EXPORTER	The exporter to be used for tracing. Default is `otlp`. `none` means no exporter.
otel.metrics.exporter	OTEL_METRICS_EXPORTER	The exporter to be used for metrics. Default is `otlp`. `none` means no exporter.

OTLP exporter (both span and metric exporters)

A simple wrapper for the OpenTelemetry Protocol (OTLP) span and metric exporters of opentelemetry-java.

System property	Environment variable	Description
otel.trace.exporter=otlp (default)	OTEL_TRACE_EXPORTER=otlp	Select the OpenTelemetry exporter for tracing (default)
otel.metrics.exporter=otlp (default)	OTEL_METRICS_EXPORTER=otlp	Select the OpenTelemetry exporter for metrics (default)
otel.exporter.otlp.endpoint	OTEL_EXPORTER_OTLP_ENDPOINT	The OTLP endpoint to connect to. Must be a URL with a scheme of either `http` or `https` based on the use of TLS. Default is `http://localhost:4317`.
otel.exporter.otlp.headers	OTEL_EXPORTER_OTLP_HEADERS	Key-value pairs separated by semicolons to pass as request headers
otel.exporter.otlp.timeout	OTEL_EXPORTER_OTLP_TIMEOUT	The maximum waiting time allowed to send each batch. Default is `1000`.

To configure the service name for the OTLP exporter, add the service.name key to the OpenTelemetry Resource (see below), e.g. OTEL_RESOURCE_ATTRIBUTES=service.name=myservice.

Jaeger exporter

A simple wrapper for the Jaeger exporter of opentelemetry-java. This exporter uses gRPC for its communications protocol.

System property	Environment variable	Description
otel.trace.exporter=jaeger	OTEL_TRACE_EXPORTER=jaeger	Select the Jaeger exporter
otel.exporter.jaeger.endpoint	OTEL_EXPORTER_JAEGER_ENDPOINT	The Jaeger gRPC endpoint to connect to. Default is `localhost:14250`.

Zipkin exporter

A simple wrapper for the Zipkin exporter of opentelemetry-java. It sends JSON in Zipkin format to a specified HTTP URL.

System property	Environment variable	Description
otel.trace.exporter=zipkin	OTEL_TRACE_EXPORTER=zipkin	Select the Zipkin exporter
otel.exporter.zipkin.endpoint	OTEL_EXPORTER_ZIPKIN_ENDPOINT	The Zipkin endpoint to connect to. Default is `http://localhost:9411/api/v2/spans`. Currently only HTTP is supported.

Prometheus exporter

A simple wrapper for the Prometheus exporter of opentelemetry-java.

System property	Environment variable	Description
otel.metrics.exporter=prometheus	OTEL_METRICS_EXPORTER=prometheus	Select the Prometheus exporter
otel.exporter.prometheus.port	OTEL_EXPORTER_PROMETHEUS_PORT	The local port used to bind the prometheus metric server. Default is `9464`.
otel.exporter.prometheus.host	OTEL_EXPORTER_PROMETHEUS_HOST	The local address used to bind the prometheus metric server. Default is `0.0.0.0`.

Logging exporter

The logging exporter prints the name of the span along with its attributes to stdout. It's mainly used for testing and debugging.

System property	Environment variable	Description
otel.trace.exporter=logging	OTEL_TRACE_EXPORTER=logging	Select the logging exporter for tracing
otel.metrics.exporter=logging	OTEL_METRICS_EXPORTER=logging	Select the logging exporter for metrics
otel.exporter.logging.prefix	OTEL_EXPORTER_LOGGING_PREFIX	An optional string printed in front of the span name and attributes.

Propagator

The propagators determine which distributed tracing header formats are used, and which baggage propagation header formats are used.

System property	Environment variable	Description
otel.propagators	OTEL_PROPAGATORS	The propagators to be used. Use a comma-separated list for multiple propagators. Supported propagators are `tracecontext`, `baggage`, `b3`, `b3multi`, `jaeger`, `ottracer`, and `xray`. Default is `tracecontext,baggage` (W3C).

OpenTelemetry Resource

The OpenTelemetry Resource is a representation of the entity producing telemetry.

System property	Environment variable	Description
otel.resource.attributes	OTEL_RESOURCE_ATTRIBUTES	Specify resource attributes in the following format: key1=val1,key2=val2,key3=val3

Peer service name

The peer service name is the name of a remote service being connected to. It corresponds to service.name in the Resource for the local service.

System property	Environment variable	Description
otel.endpoint.peer.service.mapping	OTEL_ENDPOINT_PEER_SERVICE_MAPPING	Used to specify a mapping from hostnames or IP addresses to peer services, as a comma-separated list of host=name pairs. The peer service is added as an attribute to a span whose host or IP match the mapping. For example, if set to 1.2.3.4=cats-service,dogs-abcdef123.serverlessapis.com=dogs-api, requests to `1.2.3.4` will have a `peer.service` attribute of `cats-service` and requests to `dogs-abcdef123.serverlessapis.com` will have an attribute of `dogs-api`.

Batch span processor

System property	Environment variable	Description
otel.bsp.schedule.delay	OTEL_BSP_SCHEDULE_DELAY	The interval, in milliseconds, between two consecutive exports. Default is `5000`.
otel.bsp.max.queue.size	OTEL_BSP_MAX_QUEUE_SIZE	The maximum queue size. Default is `2048`.
otel.bsp.max.export.batch.size	OTEL_BSP_MAX_EXPORT_BATCH_SIZE	The maximum batch size. Default is `512`.
otel.bsp.export.timeout	OTEL_BSP_EXPORT_TIMEOUT	The maximum allowed time, in milliseconds, to export data. Default is `30000`.

Trace config

System property	Environment variable	Description
otel.trace.sampler	OTEL_TRACE_SAMPLER	The sampler to use for tracing. Defaults to `parentbased_always_on`
otel.trace.sampler.arg	OTEL_TRACE_SAMPLER_ARG	An argument to the configured tracer if supported, for example a ratio.
otel.span.attribute.count.limit	OTEL_SPAN_ATTRIBUTE_COUNT_LIMIT	The maximum number of attributes per span. Default is `32`.
otel.span.event.count.limit	OTEL_SPAN_EVENT_COUNT_LIMIT	The maximum number of events per span. Default is `128`.
otel.span.link.count.limit	OTEL_SPAN_LINK_COUNT_LIMIT	The maximum number of links per span. Default is `32`

Supported values for otel.trace.sampler are

"always_on": AlwaysOnSampler
"always_off": AlwaysOffSampler
"traceidratio": TraceIdRatioBased. otel.trace.sampler.arg sets the ratio.
"parentbased_always_on": ParentBased(root=AlwaysOnSampler)
"parentbased_always_off": ParentBased(root=AlwaysOffSampler)
"parentbased_traceidratio": ParentBased(root=TraceIdRatioBased). otel.trace.sampler.arg sets the ratio.

Interval metric reader

System property	Environment variable	Description
otel.imr.export.interval	OTEL_IMR_EXPORT_INTERVAL	The interval, in milliseconds, between pushes to the exporter. Default is `60000`.

Customizing the OpenTelemetry SDK

Customizing the SDK is highly advanced behavior and is still in the prototyping phase. It may change drastically or be removed completely. Use with caution

The OpenTelemetry SDK exposes SPI hooks for customizing its behavior, such as the Resource attached to spans or the Sampler.

Because the automatic instrumentation runs in a different classpath than the instrumented application, it is not possible for customization in the application to take advantage of this customization. In order to provide such customization, you can provide the path to a JAR file, including an SPI implementation using the system property otel.initializer.jar. Note that this JAR needs to shade the OpenTelemetry API in the same way as the agent does. The simplest way to do this is to use the same shading configuration as the agent from here. In addition, you must specify the io.opentelemetry.javaagent.shaded.io.opentelemetry.api.trace.spi.TraceProvider to the name of the class that implements the SPI.

Manually instrumenting

⚠️ starting with 0.6.0, and prior to version 1.0.0, opentelemetry-javaagent-all.jar only supports manual instrumentation using the opentelemetry-api version with the same version number as the Java agent you are using. Starting with 1.0.0, the Java agent will start supporting multiple (1.0.0+) versions of opentelemetry-api.

You'll need to add a dependency on the opentelemetry-api library to get started; if you intend to use the @WithSpan annotation, also include the opentelemetry-extension-annotations dependency.

Maven

  <dependencies>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-api</artifactId>
      <version>0.15.0</version>
    </dependency>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-extension-annotations</artifactId>
      <version>0.15.0</version>
    </dependency>
  </dependencies>

Gradle

dependencies {
    implementation('io.opentelemetry:opentelemetry-api:0.15.0')
    implementation('io.opentelemetry:opentelemetry-extension-annotations:0.15.0')
}

Adding attributes to the current span

A common need when instrumenting an application is to capture additional application-specific or business-specific information as additional attributes to an existing span from the automatic instrumentation. Grab the current span with Span.current() and use the setAttribute() methods:

import io.opentelemetry.api.trace.Span;

// ...

Span span = Span.current();
span.setAttribute(..., ...);

Creating spans around methods with `@WithSpan`

Another common situation is to capture a span around an existing first-party code method. The @WithSpan annotation makes this straightforward:

import io.opentelemetry.extension.annotations.WithSpan;

public class MyClass {
  @WithSpan
  public void MyLogic() {
      <...>
  }
}

Each time the application invokes the annotated method, it creates a span that denote its duration and provides any thrown exceptions. Unless specified as an argument to the annotation, the span name will be <className>.<methodName>.

Suppressing `@WithSpan` instrumentation

Suppressing @WithSpan is useful if you have code that is over-instrumented using @WithSpan and you want to suppress some of them without modifying the code.

System property	Environment variable	Purpose
trace.annotated.methods.exclude	TRACE_ANNOTATED_METHODS_EXCLUDE	Suppress `@WithSpan` instrumentation for specific methods.
Format is "my.package.MyClass1[method1,method2];my.package.MyClass2[method3]"

Creating spans manually with a Tracer

OpenTelemetry offers a tracer to easily enable custom instrumentation throughout your application. See the OpenTelemetry Java QuickStart for an example of how to configure the tracer and use the Tracer, Scope and Span interfaces to instrument your application.

Supported libraries, frameworks, and application servers

These are the supported libraries and frameworks:

Library/Framework	Versions
Akka HTTP	10.0+
Apache HttpAsyncClient	4.1+
Apache HttpClient	2.0+
Apache Wicket	8.0+
Armeria	1.3+
AsyncHttpClient	1.9+ (not including 2.x yet)
AWS Lambda	1.0+
AWS SDK	1.11.x and 2.2.0+
Cassandra Driver	3.0+
Couchbase Client	2.0+ (not including 3.x yet)
Dropwizard Views	0.7+
Elasticsearch API	5.0+ (not including 7.x yet)
Elasticsearch REST Client	5.0+
Finatra	2.9+
Geode Client	1.4+
Google HTTP Client	1.19+
Grizzly	2.0+ (disabled by default)
gRPC	1.5+
Hibernate	3.3+
HttpURLConnection	Java 7+
http4k ^†	3.270.0+
Hystrix	1.4+
JAX-RS	0.5+
JAX-RS Client	2.0+
JDBC	Java 7+
Jedis	1.4+
JMS	1.1+
JSP	2.3+
Kafka	0.11+
khttp	0.1+
Kubernetes Client	7.0+
Lettuce	4.0+ (not including 6.x yet)
Log4j 1	1.2+
Log4j 2	2.7+
Logback	1.0+
Mojarra	1.2+ (not including 3.x yet)
MongoDB Drivers	3.3+
MyFaces	1.2+ (not including 3.x yet)
Netty	3.8+
OkHttp	3.0+
Play	2.3+ (not including 2.8.x yet)
Play WS	1.0+
RabbitMQ Client	2.7+
Ratpack	1.4+
Reactor	3.1+
Reactor Netty	0.9+ (not including 1.0)
Rediscala	1.8+
Redisson	3.0+
RMI	Java 7+
RxJava	1.0+
Servlet	2.2+ (not including 5.x yet)
Spark Web Framework	2.3+
Spring Batch	3.0+
Spring Data	1.8+
Spring Scheduling	3.1+
Spring Web MVC	3.1+
Spring Webflux	5.0+
Spring Web Services	2.0+
Spymemcached	2.12+
Struts2	2.3+
Twilio	6.6+ (not including 8.x yet)
Vert.x	3.0+
Vert.x RxJava2	3.5+

^† OpenTelemetry support provided by the library

These are the supported application servers:

Application server	Version	JVM	OS
Glassfish	5.0.x, 5.1.x	OpenJDK 8, 11	Ubuntu 18, Windows Server 2019
JBoss EAP	7.1.x, 7.3.x	OpenJDK 8, 11	Ubuntu 18, Windows Server 2019
Jetty	9.4.x, 10.0.x, 11.0.x	OpenJDK 8, 11	Ubuntu 20
Payara	5.0.x, 5.1.x	OpenJDK 8, 11	Ubuntu 18, Windows Server 2019
Tomcat	7.0.x, 8.5.x, 9.0.x, 10.0.x	OpenJDK 8, 11	Ubuntu 18
TomEE	7.x, 8.x	OpenJDK 8, 11	Ubuntu 18
Weblogic	12	Oracle JDK 8	Oracle Linux 7, 8
Weblogic	14	Oracle JDK 8, 11	Oracle Linux 7, 8
Websphere Liberty Profile	20.0.0.12	OpenJDK 8, 11	Ubuntu 18, Windows Server 2019
WildFly	13.0.x	OpenJDK 8	Ubuntu 18, Windows Server 2019
WildFly	17.0.1, 21.0.0	OpenJDK 8, 11	Ubuntu 18, Windows Server 2019

Disabled instrumentations

Some instrumentations can produce too many spans and make traces very noisy. For this reason, the following instrumentations are disabled by default:

jdbc-datasource which creates spans whenever the java.sql.DataSource#getConnection method is called.

To enable them, add the otel.instrumentation.<name>.enabled system property: -Dotel.instrumentation.jdbc-datasource.enabled=true

Grizzly instrumentation

When you use Grizzly for Servlet-based applications, you get better experience from Servlet-specific support. As these two instrumentations conflict with each other, more generic instrumentation for Grizzly HTTP server is disabled by default. If needed, you can enable it by adding the following system property: -Dotel.instrumentation.grizzly.enabled=true

Suppressing specific auto-instrumentation

See Suppressing specific auto-instrumentation

Logger MDC auto-instrumentation

See Logger MDC auto-instrumentation

Troubleshooting

To turn on the agent's internal debug logging:

-Dotel.javaagent.debug=true

Note: These logs are extremely verbose. Enable debug logging only when needed. Debug logging negatively impacts the performance of your application.

Roadmap to 1.0 (GA)

See GA Requirements

Contributing

See CONTRIBUTING.md.

Approvers (@open-telemetry/java-instrumentation-approvers):

John Watson, Splunk
Mateusz Rzeszutek, Splunk
Pavol Loffay, Traceable.ai

Maintainers (@open-telemetry/java-instrumentation-maintainers):

Anuraag Agrawal, AWS
Nikita Salnikov-Tarnovski, Splunk
Trask Stalnaker, Microsoft
Tyler Benson, DataDog

Learn more about roles in the community repository.

Thanks to all the people who already contributed!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

OpenTelemetry Instrumentation for Java

Getting Started

Configuration parameters (subject to change!)

Exporters

OTLP exporter (both span and metric exporters)

Jaeger exporter

Zipkin exporter

Prometheus exporter

Logging exporter

Propagator

OpenTelemetry Resource

Peer service name

Batch span processor

Trace config

Interval metric reader

Customizing the OpenTelemetry SDK

Manually instrumenting

Maven

Gradle

Adding attributes to the current span

Creating spans around methods with `@WithSpan`

Suppressing `@WithSpan` instrumentation

Creating spans manually with a Tracer

Supported libraries, frameworks, and application servers

Disabled instrumentations

Grizzly instrumentation

Suppressing specific auto-instrumentation

Logger MDC auto-instrumentation

Troubleshooting

Roadmap to 1.0 (GA)

Contributing

Files

README.md

Latest commit

History

README.md

File metadata and controls

OpenTelemetry Instrumentation for Java

Getting Started

Configuration parameters (subject to change!)

Exporters

OTLP exporter (both span and metric exporters)

Jaeger exporter

Zipkin exporter

Prometheus exporter

Logging exporter

Propagator

OpenTelemetry Resource

Peer service name

Batch span processor

Trace config

Interval metric reader

Customizing the OpenTelemetry SDK

Manually instrumenting

Maven

Gradle

Adding attributes to the current span

Creating spans around methods with @WithSpan

Suppressing @WithSpan instrumentation

Creating spans manually with a Tracer

Supported libraries, frameworks, and application servers

Disabled instrumentations

Grizzly instrumentation

Suppressing specific auto-instrumentation

Logger MDC auto-instrumentation

Troubleshooting

Roadmap to 1.0 (GA)

Contributing

Creating spans around methods with `@WithSpan`

Suppressing `@WithSpan` instrumentation