Distributed tracing

2.0.2

Tracing is enabled thanks to the integration with Spring Cloud Sleuth. Please check the Readme of that project for more information. I'll paste part of it here for readability purposes.

Terminology

Spring Cloud Sleuth borrows Dapper’s terminology.

Span: The basic unit of work. For example, sending an RPC is a new span, as is sending a response to an RPC. Span’s are identified by a unique 64-bit ID for the span and another 64-bit ID for the trace the span is a part of. Spans also have other data, such as descriptions, key-value annotations, the ID of the span that caused them, and process ID’s (normally IP address).

Spans are started and stopped, and they keep track of their timing information. Once you create a span, you must stop it at some point in the future.

Trace: A set of spans forming a tree-like structure. For example, if you are running a distributed big-data store, a trace might be formed by a put request.

Configuration

If you're using Spring Boot you'll get tracing configuration out of the box.

Features

• Data will be sent to Twitter Zipkin ONLY if the profile is equal to production and APP_ENV variable DOES NOT CONTAIN either rbt or stage or test. You can disable this check by setting tracing.properties.enabled to false
• Spans are set for
  • all HTTP requests
  • Camel Integration (you have to add micro-infra-camel module)
  • Hystrix integration (use CorrelatedCommand)
  • ServiceRestClient calls
  • @Async annotated methods
  • @Scheduled annotated methods
  • Runnables (use com.ofg.infrastructure.tracing.SpanRemovingRunnable) like this new SpanRemovingRunnable(trace.wrap(myRunnable))
  • Callables (use com.ofg.infrastructure.tracing.SpanRemovingCallable) like this new SpanRemovingCallable(trace.wrap(myCallable))
  • any public controller method returning WebAsyncTask

What is crucial is that this feature is backwards compatible. That means that correlationId header is recognized and will be reused as the new traceId functionality. This means that it will be mapped to the X-Trace-Id header. Also we will emit both correlationId and X-Trace-Id headers together.

Logging and metrics

Everything related to metrics stays as it did.

There is a change in the logging approach since the pattern changed:

'%d{yyyy-MM-dd HH:mm:ss.SSSZ, Europe/Warsaw} | %-5level | %X{X-Trace-Id} | %thread | %logger{1} | %m%n'

What changed is that there was the correlationId MDC entry and now it's X-Trace-Id. Please ensure that your Logback configuration can parse both MDC entries.

x.x.x -> 2.0.1

Description

We are working with microservices. Many microservices. Imagine a series of 20 microservices processing one request - let's say that we want to grant a loan to a Mr Smith. Since we are professionals we have log collecting tools like logstash and kibana. Now imagine that something broke - an exception occurred. How can you find in those hundreds of lines of logs which ones are related to Mr Smith's case? Correlation id will speed up your work effortlessly.

Since we are using Spring then most likely we can receive or send a request by

• a @Controller annotated controller
• a @RestController annotated controller
• by sending a request via a RestTemplate

To deal with all of those approaches we:

• created a filter CorrelationIdFilter that will set a correlation id header named correlationId on your request
• created an aspect CorrelationIdAspect that makes it possible to work with Servlet 3.0 async feature (you have to have a controller method that returns a Callable)
• the very same aspect allows checks if you are using a RestOperations- base interface of RestTemplate. If that is the case then we are setting the correlationId header on the request that you are sending (via exchange method).

Hystrix

Additionally we support Hystrix. CorrelatedCommand class allows propagating correlation ID through Hystrix commands. Because Hystrix uses separate thread pools, correlation ID will get lost. In order to take advantage of request tracking across multiple systems, replace built-in HystrixCommand with CorrelatedCommand:

new CorrelatedCommand<String>(withGroupKey(asKey("KEY"))) {
    String doRun() {
        //your code here
    }
}.execute();

Module configuration

If you want to use only this module just add a dependency:

repositories {
    jcenter()
}

dependencies {
    compile 'com.ofg:micro-infra-hystrix:0.8.2'
}

and enable this feature via annotation:

@Configuration
@EnableCorrelationId
class MyWebAppConfiguration {

}