Distributed tracing

Elastic APM supports distributed tracing to enable you to analyze performance throughout your microservices architecture by tracing all requests — from the initial web request to your front-end service, to queries made to your back-end services — all in one view.

Distributed tracing is enabled by default in the RUM agent, however, it only includes requests made to the same origin. In order to include cross-origin requests, you must set the distributedTracingOrigins configuration option.

For example, consider an application that is served from: https://example.com. By default, all of the HTTP requests made to https://example.com will be included in the trace. To also include requests made to: https://api.example.com, you would need to add the following configuration:

var apm = initApm({
  ...
  distributedTracingOrigins: ['https://api.example.com']
})

This effectively tells the agent to add the distributed tracing HTTP header (traceparent) to requests made to https://api.example.com.

Tracestate can be used to provide additional vendor specific trace information across different tracing systems and is a companion header for the traceparent field.

By default, the RUM agent does not propagate the tracestate HTTP header to the configured origins. However, the user can change that behaviour by enabling the propagateTracestate flag which effectively adds tracestate HTTP header to the configured origins. As of today, only the sampling decision is propagated through the tracestate header. This information is then used by the APM server and the UI to calculate the service metrics like distributions and throughput (Service Maps).

The RUM agent is only one of the components in a distributed trace, so you must properly configure other components in order to use distributed tracing. In the example above, you need to make sure https://api.example.com can respond to requests that include the distributed tracing header. Specifically, https://api.example.com will receive an OPTIONS request with the following headers:

Access-Control-Request-Headers: traceparent, tracestate
Access-Control-Request-Method: [request-method]
Origin: [request-origin]

And should respond to it with these headers:

Access-Control-Allow-Headers: traceparent, tracestate
Access-Control-Allow-Methods: [allowed-methods]
Access-Control-Allow-Origin: [request-origin]

Starting in version 5.0, the RUM Agent supports the official W3C tracecontext traceparent header, instead of the previously used elastic-apm-traceparent header. Use the table below to determine which versions of our backend agents also support the official W3C tracecontext headers. Compatible agents use the official tracecontext spec to propagate traces and can therefor be used with the RUM Agent version >=5.0 for distributed tracing.

If your backend service generates an HTML page dynamically, you need to inject the trace ID and parent span ID into the page when you initialize the RUM Agent. This ensures that the web browser’s page load appears as the root of the trace. As an example:

var apm = initApm({
    ...
    pageLoadTraceId: <trace-id>,
    pageLoadSpanId: <span-id>,
    pageLoadSampled: <is-sampled>
})

The pageLoadSpanId should be set to the parent ID of the backend transaction. Most Elastic APM backend agents provide methods to extract this information. Please refer to the relevant Agent’s API for more information:

• .NET Agent: EnsureParentId()

• Java Agent: ensureParentId()

• Python Agent:

  • Flask integration
  • Django integration

• Ruby Agent: EnsureParent()

• Go Agent: EnsureParent()

• Node.js Agent distributed tracing guide