--- title: Contrib OpenTelemetry Collectors and language SDKs description: The Elastic Stack natively supports the OpenTelemetry protocol (OTLP). This means logs, metrics, and trace data collected from your applications and infrastructure... url: https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/upstream-opentelemetry-collectors-language-sdks products: - APM - Elastic Cloud Serverless - Elastic Observability applies_to: - Elastic Cloud Serverless: Generally available - Elastic Stack: Generally available --- # Contrib OpenTelemetry Collectors and language SDKs The Elastic Stack natively supports the OpenTelemetry protocol (OTLP). This means logs, metrics, and trace data collected from your applications and infrastructure can be sent directly to the Elastic Stack. - Send data to Elastic from a contrib [OpenTelemetry Collector](#apm-connect-open-telemetry-collector) - Send data to Elastic from a contrib [OpenTelemetry language SDK](#apm-instrument-apps-otel) To compare approaches and choose the best approach for your use case, refer to [OpenTelemetry](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry). The Elastic Distribution of OpenTelemetry Collector (EDOT Collector) include additional features and configurations to seamlessly integrate with Elastic. Refer to [EDOT compared to contrib OpenTelemetry](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/opentelemetry/compatibility/edot-vs-upstream) for a comparison. ## Send data from a contrib OpenTelemetry Collector Connect your OpenTelemetry Collector instances to Elastic Observability or Elastic Observability Serverless using the OTLP exporter: ```yaml receivers: # ... otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 processors: # ... memory_limiter: check_interval: 1s limit_mib: 2000 batch: exporters: debug: verbosity: detailed otlp: # Elastic APM server https endpoint without the "https://" prefix endpoint: "${env:ELASTIC_APM_SERVER_ENDPOINT}" <5> headers: # Elastic APM Server secret token Authorization: "Bearer ${env:ELASTIC_APM_SECRET_TOKEN}" <6> service: pipelines: traces: receivers: [otlp] processors: [..., memory_limiter, batch] exporters: [debug, otlp] metrics: receivers: [otlp] processors: [..., memory_limiter, batch] exporters: [debug, otlp] logs: receivers: [otlp] processors: [..., memory_limiter, batch] exporters: [debug, otlp] ``` ```yaml receivers: # ... otlp: processors: # ... memory_limiter: check_interval: 1s limit_mib: 2000 batch: exporters: logging: loglevel: warn otlp/elastic: # Elastic https endpoint without the "https://" prefix endpoint: "${ELASTIC_APM_SERVER_ENDPOINT}" <5> headers: # Elastic API key Authorization: "ApiKey ${ELASTIC_APM_API_KEY}" <6> service: pipelines: traces: receivers: [otlp] processors: [..., memory_limiter, batch] exporters: [logging, otlp/elastic] metrics: receivers: [otlp] processors: [..., memory_limiter, batch] exporters: [logging, otlp/elastic] logs: receivers: [otlp] processors: [..., memory_limiter, batch] exporters: [logging, otlp/elastic] ``` You’re now ready to export traces and metrics from your services and applications. When using the OpenTelemetry Collector, send data through the [`OTLP` exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter). Using other methods, like the [`elasticsearch` exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter), bypasses all of the validation and data processing that Elastic performs. In addition, your data will not be viewable in your Observability project if you use the `elasticsearch` exporter. ## Send data from a contrib OpenTelemetry SDK The following instructions show how to send data directly from a contrib OpenTelemetry SDK to Elastic, which is appropriate when getting started. However, sending data from an OpenTelemetry SDK to the OpenTelemetry Collector is preferred, as the Collector processes and exports data to Elastic. Read more about when and how to use a collector in the [OpenTelemetry documentation](https://opentelemetry.io/docs/collector/#when-to-use-a-collector). To export traces and metrics to Elastic, instrument your services and applications with the OpenTelemetry API, SDK, or both. For example, if you are a Java developer, you need to instrument your Java app with the [OpenTelemetry agent for Java](https://github.com/open-telemetry/opentelemetry-java-instrumentation). See the [OpenTelemetry Instrumentation guides](https://opentelemetry.io/docs/instrumentation/) to download the OpenTelemetry agent or SDK for your language. Define environment variables to configure the OpenTelemetry agent or SDK and enable communication with Elastic APM. For example, if you are instrumenting a Java app, define the following environment variables: ```bash export OTEL_RESOURCE_ATTRIBUTES=service.name=checkoutService,service.version=1.1,deployment.environment=production export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200 export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer an_apm_secret_token" export OTEL_METRICS_EXPORTER="otlp" \ export OTEL_LOGS_EXPORTER="otlp" \ java -javaagent:/path/to/opentelemetry-javaagent-all.jar \ -classpath lib/*:classes/ \ com.mycompany.checkout.CheckoutServiceServer ``` Fields that describe the service and the environment that the service runs in. See [attributes](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/attributes) for more information. APM Server URL. The host and port that APM Server listens for events on. Authorization header that includes the Elastic APM Secret token or API key: `"Authorization=Bearer an_apm_secret_token"` or `"Authorization=ApiKey an_api_key"`. For information on how to format an API key, see [API keys](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/api-keys). Note the required space between `Bearer` and `an_apm_secret_token`, and `ApiKey` and `an_api_key`. If you are using a version of the Python OpenTelemetry agent *before* 1.27.0, the content of the header *must* be URL-encoded. You can use the Python standard library’s `urllib.parse.quote` function to encode the content of the header. Metrics exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. Logs exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. ```bash export OTEL_RESOURCE_ATTRIBUTES=service.name=checkoutService,service.version=1.1,deployment.environment=production export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200 export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey an_apm_api_key" export OTEL_METRICS_EXPORTER="otlp" \ export OTEL_LOGS_EXPORTER="otlp" \ java -javaagent:/path/to/opentelemetry-javaagent-all.jar \ -classpath lib/*:classes/ \ com.mycompany.checkout.CheckoutServiceServer ``` Fields that describe the service and the environment that the service runs in. See [attributes](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/attributes) for more information. Elastic URL. The host and port that Elastic listens for APM events on. Authorization header that includes the Elastic APM API key: `"Authorization=ApiKey an_api_key"`. Note the required space between `ApiKey` and `an_api_key`. For information on how to format an API key, refer to [Secure communication with APM agents](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/use-apm-securely). If you are using a version of the Python OpenTelemetry agent *before* 1.27.0, the content of the header *must* be URL-encoded. You can use the Python standard library’s `urllib.parse.quote` function to encode the content of the header. Metrics exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. Logs exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. You are now ready to collect traces and [metrics](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/collect-metrics) before [verifying metrics](/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/collect-metrics#apm-open-telemetry-verify-metrics) and [visualizing metrics](/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/collect-metrics#apm-open-telemetry-visualize). ## Proxy requests to APM Server - : Preview APM Server supports both the [OTLP/gRPC](https://opentelemetry.io/docs/specs/otlp/#otlpgrpc) and [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol on the same port as Elastic APM agent requests. For ease of setup, use OTLP/HTTP when proxying or load balancing requests to Elastic. If you use the OTLP/gRPC protocol, requests to Elastic must use either HTTP/2 over TLS or HTTP/2 Cleartext (H2C). No matter which protocol is used, OTLP/gRPC requests will have the header: `"Content-Type: application/grpc"`. When using a layer 7 (L7) proxy like AWS ALB, proxy the requests in a way that ensures they follow the rules outlined previously. For example, with ALB you can create rules to select an alternative backend protocol based on the headers of requests coming into ALB. In this example, you’d select the gRPC protocol when the `"Content-Type: application/grpc"` header exists on a request. Many L7 load balancers handle HTTP and gRPC traffic separately and rely on explicitly defined routes and service configurations to correctly proxy requests. Since APM Server serves both protocols on the same port, it may not be compatible with some L7 load balancers. For example, to work around this issue in [Ingress NGINX Controller for Kubernetes](https://github.com/kubernetes/ingress-nginx), either: - Use the `otlp` exporter in the EDOT collector. Set annotation `nginx.ingress.kubernetes.io/backend-protocol: "GRPC"` on the K8s Ingress object proxying to APM Server. - Use the `otlphttp` exporter in the EDOT collector. Set annotation `nginx.ingress.kubernetes.io/backend-protocol: "HTTP"` (or `"HTTPS"` if APM Server expects TLS) on the K8s Ingress object proxying to APM Server. The preferred approach is to deploy a L4 (TCP) load balancer (for example, [NLB](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) on AWS) in front of APM Server, which forwards raw TCP traffic transparently without protocol inspection. For more information on how to configure an AWS ALB to support gRPC, see this AWS blog post: [Application Load Balancer Support for End-to-End HTTP/2 and gRPC](https://aws.amazon.com/blogs/aws/new-application-load-balancer-support-for-end-to-end-http-2-and-grpc/). For more information on how APM Server services gRPC requests, see [Muxing gRPC and HTTP/1.1](https://github.com/elastic/apm-server/blob/main/dev_docs/otel.md#muxing-grpc-and-http11). In Elastic Cloud Hosted, the _APM Server_ receives data from Elastic APM agents and transforms it into Elasticsearch documents. In Elastic Cloud Serverless there is in fact no APM Server running, instead the _managed intake service_ receives and transforms data. ## Next steps - [Collect metrics](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/collect-metrics) - Add [resource attributes](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/attributes) - Learn about the [limitations of this integration](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/apm/opentelemetry/limitations)