--- title: Elastic APM connector description: The Elastic {{product.apm}} connector is an OpenTelemetry Collector component that generates pre-aggregated APM metrics from trace data. url: https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/components/elasticapmconnector products: - Elastic Agent - Elastic Distribution of OpenTelemetry Collector - Elastic Observability applies_to: - Serverless Observability projects: Generally available - Elastic Stack: Generally available --- # Elastic APM connector The Elastic APM connector generates pre-aggregated metrics from OpenTelemetry trace data. These metrics enable key APM features like service maps, transaction histograms, and service-level indicators with fast query performance. Instead of calculating metrics on the fly from potentially millions of transactions, the APM UIs can query pre-computed metric documents for quick data visualization. The connector works together with the [Elastic APM processor](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/components/elasticapmprocessor), which enriches OpenTelemetry traces with Elastic-specific attributes to ensure optimal compatibility with Elastic APM UIs. ## Default usage in EDOT The `elasticapm` connector is included by default in EDOT Collector deployments that ingest trace data directly into Elasticsearch. It's not needed when using the [Elastic Cloud Managed OTLP Endpoint](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/opentelemetry/motlp), as the metric aggregation happens server-side. ### Standalone deployments In standalone deployments, the Elastic APM connector is used in both agent and gateway modes: - **Agent mode**: The connector is part of the default [application and traces collection pipeline](/elastic/docs-builder/docs/3028/reference/edot-collector/config/default-config-standalone#application-and-traces-collection-pipeline). It receives trace data from the pipeline, generates metrics, and forwards them to Elasticsearch. - **Gateway mode**: The connector is part of the [Gateway mode pipeline](/elastic/docs-builder/docs/3028/reference/edot-collector/config/default-config-standalone#gateway-mode), where it generates metrics from traces received from other collectors running in agent mode before ingesting them into Elasticsearch. ### Kubernetes deployment In Kubernetes, the Elastic APM connector runs in the [Gateway collectors pipeline](/elastic/docs-builder/docs/3028/reference/edot-collector/config/default-config-k8s#gateway-collectors-pipeline) when using direct ingestion to Elasticsearch. The Gateway receives traces from DaemonSet collectors, generates APM metrics, and writes both metrics and traces to Elasticsearch. For more details about the Kubernetes configuration, refer to [Default configuration (Kubernetes)](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/config/default-config-k8s). ## Example configuration The Elastic APM connector typically requires minimal configuration. Usually, an empty configuration block is sufficient: ```yaml connectors: elasticapm: {} ``` When combined with the `elasticapm` processor in a complete pipeline: ```yaml receivers: otlp: protocols: grpc: endpoint: 0.0.0.0:4317 http: endpoint: 0.0.0.0:4318 connectors: elasticapm: {} processors: batch: send_batch_size: 1000 timeout: 1s elasticapm: {} exporters: elasticsearch/otel: endpoints: - ${ELASTIC_ENDPOINT} api_key: ${ELASTIC_API_KEY} mapping: mode: otel service: pipelines: traces: receivers: [otlp] processors: [batch, elasticapm] exporters: [elasticapm, elasticsearch/otel] metrics/aggregated-otel-metrics: receivers: [elasticapm] processors: [] exporters: [elasticsearch/otel] ``` In this configuration, the `elasticapm` connector appears as both an exporter in the traces pipeline to generate APM metrics and as a receiver in the metrics pipeline to forward those metrics to Elasticsearch. ## Generated metrics The Elastic APM connector generates the following types of aggregated metrics from trace data: | Metric Type | Description | |-----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Transaction metrics | Aggregated statistics about service transactions, including throughput, latency distributions, and success rates. These metrics power service overview pages and transaction group views. | | Service destination metrics | Metrics that track dependencies between services, showing how services communicate with databases, message queues, and other external systems. These metrics are used to build service maps. | | Span metrics | Detailed metrics about individual span operations, including database queries, HTTP calls, and other operations. These provide granular performance insights for specific operations. | | Service summary metrics | High-level service health metrics including error rates, throughput, and overall latency. These metrics enable quick health checks and alerting. | All metrics are generated with appropriate aggregation periods and follow Elastic's metric schema for seamless integration with APM UIs. ## Aggregation intervals The connector aggregates metrics over multiple time intervals to provide different granularities for analysis. By default, metrics are aggregated at three intervals: 1 minute, 10 minutes, and 60 minutes. ## Best practices Follow these recommendations when using the Elastic APM connector: - **Always pair with the elasticapm processor**: The connector and processor work together to provide the full Elastic APM experience. The processor enriches traces while the connector generates APM metrics. Include both in your pipeline configuration for complete functionality. - **Place the connector as an exporter in the traces pipeline**: Configure the Elastic APM connector as an exporter in your traces pipeline, alongside your final data destination. This ensures the connector receives processed trace data and can generate accurate metrics. - **Create a separate metrics pipeline for connector output**: Set up a dedicated metrics pipeline with the connector as the receiver. This isolates metric handling and makes it easier to apply metric-specific processing if needed. - **Use only for direct Elasticsearch ingestion**: If you're using the Elastic Cloud Managed OTLP Endpoint, you don't need the Elastic APM connector, because the endpoint handles metric aggregation automatically. Using both can cause conflicts or duplicate metrics. - **Keep the connector updated**: The Elastic APM connector evolves with new Elastic APM features. Keep your EDOT Collector version current to benefit from the latest enhancements and compatibility improvements. - **Monitor connector performance**: The connector aggregates metrics in memory before flushing. For high-throughput environments, monitor memory usage and adjust the aggregation interval or deployment resources as needed. ## Limitations Be aware of these constraints and behaviors when using the Elastic APM connector: - **Required for Elastic APM metrics**: Without the Elastic APM connector, you'll only have raw trace data in Elasticsearch. Service maps, transaction histograms, and other metric-driven APM features might require the pre-aggregated metrics that this connector generates. - **Not available in contrib OTel Collector**: The Elastic APM connector is an Elastic-specific component not included in the standard OpenTelemetry Collector or Collector Contrib distributions. To use it, you must either use EDOT Collector or [build a custom collector](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/custom-collector) that includes Elastic's components. - **Memory usage scales with cardinality**: The connector maintains in-memory aggregations for unique combinations of service names, transaction names, and other dimensions. High-cardinality data (many unique values) increases memory requirements. Monitor memory usage in high-cardinality environments. - **Minimal configuration options**: Unlike some connectors, the Elastic APM connector operates with mostly fixed behavior and offers few configuration parameters. While this simplifies setup, it also means you have limited ability to customize the aggregation logic. ## Troubleshooting Read the following sections to troubleshoot issues with the Elastic APM connector. If you detect unexpected spikes in storage usage for APM metrics, high-cardinality data is often the cause. The connector aggregates metrics across multiple time intervals, and the volume of aggregated metrics is directly proportional to the cardinality of your data. The more unique combinations of service names, transaction names, and other dimensions, the more metric documents are produced.High cardinality in aggregations often points to an instrumentation issue, such as a field with many unique values that shouldn't vary. For example, including user IDs or request IDs in transaction names. #### SolutionTo limit the cardinality of aggregations and reduce storage usage, you can configure limits lower than the defaults. The connector supports four cardinality limits: | Limit | Description | Default | |-------------|---------------------------------------------------|---------| | `resource` | Maximum cardinality of resources | 8000 | | `scope` | Maximum cardinality of scopes within a resource | 4000 | | `metric` | Maximum cardinality of metrics within a scope | 4000 | | `datapoint` | Maximum cardinality of datapoints within a metric | 4000 | Here's an example configuration that halves all the default cardinality limits: ```yaml connectors: elasticapm: aggregation: limits: resource: max_cardinality: 4000 scope: max_cardinality: 2000 metric: max_cardinality: 2000 datapoint: max_cardinality: 2000 ``` When configured limits are reached, additional metrics are placed into a separate overflow bucket. This bounds the resources consumed, but if overflow occurs frequently, it usually indicates an instrumentation problem that should be addressed at the source. #### Detecting overflowTo check if overflow is occurring, look for overflow-related log messages from the connector. Frequent overflow events suggest that your cardinality limits are being exceeded regularly, which might affect the completeness of your APM metrics. If overflow is happening consistently, consider: 1. Investigating the source of high-cardinality data and fixing the instrumentation. 2. Lowering the `max_cardinality` settings as a temporary measure to bound resource usage. Lowering cardinality limits should be a last resort after confirming that high cardinality is expected for your use case and cannot be reduced through better instrumentation practices. ## Resources - [Contrib component: elasticapmconnector](https://github.com/elastic/opentelemetry-collector-components/tree/main/connector/elasticapmconnector) - [Elastic APM processor](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/components/elasticapmprocessor) - [Default configuration (Standalone)](/elastic/docs-builder/docs/3028/reference/edot-collector/config/default-config-standalone#application-and-traces-collection-pipeline) - [Default configuration (Kubernetes)](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/config/default-config-k8s) - [Build a custom collector with Elastic components](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/edot-collector/custom-collector)