Quickstart: Ingest custom metrics with Elastic OpenTelemetry

Use this quickstart to send custom metrics to Elastic using the Elastic Agent.

You’ll install a lightweight Elastic Agent, configure a minimal OpenTelemetry Protocol (OTLP) metrics pipeline, and verify the data in Elastic Observability.

  • An Elastic deployment (Serverless, Elastic Cloud Hosted, or self-managed)
  • An Observability project or Kibana instance
  • Permissions to create API keys
  • A system to run the Elastic Agent (Docker, host, or VM)
  • Optional: An application that emits OpenTelemetry metrics

  1. Create an Elastic API key

    In your Elastic Observability deployment:

    1. Go to Management → Stack Management → API keys.
    2. Create a new API key and copy the value.
    3. Note your deployment's OTLP ingest endpoint.

  2. Run the Elastic Agent with a minimal metrics pipeline

    Create a collector-config.yaml file with the following Collector configuration to receive OTLP metrics and export them to Elastic:

    receivers:
  otlp:
    protocols:
      http:
      grpc:

processors:
  batch: {}

exporters:
 otlphttp:
   endpoint: "<OTLP_ENDPOINT>"
   headers:
     Authorization: "ApiKey <YOUR_API_KEY>"

service:
 pipelines:
   metrics:
     receivers: [otlp]
     processors: [batch]
     exporters: [otlphttp]

    Run the configuration, for example with Docker:

    docker run --rm \
  -e ELASTIC_AGENT_OTEL=true \
  -v $(pwd)/collector-config.yaml:/etc/otel/config.yaml \
  -p 4317:4317 -p 4318:4318 \
  elastic/elastic-agent:9.5.5 \
  --config /etc/otel/config.yaml

  3. Optional: Port conflict handling

    If you encounter a port conflict error like:

    bind: address already in use

    Add the following to the service section:

    service:
 telemetry:
   metrics:
     address: localhost:8889
    1. Using different port if 8888 is already in use

    You can also verify if the Collector is listening on the correct ports:

    lsof -i :4318 -i :4317

  4. Send a custom metric

    In this Python example, you use an application that emits OTLP metrics. For other languages, refer to the contrib OpenTelemetry documentation.

    from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader

exporter = OTLPMetricExporter(endpoint="http://localhost:4318/v1/metrics")
reader = PeriodicExportingMetricReader(exporter)
provider = MeterProvider(metric_readers=[reader])
metrics.set_meter_provider(provider)

meter = metrics.get_meter("custom-app")

temperature = meter.create_observable_gauge(
    "custom.temperature",
    callbacks=[lambda options: [metrics.Observation(26.7)]],
)

input("Sending metrics periodically... press Enter to stop")

  5. Verify metrics in Elastic Observability

    In Kibana:

    1. Find Discover in the navigation menu or use the global search field.

    2. Select ES|QL to switch to ES|QL mode, then run a TS query to select your metrics data:

      TS metrics-*

    3. Search the chart grid for custom.temperature, then break the metric down by dimension or add its chart to a dashboard.

    To learn how to explore metrics in Discover, refer to Explore metrics data with Discover in Kibana.

    1. Find Infrastructure in the navigation menu or use the global search field, then open Metrics Explorer.
    2. Search for custom.temperature.
    3. Visualize or aggregate the metric data.

    For the Metrics Explorer workflow, refer to Explore infrastructure metrics over time.

You've successfully set up a minimal OTLP metrics pipeline with the Elastic Agent. Your custom metrics are flowing into Elastic Observability and can be visualized in Kibana.

Now you can:

  • Build dashboards from your custom metrics
  • Set up alerts based on your custom metrics
  • Aggregate and analyze metric trends over time

You can expand your metrics collection setup in several ways:

  • Add more receivers to collect additional metrics
  • Configure the same Collector to send logs and traces alongside metrics

To learn more, refer to the Elastic Agent documentation.