Loading

Ingest Prometheus metrics with managed inputs

The Managed Prometheus Remote Write endpoint ingests metrics sent in the Prometheus Remote Write v1 (PRW) protocol. It accepts PRW traffic natively, so you don't need to convert metrics to OTLP, and it's a dedicated managed input separate from the Managed OTLP Endpoint. Metrics land in Elasticsearch time series data streams (TSDS), the same result as sending PRW directly to Elasticsearch.

For Elastic Cloud Serverless projects, the Managed Prometheus Remote Write endpoint is the recommended way to ingest Prometheus metrics. Compared to sending metrics directly to the Elasticsearch Prometheus Remote Write endpoint, it provides:

  • A single ingest endpoint and API key shared with the other managed inputs.
  • Durable buffering, back-pressure, and retry on 429 Too Many Requests.
  • The same Prometheus-to-TSDS mapping as the Elasticsearch PRW endpoint, so the resulting data is identical.
Warning

On Elastic Cloud Serverless, use the Managed Prometheus Remote Write endpoint rather than sending metrics directly to the Elasticsearch Prometheus Remote Write endpoint.

Direct ingest bypasses managed inputs and has no buffering or processing before data reaches Elasticsearch. It also authenticates differently: the direct Elasticsearch endpoint uses Elasticsearch credentials or an API key with index privileges, while the Managed Prometheus Remote Write endpoint uses a managed inputs API key with the event:write privilege for the apm application. Use the direct Elasticsearch endpoint only for self-managed deployments, where managed inputs aren't available. Elastic Cloud Hosted support for the Managed Prometheus Remote Write endpoint is planned.

  • An Elastic Cloud Serverless Observability or Security project.
  • A managed inputs API key with the event:write privilege for the apm application. Refer to Authentication for the required key format and generation steps.

Follow these steps to configure Prometheus to send metrics to the Managed Prometheus Remote Write endpoint.

  1. Configure Prometheus

    Add a remote_write entry to your Prometheus configuration:

    remote_write:
  - url: <prometheus-endpoint>
    authorization:
      type: ApiKey
      credentials: <api-key>

    To find <prometheus-endpoint>:

    1. Log in to the Elastic Cloud Console.
    2. Find your project and select Manage.
    3. In the Application endpoints, cluster and component IDs section, select Prometheus.
    4. Copy the Prometheus endpoint value, which looks similar to https://<project>.ingest.<region>.<csp>.elastic.cloud/api/v1/write

  2. Route metrics to custom data streams (optional)

    By default, all PRW metrics land in metrics-generic.prometheus-default.

    To route to a custom data stream, attach the data_stream_dataset and data_stream_namespace labels to your time series:

    Label Sets Example
    data_stream_dataset Dataset component of the data stream name myapp
    data_stream_namespace Namespace component of the data stream name production

    A time series with data_stream_dataset: myapp and data_stream_namespace: production routes to metrics-myapp.prometheus-production.

    In Prometheus, use write_relabel_configs to add these labels to every time series sent to a remote_write target:

    remote_write:
  - url: <prometheus-endpoint>
    authorization:
      type: ApiKey
      credentials: <api-key>
    write_relabel_configs:
      - target_label: data_stream_dataset
        replacement: myapp
      - target_label: data_stream_namespace
        replacement: production

Prometheus labels are mapped as TSDS dimensions in Elasticsearch, and metric types are inferred from field naming conventions:

  • Fields ending in _sum, _count, _total, or _bucket are mapped as counters.
  • All other fields are mapped as gauges.

For details on the full mapping behavior, refer to the Elasticsearch Prometheus remote write endpoint documentation.

  • URL-path routing (for example, /_prometheus/metrics/{dataset}/api/v1/write) to custom data streams is not supported through managed inputs. Use label-based routing instead.
  • Available on Elastic Cloud Serverless only.
  • Samples with non-finite values (NaN, Infinity) are silently dropped by Elasticsearch, and staleness markers are not supported.