---
title: Authentication, delivery, and failure handling with managed inputs
description: Learn how managed inputs share authentication, buffering and delivery, failure handling, and how network security applies on Serverless and Elastic Cloud Hosted.
url: https://www.elastic.co/elastic/docs-builder/docs/4614/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling
products:
  - Elastic Cloud Hosted
  - Elastic Cloud Serverless
  - Elastic Observability
  - Elastic Security
applies_to:
  - Elastic Cloud Serverless: Generally available
  - Elastic Cloud Hosted: Generally available
---

# Authentication, delivery, and failure handling with managed inputs
This page covers what all managed inputs share: how you authenticate, how data is stored for a limited time, how it is delivered, what happens when indexing fails, and how network security applies to your project or deployment. For protocol-specific setup, choose your endpoint in [Next steps](#next-steps).

## Authentication

Managed inputs authenticate with an Elasticsearch API key that includes the `event:write` privilege for the `apm` application. The same `event:write` / `apm` privilege applies to every managed endpoint, including the OTLP, Prometheus Remote Write, and _bulk endpoints.
<note>
  Index-level privilege scoping is not supported for managed inputs.
</note>

You can create this API key in one of the following ways:
<dropdown title="From the Add data page in Kibana" applies-to="Serverless Observability projects: Generally available, Elastic Cloud Hosted: Generally available">
  1. Select **Add data** in the navigation menu of your Serverless Observability project or Elastic Cloud Hosted deployment.
  2. In the **Connect directly to the endpoint** section, select the managed endpoint for which you want to create an API key.
  3. Select **Create key**, then copy the encoded value from the **API key** field.
</dropdown>

<dropdown title="From the API keys management page in Kibana">
  1. Go to the **API keys** management page in the navigation menu or use the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/explore-analyze/find-and-organize/find-apps-and-objects).
  2. Select **Create API key**, enter a name for the key, and enable **Control security privileges**.
  3. In the role descriptors box, enter the following privileges:
     ```json
     {
       "managed-inputs-writer": {
         "applications": [
           {
             "application": "apm",
             "resources": ["*"],
             "privileges": ["event:write"]
           }
         ]
       }
     }
     ```
  4. Select **Create API key** and copy the encoded value.
  For more details, refer to [Elasticsearch API keys](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/api-keys/elasticsearch-api-keys) and [Serverless project API keys](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/api-keys/serverless-project-api-keys).
</dropdown>

<dropdown title="Using the Create API key API">
  Use the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch-serverless/operation/operation-security-create-api-key). The same request works for both Elastic Cloud Serverless and Elastic Cloud Hosted. For example, to create an API key named `managed-inputs-api-key`:
  ```json

  {
    "name": "managed-inputs-api-key",
    "role_descriptors": {
      "managed-inputs-writer": {
        "applications": [
          {
            "application": "apm",
            "resources": ["*"],
            "privileges": ["event:write"]
          }
        ]
      }
    }
  }
  ```
</dropdown>

Send the encoded API key in the `Authorization` header of each request to the managed endpoint as `ApiKey <encoded-api-key>`. For example:
```http
Authorization: ApiKey <api-key>
```

Elastic Agent is the exception: its `api_key` output setting takes the key as `<id>:<api_key>` and encodes it itself, so the encoded value results in `401` errors. The **Add data** page provides only the encoded value. To get the key as `<id>:<api_key>`, create it on the Kibana **API keys** page and, in the **Created API key** panel, select **Beats** from the format dropdown before you copy the key. Alternatively, join the `id` and `api_key` fields from the Create API key API response with a colon.

## Buffering and delivery

Managed inputs buffer data before delivering it to Elasticsearch:
- Incoming data is stored (buffered) until it's delivered to Elasticsearch, and then it leaves the buffer. If Elasticsearch is lagging or temporarily unable to index, undelivered data can stay in the buffer for up to 12 hours. Delivered data isn't kept, so you can't replay it.
- A success response means the data was buffered, not yet indexed. Managed inputs retry delivery while Elasticsearch is temporarily unavailable or rejecting requests, which covers typical short interruptions. Retries and buffer retention are limited, so if Elasticsearch can't accept data for an extended period, for example during a prolonged outage or under sustained indexing pressure, data that couldn't be delivered in time is discarded. Because it never reached Elasticsearch, it isn't recorded in the failure store, and the success response your client already received isn't revised.
- Delivery time depends on Elasticsearch. Buffered data is indexed as fast as Elasticsearch accepts it, so the delay grows when Elasticsearch is slow or unavailable. Managed inputs don't expose ingestion health or delay metrics to you, so monitor the destination data streams to detect delays, especially during Elasticsearch maintenance.
- When capacity controls reject data, endpoints can respond with `429 Too Many Requests`, so clients should retry with backoff. Other temporary service failures can return `503 Service Unavailable`. Refer to [Managed inputs rate limiting](https://www.elastic.co/elastic/docs-builder/docs/4614/reference/opentelemetry/managed-inputs/rate-limiting).

<note>
  For the Managed Elasticsearch _bulk endpoint, documents with disallowed targets are rejected individually, and a `201` status on an item in the response means the document is enqueued, not indexed. For details, refer to [Delivery behavior](/elastic/docs-builder/docs/4614/reference/opentelemetry/managed-inputs/elasticsearch-bulk#delivery-behavior).
</note>


## Indexing errors and the failure store

To confirm your data was indexed, verify that documents landed in the destination data stream, and use [Data Set Quality](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/solutions/observability/data-set-quality-monitoring) to monitor and triage indexing issues.
A successful accept response from a managed input means the data was accepted for processing, not that Elasticsearch has indexed it. Indexing errors, such as mapping conflicts, malformed fields, or ingest pipeline errors, can happen asynchronously after the data is accepted, and aren't reported back to the client.
If the destination data stream has the [failure store](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/manage-data/data-store/data-streams/failure-store) enabled, documents that fail indexing are written there. Managed inputs don't enable or manage the failure store. You configure the failure store on the destination data stream.
<warning>
  If a document fails indexing and the destination data stream doesn't have the failure store enabled, the document is dropped. Because indexing errors happen after the data is accepted, your shipper still reports success, so this data loss is silent.
</warning>


## Network security

Support for IP filters and private connectivity on managed endpoints depends on whether you use a Elastic Cloud Serverless project or an Elastic Cloud Hosted deployment.
<applies-switch>
  <applies-item title="serverless:" applies-to="Elastic Cloud Serverless: Generally available">
    In Elastic Cloud Serverless projects, managed inputs follow the [network security policies](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/security/network-security-policies) applied to your project:
    - [IP filters](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/security/ip-filtering-cloud) apply to managed endpoints.
    - Managed endpoints are available over a private connection on supported cloud providers. For the current list, refer to [Private connectivity](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/security/private-connectivity).
  </applies-item>

  <applies-item title="ech:" applies-to="Elastic Cloud Hosted: Generally available">
    In Elastic Cloud Hosted deployments, the following limitations apply to managed inputs:
    - [IP filters](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/security/ip-filtering-cloud) do not apply to managed endpoints.
    - Managed endpoints are not available over a [private connection](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4614/deploy-manage/security/private-connectivity). When private connectivity is configured, the public managed endpoint is still available.
  </applies-item>
</applies-switch>


## Next steps

After you understand authentication and delivery, configure the endpoint for your protocol:
- [Ingest OpenTelemetry data with the Managed OTLP endpoint](https://www.elastic.co/elastic/docs-builder/docs/4614/reference/opentelemetry/managed-inputs/managed-otlp-endpoint)
- [Ingest Prometheus metrics with the Managed Prometheus Remote Write endpoint](https://www.elastic.co/elastic/docs-builder/docs/4614/reference/opentelemetry/managed-inputs/prometheus-remote-write)
- [Ingest data with the Managed Elasticsearch _bulk endpoint](https://www.elastic.co/elastic/docs-builder/docs/4614/reference/opentelemetry/managed-inputs/elasticsearch-bulk)