﻿---
title: Authentication, delivery, and failure handling with managed inputs
description: Learn how managed inputs share authentication, buffering and delivery, failure handling, and Elastic Cloud Hosted network limitations.
url: https://docs-v3-preview.elastic.dev/elastic/opentelemetry/tree/main/reference/managed-inputs/authentication-delivery-and-failure-handling
products:
  - Elastic Cloud Hosted
  - Elastic Cloud Serverless
  - Elastic Observability
  - Elastic Security
applies_to:
  - Elastic Cloud Serverless: Generally available
  - Elastic Cloud Hosted: Generally available
---

# Authentication, delivery, and failure handling with managed inputs
This page covers what all managed inputs share: how you authenticate, how data is stored for a limited time, how it is delivered, what happens when indexing fails, and network limitations on Elastic Cloud Hosted. For protocol-specific setup, choose your endpoint in [Next steps](#next-steps).

## Authentication

Managed inputs authenticate with an Elasticsearch API key that includes the `event:write` privilege for the `apm` application. The same `event:write` / `apm` privilege applies to every managed endpoint, including the OTLP, Prometheus Remote Write, and _bulk endpoints.
You can create this API key in one of the following ways:
<dropdown title="From the Add data flow" applies-to="Serverless Observability projects: Generally available, Elastic Cloud Hosted: Generally available">
  1. Select **Add data** in the navigation menu of your Serverless Observability project or Elastic Cloud Hosted deployment.
  2. In the **Connect directly to the endpoint** section, select the managed endpoint for which you want to create an API key.
  3. Select **Create key**, then copy the encoded value from the **API key** field.
</dropdown>

<dropdown title="From the API keys management page in Kibana">
  1. Go to the **API keys** management page in the navigation menu or use the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/explore-analyze/find-and-organize/find-apps-and-objects).
  2. Select **Create API key**, enter a name for the key, and enable **Control security privileges**.
  3. In the role descriptors box, enter the following privileges:
     ```json
     {
       "managed-inputs-writer": {
         "applications": [
           {
             "application": "apm",
             "resources": ["*"],
             "privileges": ["event:write"]
           }
         ]
       }
     }
     ```
  4. Select **Create API key** and copy the encoded value.
  For more details, refer to [Elasticsearch API keys](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/deploy-manage/api-keys/elasticsearch-api-keys) and [Serverless project API keys](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/deploy-manage/api-keys/serverless-project-api-keys).
</dropdown>

<dropdown title="Using the Create API key API">
  Use the [Create API key API](https://www.elastic.co/docs/api/doc/elasticsearch-serverless/operation/operation-security-create-api-key). The same request works for both Elastic Cloud Serverless and Elastic Cloud Hosted. For example, to create an API key named `managed-inputs-api-key`:
  ```json

  {
    "name": "managed-inputs-api-key",
    "role_descriptors": {
      "managed-inputs-writer": {
        "applications": [
          {
            "application": "apm",
            "resources": ["*"],
            "privileges": ["event:write"]
          }
        ]
      }
    }
  }
  ```
</dropdown>

Send the encoded API key in the `Authorization` header of each request to the managed endpoint as `ApiKey <encoded-api-key>`. For example:
```http
Authorization: ApiKey <api-key>
```

<note>
  Index-level privilege scoping is not supported for managed inputs.
</note>


## Buffering and delivery

Managed inputs provide a durable ingest layer in front of Elasticsearch:
- Incoming data is stored (buffered) in a durable ingest layer before it reaches your Elasticsearch cluster. Buffered data is held for a limited time before it must be delivered.
- When capacity controls reject data, endpoints can respond with `429 Too Many Requests`, so clients should retry with backoff. Other temporary service failures can return `503 Service Unavailable`. Refer to [Managed inputs rate limiting](https://docs-v3-preview.elastic.dev/elastic/opentelemetry/tree/main/reference/managed-inputs/rate-limiting).

<note>
  For the Managed Elasticsearch _bulk endpoint, a batch is atomic: the endpoint accepts or rejects the whole batch, and a `201` per item means the data is durably enqueued, not indexed. For details, refer to [Delivery behavior](/elastic/opentelemetry/tree/main/reference/managed-inputs/elasticsearch-bulk#delivery-behavior).
</note>


## Indexing errors and the failure store

To confirm your data was indexed, verify that documents landed in the destination data stream, and use [Data Set Quality](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/solutions/observability/data-set-quality-monitoring) to monitor and triage indexing issues.
A successful accept response from a managed input means the data was durably accepted for processing, not that Elasticsearch has indexed it. Indexing errors, such as mapping conflicts or ingest pipeline errors, can happen asynchronously after the data is accepted, and aren't reported back to the client.
Managed inputs don't enable or manage the [failure store](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/manage-data/data-store/data-streams/failure-store). The failure store is an Elasticsearch data stream setting. If the destination data stream has it enabled, documents that fail indexing are written there. If it isn't enabled, those documents aren't captured.
<warning>
  If a document fails indexing and the destination data stream doesn't have the failure store enabled, the document is dropped. Because indexing errors happen after the data is accepted, your shipper still reports success, so this data loss is silent.
</warning>


## Elastic Cloud Hosted limitations

<applies-to>
  - Elastic Cloud Hosted: Generally available
</applies-to>

In Elastic Cloud Hosted deployments, the following limitations apply to managed inputs:
- [IP filters](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/deploy-manage/security/ip-filtering-cloud) do not apply to managed endpoints.
- Managed endpoints are not available over a [private connection](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/deploy-manage/security/private-connectivity). When private connectivity is configured, the public managed endpoint is still available.


## Next steps

After you understand authentication and delivery, configure the endpoint for your protocol:
- [Ingest OpenTelemetry data with the Managed OTLP endpoint](https://docs-v3-preview.elastic.dev/elastic/opentelemetry/tree/main/reference/managed-inputs/managed-otlp-endpoint)
- [Ingest Prometheus metrics with the Managed Prometheus Remote Write endpoint](https://docs-v3-preview.elastic.dev/elastic/opentelemetry/tree/main/reference/managed-inputs/prometheus-remote-write)
- [Ingest data with the Managed Elasticsearch _bulk endpoint](https://docs-v3-preview.elastic.dev/elastic/opentelemetry/tree/main/reference/managed-inputs/elasticsearch-bulk)