---
title: Ingest data with the Managed Elasticsearch _bulk endpoint
description: Ingest log data from Elasticsearch _bulk shippers through the Elastic Cloud Managed Elasticsearch _bulk endpoint, and decide when to choose direct Elasticsearch instead.
url: https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/elasticsearch-bulk
products:
  - Elastic Cloud Hosted
  - Elastic Cloud Serverless
  - Elastic Observability
  - Elastic Security
applies_to:
  - Serverless Observability projects: Generally available
  - Serverless Security projects: Generally available
  - Elastic Cloud Hosted: Generally available
---

# Ingest data with the Managed Elasticsearch _bulk endpoint
The Managed Elasticsearch _bulk endpoint ingests data sent in the [Elasticsearch `_bulk` API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) format. It accepts `_bulk` traffic natively, so shippers that already write to Elasticsearch can send data through [managed inputs](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs) by pointing their existing Elasticsearch output at the endpoint. It's a dedicated managed input exposed on the `/_es` path of the same ingest host as the [Managed OTLP Endpoint](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/managed-otlp-endpoint) and the [Managed Prometheus Remote Write endpoint](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/prometheus-remote-write).
The endpoint is Elasticsearch-compatible: it emulates a subset of the `_bulk` API, so most shippers need only a new endpoint and credentials to start sending data. The managed input buffers the data and then indexes it into Elasticsearch asynchronously. Only log data is supported, and bulk actions must use the `create` action.

## When to use the Managed Elasticsearch _bulk endpoint

The Managed Elasticsearch _bulk endpoint isn't a drop-in replacement for sending `_bulk` requests directly to Elasticsearch. It supports only part of the `_bulk` API: It accepts only the `create` action, and requests that use `index`, `update`, or `delete` fail with `400 Bad Request`. Because the managed endpoint buffers data before indexing it, it also acknowledges data earlier, reports failures differently, and can add latency. Use the following sections to decide which endpoint fits your workload.

### When to send data to the managed endpoint

- **You already use other managed inputs.** Your `_bulk` shippers can send data to the same ingest host as the [Managed OTLP Endpoint](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/managed-otlp-endpoint) and the [Managed Prometheus Remote Write endpoint](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/prometheus-remote-write), using the same API key.
- **Your ingest volume is spiky.** The managed endpoint absorbs short spikes well above the usual rate, so you don't need to size every shipper's output for peak load.
  On Elastic Cloud Hosted, volume that your cluster can't keep up with still results in `429` responses. Refer to [Managed inputs rate limiting](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/rate-limiting).
- **You want to limit local buffering on your shippers.** The managed endpoint buffers data while Elasticsearch is slow or unavailable, so your shippers need less local buffering. They still need to retry `429` and `503` responses, and a delay in indexing must be acceptable.

If you switch an existing shipper to the managed endpoint, compare document counts between what your shipper sends and the destination data stream for a period after the switch, and monitor the destination with [Data Set Quality](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4572/solutions/observability/data-set-quality-monitoring).

### When to send data directly to Elasticsearch

- **You send data other than logs, or write logs to targets that don't start with `logs-`.** The managed endpoint supports only log data in `logs-*` data streams. Refer to [Limitations](#limitations).
- **Your Elastic Agents are managed by Fleet.** Fleet-managed outputs can't use the managed endpoint.
- **You need confirmation that data was indexed.** Your shipper must know that data was indexed before it marks the data as sent. The managed endpoint returns success when data is buffered. Delivery to Elasticsearch completes later, and data that can't be delivered before the buffer retention period ends is discarded. Refer to [Delivery behavior](#delivery-behavior).
- **You need predictable latency**, for example for alerting. The managed endpoint can add latency, and the delay grows when Elasticsearch is slow or unavailable.
- **You monitor ingestion with shipper metrics.** With the managed endpoint, those metrics tell you whether the endpoint accepted the data, not whether it was indexed. Managed inputs don't expose ingestion health or throughput metrics to you.
- **You need control over how data is batched and retried into Elasticsearch.** Batch size, flush interval, retries, and worker count are fixed on the managed side.
- **You use IP filters or private connectivity on Elastic Cloud Hosted.** IP filters don't apply to the managed endpoint, and it isn't available over private connections. Refer to [Network security](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#network-security) for more information.

For the full list of constraints, refer to [Limitations](#limitations).

## Prerequisites

- An Elastic Cloud Serverless Observability or Security project, or an Elastic Cloud Hosted deployment on Elastic Stack version 9.0 or later.
- A `_bulk`-compatible shipper that sends `create` actions to log data streams, such as standalone Elastic Agent, Logstash, or another shipper with an Elasticsearch output.
- An API key with the `event:write` privilege for the `apm` application. Refer to [Authentication](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#authentication) for the required key format and generation steps.
- Any Elasticsearch index templates and Kibana assets your shipper relies on, installed beforehand. The endpoint doesn't install them for you. Refer to [Limitations](#limitations).


## Set up the Managed Elasticsearch _bulk endpoint

The Managed Elasticsearch _bulk endpoint uses the same ingest host as the [Managed OTLP Endpoint](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/managed-otlp-endpoint), with the `/_es` path appended. Its exact format depends on your deployment type. Copy the exact value from the Elastic Cloud Console, as described in the following steps.
<stepper>
  <step title="Find your endpoint">
    To find your Managed Elasticsearch _bulk endpoint:
    <applies-switch>
      <applies-item title="serverless:" applies-to="Elastic Cloud Serverless: Generally available">
        1. Log in to the Elastic Cloud Console.
        2. Find your project and select **Manage**.
        3. In the **Application endpoints** section, select **Elasticsearch**. A flyout opens that lists the endpoints for each application as expandable sections, with **Elasticsearch** expanded.
        4. Copy the Elasticsearch **Endpoint** value. Don't use the **Direct endpoint** value, which is the endpoint for sending data directly to Elasticsearch.
      </applies-item>

      <applies-item title="ech:" applies-to="Elastic Cloud Hosted: Generally available">
        1. Log in to the Elastic Cloud Console.
        2. Find your deployment in **Hosted deployments** and select **Manage**.
        3. In the **Application endpoints, cluster and component IDs** section, select **Elasticsearch**. A flyout opens that lists the endpoints for each application as expandable sections, with **Elasticsearch** expanded.
        4. Copy the Elasticsearch **Endpoint** value. Don't use the **Direct endpoint** value, which is the endpoint for sending data directly to Elasticsearch.
      </applies-item>
    </applies-switch>
    This endpoint value is shown as `<managed-_bulk-endpoint>` in the examples that follow. Your shipper appends the `_bulk` path (and any target index path) itself, so requests are sent to `POST <managed-_bulk-endpoint>/_bulk`.
  </step>

  <step title="Authenticate">
    Create an API key suitable for managed inputs as described in [Authentication](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#authentication). Send this key in each request's `Authorization` header as `ApiKey <api-key>`:
    ```http
    Authorization: ApiKey <api-key>
    ```
    Shippers that call `GET /_es` or `GET /_es/_license` at startup can use the same API key. You don't need to grant additional privileges for those requests.
  </step>

  <step title="Configure your shipper and send data">
    To send data, configure your shipper's Elasticsearch output with the following:
    - **Endpoint**: Use your Managed Elasticsearch _bulk endpoint value (`<managed-_bulk-endpoint>`) in the host format for your deployment type.
      - If your shipper has a separate path or URL-prefix setting, set it to `/_es` instead of including it in the host.
    - The Logstash and Elastic Agent `elasticsearch` outputs use port `9200` when the host URL omits a port, so add `:443` before the `/_es` path, for example `https://<host>:443/_es`.
    - **Authentication**: The API key you created for the managed inputs. How you pass it depends on the key format you have:
      - If you have the encoded key, for example from the **Add data** page in Kibana, send it as the HTTP header `Authorization: ApiKey <api-key>`. This works with any shipper. Refer to [Authentication](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#authentication) for the ways to create a key.
    - If you have the key as `<id>:<api_key>`, use the `api_key` setting in Logstash or standalone Elastic Agent.
    - **Action**: Use `create`. Shippers that write to data streams already use this action.
    The following examples show the configuration for Logstash and standalone Elastic Agent. Setting names and the exact way to pass credentials vary by shipper and version, so validate the configuration for your shipper.
    <dropdown title="Logstash example">
      Configure the `elasticsearch` output:
      ```ruby
      output {
        elasticsearch {
          hosts => ["<managed-_bulk-endpoint>"] 
          api_key => "<id>:<api_key>"
          action => "create"
        }
      }
      ```
      When you create the API key on the Kibana **API keys** page, select **Logstash** from the format dropdown in the **Created API key** panel before you copy it, or join the `id` and `api_key` fields from the Create API key API response with a colon.If you only have the encoded key, for example if you generated it from the **Add data** page, replace the `api_key` setting with `custom_headers` and pass the key in the `Authorization` header:
      ```ruby
      custom_headers => {
        "Authorization" => "ApiKey <api-key>"
      }
      ```

      <warning>
        By default, Logstash indefinitely retries documents that the endpoint rejects with a `403` status, which can stall the pipeline. The endpoint rejects documents this way when their target isn't allowed. Refer to [Delivery behavior](#delivery-behavior).If the dead letter queue is enabled, add `403` to `dlq_custom_codes` to send these documents there. To drop them instead, add `security_exception` to `drop_error_types`.
      </warning>
    </dropdown>

    <dropdown title="Standalone Elastic Agent example">
      Configure the `elasticsearch` output. Elastic Agent encodes the `<id>:<api_key>` value itself, so the encoded value from the **Add data** page results in `401` errors. When you create the key on the Kibana **API keys** page, select **Beats** from the format dropdown in the **Created API key** panel before you copy it, or join the `id` and `api_key` fields from the Create API key API response with a colon:
      ```yaml
      outputs:
        default:
          type: elasticsearch
          hosts: ["<managed-_bulk-endpoint>"] 
          api_key: "<id>:<api_key>"
      ```
    </dropdown>
    If your shipper uses the `index`, `update`, or `delete` action, switch to `create`. Refer to [Limitations](#limitations) for more information.To confirm your setup is working, check that your shipper reports successful (`2xx`) responses, with no authentication errors and no documents rejected with a `403` status. Then open **Discover** and verify that new documents are landing in your target data stream.If documents don't appear, they might have failed during asynchronous indexing. These errors aren't reported in the bulk response, so use [Data Set Quality](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4572/solutions/observability/data-set-quality-monitoring) to monitor and triage indexing issues. For more details, refer to [Indexing errors and the failure store](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#failure-store).
  </step>
</stepper>


## How _bulk data appears in Elasticsearch

Each action in a `_bulk` request can specify its target through the `_index` field, so data lands in the data stream that your shipper already targets. For example, a shipper writing nginx access logs to `logs-nginx.access-default` continues to land there. If your shipper sets a fallback target in the request path (`/_es/<target>/_bulk`), that target is used for actions that omit `_index`. Only log data streams are supported targets. Refer to [Limitations](#limitations) for more information.

## Delivery behavior

The Managed Elasticsearch _bulk endpoint emulates the Elasticsearch `_bulk` API, but because it ingests through managed inputs, it behaves differently from indexing directly into Elasticsearch. Keep the following in mind:
- **Documents with disallowed targets are rejected individually.** A document whose target doesn't start with `logs-`, `metrics-`, or `traces-` gets a `403` status with a `security_exception` error in its response item, and the response sets `errors: true`. The other documents in the request are still accepted. The response is `200 OK` even if every document is rejected, so check `errors` and the status of each item. For allowed targets, refer to [Limitations](#limitations).
- **Rejected documents are handled by your shipper.** Elastic Agent drops a rejected document and logs a warning. Logstash retries it indefinitely by default. To change this, refer to the Logstash example in [Ingest data with the Managed Elasticsearch _bulk endpoint > Configure your shipper and send data](#configure-your-shipper-and-send-data).
- **Accepted documents are enqueued together.** The endpoint either enqueues all accepted documents in a request, or fails the entire request. If they can't be enqueued, the whole request fails with `503 Service Unavailable`, and no document is reported as successful. Malformed requests, unsupported actions, or missing targets fail with `400 Bad Request`, and nothing in the request is enqueued.
- **A success response means the data is enqueued, not indexed.** A successful response returns an Elasticsearch-compatible body in which each accepted item reports a `201` status. This confirms the managed input accepted the document, not that Elasticsearch has indexed it. Errors that occur later during indexing, such as mapping conflicts, happen asynchronously and aren't reported in the bulk response. Your shipper counts these documents as sent.
- **Undelivered data is discarded.** The endpoint retries delivery while Elasticsearch is briefly unavailable or rejecting requests. If Elasticsearch still can't accept the data when buffer retention ends, the endpoint discards it. The success response your shipper already received doesn't change.
- **Delivery delay isn't visible to your shipper.** A direct `_bulk` request slows down or fails when Elasticsearch is under pressure, so your shipper can retry or back off. The managed endpoint returns success as soon as the batch is enqueued, even if delivery to Elasticsearch is delayed.
- **`require_data_stream` and `require_alias` are ignored.** The endpoint doesn't enforce these query parameters, so they don't protect you from writing to an unintended target the way they do with Elasticsearch, although targets outside the allowed prefixes are rejected anyway.
- **Compressed requests are supported.** The endpoint accepts `Content-Encoding: gzip` request bodies.

For how long undelivered data can stay in the buffer, and what happens to it after that, refer to [Buffering and delivery](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#delivery).

## Indexing errors and rate limiting

Because indexing happens asynchronously, indexing failures such as mapping conflicts aren't reported in the bulk response, and the endpoint doesn't provide client-side visibility into them. To confirm your data was indexed, verify that documents landed in the destination data stream and use [Data Set Quality](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4572/solutions/observability/data-set-quality-monitoring) to monitor indexing issues. For more detail on how indexing errors are handled, refer to [Indexing errors and the failure store](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#failure-store).
Under load, or when the service can't accept more data, the endpoint can respond with `429 Too Many Requests` or `503 Service Unavailable`. Configure your shipper to retry these responses with backoff and to queue data locally during transient rejections. For how rate limiting works and how it differs between Elastic Cloud Serverless and Elastic Cloud Hosted, refer to [Managed inputs rate limiting](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/rate-limiting).

## Limitations

The following limitations apply when using the Managed Elasticsearch _bulk endpoint:
- **Only log data is supported.** It must target `logs-*` data streams, for example `logs-my_dataset-default`. The endpoint accepts only targets that start with `logs-`, `metrics-`, or `traces-`, and rejects any other target, including wired streams such as `logs.otel` and system indices, with a `403` status in the response item. `metrics-*` and `traces-*` targets are accepted but not supported. To send metrics and traces, use the direct Elasticsearch endpoint. For OpenTelemetry data, use the [Managed OTLP Endpoint](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/managed-otlp-endpoint).
- **Beats aren't supported with their default settings.** They write to `<beat>-*` indices such as `filebeat-*` by default, which the endpoint rejects with a `403` status for each document.
- **Only `create` actions are supported.** Requests that use `index`, `update`, or `delete` actions are rejected with `400 Bad Request`. Features that depend on other actions, such as scripted upserts, aren't supported.
- **`require_data_stream` and `require_alias` aren't enforced.** Refer to [Delivery behavior](#delivery-behavior).
- **Duplicate detection isn't applied.** The endpoint doesn't deduplicate documents by `_id`, so client retries can produce duplicate documents.
- **Indexing outcomes aren't reported to your shipper.** The bulk response confirms that data was enqueued, not indexed, so errors such as mapping conflicts never reach your shipper. Managed inputs don't expose ingestion health metrics either, so monitor indexing with [Data Set Quality](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/4572/solutions/observability/data-set-quality-monitoring).
- **Delivery settings on the managed side aren't configurable.** Batching, flush intervals, retries, and worker counts are fixed, unlike the equivalent settings of a shipper's Elasticsearch output.
- **Fleet-managed Elastic Agent outputs can't use the endpoint.** Fleet issues its agents Elasticsearch API keys with index privileges, which the endpoint doesn't accept. Only shippers you configure yourself, such as standalone Elastic Agent and Logstash, can use an API key for managed inputs.
- **Index templates, index lifecycle management (ILM) policies, and Kibana assets can't be installed through the endpoint.** It serves only the root (`/_es`), license (`/_es/_license`), and `_bulk` paths (`/_es/_bulk` and `/_es/<target>/_bulk`). Elastic Agent and Logstash setup steps that create index templates or load dashboards must run against Elasticsearch and Kibana directly before you send data.
- **IP filters and private connectivity depend on your deployment type.** For how they apply to all managed inputs, refer to [Network security](/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling#network-security).


## Related pages

- [Authentication, delivery, and failure handling with managed inputs](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/authentication-delivery-and-failure-handling): Shared authentication, buffering and delivery, and indexing-error handling across all managed inputs.
- [Managed inputs rate limiting](https://www.elastic.co/elastic/docs-builder/docs/4572/reference/opentelemetry/managed-inputs/rate-limiting): How `429` responses work and how capacity limits differ between Elastic Cloud Serverless and Elastic Cloud Hosted.