Managed inputs rate limiting

Rate limiting occurs when a managed input receives data faster than it can process and index into Elasticsearch. The endpoint responds with HTTP 429 errors until the data volume is reduced.

Rate limiting behavior differs by deployment type:

• Elastic Cloud Hosted: Rate limits depend on your Elasticsearch cluster capacity. If your cluster can't keep up with incoming data, the endpoint starts rejecting requests with 429 errors.
• Elastic Cloud Serverless: Elastic manages scaling automatically. Rate limiting is rare and typically indicates a temporary event to protect our system.

When rate limiting occurs, the managed input responds with an HTTP 429 Too Many Requests status code.

For the Managed OTLP Endpoint, a log message similar to this can appear in the OpenTelemetry Collector's output:

"error": "rpc error: code = ResourceExhausted desc = request exceeded available capacity"
		

For OTLP troubleshooting steps, refer to Error: too many requests.

Elasticsearch output shippers such as Beats, Elastic Agent, and Logstash typically retry 429 responses automatically with backoff. Configure your shipper to queue data so transient rejections don't lose data.

For Elastic Cloud Hosted deployments, 429 errors typically indicate that your Elasticsearch cluster is undersized for the current data volume. If AutoOps is available in your region, use it to check CPU utilization, index queue depth, and node load to confirm whether your cluster is under-resourced. If AutoOps is not available in your region, contact Elastic Support.

If metrics confirm the cluster needs more capacity, scale your deployment:

• Scaling considerations
• Resize deployment
• Autoscaling in ECE and ECH

Once your Elasticsearch capacity is scaled up or is able to accept the incoming data volume, requests to managed inputs will be accepted again.

For Elastic Cloud Serverless projects, Elastic manages scaling automatically. If you experience persistent 429 errors, contact Elastic Support.