---
title: Split an event into multiple documents
description: Split one incoming event into multiple Elasticsearch documents, either at collection time or with a Logstash pipeline that uses the split filter.
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8110/manage-data/ingest/transform-enrich/split-events-into-multiple-documents
products:
  - Elastic Agent
  - Logstash
applies_to:
  - Elastic Cloud Serverless: Generally available
  - Elastic Stack: Generally available
---

# Split an event into multiple documents
Sometimes a single incoming event contains multiple records. For example, an API response or a batched TCP payload might hold an array of items, and you want to index each item as its own document in Elasticsearch.
Elastic Agent processors and Elasticsearch ingest pipelines process each event individually: they can transform or drop an event, but they can't generate multiple documents from a single event.
To split one event into multiple documents, you have two options:
- [Split events at collection time](#split-events-at-collection-time), if your data source's input supports it.
- [Split events with Logstash](#split-events-with-logstash), for everything else.

Most data sources don't need splitting. Elastic Integrations are designed to deliver one document per record. Splitting mainly matters for custom and input-only integrations, such as a Custom TCP Logs integration that receives batched JSON payloads.
<note>
  The Elasticsearch [`split` processor](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/ingest-processor/split-processor) splits a field value into an array within the same document. It doesn't create new documents.
</note>


## Splitting compared to rerouting

Splitting turns one event into many documents. Rerouting sends each document to a different destination based on its content. If your goal is to route documents to different data streams or indices, you don't need to split, and you don't need Logstash. Instead, use the Elasticsearch [`reroute` processor](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/ingest-processor/reroute-processor), typically in a [`@custom` ingest pipeline](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8110/reference/fleet/data-streams-pipeline-tutorial).

## Split events at collection time

Some Elastic Agent and Filebeat inputs can split an incoming payload into separate events as the data is collected, before anything reaches Elasticsearch. If your data source uses one of these inputs, this is the easiest option as it requires no extra components and no changes to how your data flows.
For example:
- The [Custom API integration](https://docs-v3-preview.elastic.dev/elastic/integration-docs/tree/main/reference/httpjson) and the underlying [HTTP JSON input](https://docs-v3-preview.elastic.dev/elastic/beats/tree/main/reference/filebeat/filebeat-input-httpjson) can split API responses into separate events with the `response.split` setting.
- The [CEL Custom API input integration](https://docs-v3-preview.elastic.dev/elastic/integration-docs/tree/main/reference/cel) can emit multiple events per request when the CEL program returns a list of events.
- The [AWS S3 input](https://docs-v3-preview.elastic.dev/elastic/beats/tree/main/reference/filebeat/filebeat-input-aws-s3) can create one event per element of a JSON array with the `expand_event_list_from_field` setting.

Check your input's reference documentation for similar settings.

## Split events with Logstash

When your input has no native split option, run a Logstash pipeline between Elastic Agent and Elasticsearch. The Logstash [`split` filter](https://docs-v3-preview.elastic.dev/elastic/logstash-docs-md/tree/main/lsr/plugins-filters-split) creates a copy of the event for each element of an array field, so each element becomes its own document.
If your data flows through an Elastic integration, pair the `split` filter with the [`elastic_integration` filter](https://docs-v3-preview.elastic.dev/elastic/logstash-docs-md/tree/main/lsr/plugins-filters-elastic_integration), which runs the integration's ingest pipeline inside Logstash. This keeps the integration's processing intact while letting you split the results. For more details about this pattern, refer to [Using Logstash with Elastic integrations](https://docs-v3-preview.elastic.dev/elastic/logstash/tree/main/reference/using-logstash-with-elastic-integrations).
<warning>
  Splitting multiplies your document count. An event with hundreds of array elements becomes hundreds of documents, which can significantly increase ingest volume, storage, and indexing load. Test with realistic data volumes before you rely on splitting in production.
</warning>


### Requirements

- A [Logstash instance](https://docs-v3-preview.elastic.dev/elastic/logstash/tree/main/reference) that your Elastic Agents can reach.
- Elastic Agents [configured to output to Logstash](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8110/reference/fleet/logstash-output).
- If you use the `elastic_integration` filter: the required [subscription](https://www.elastic.co/subscriptions), and credentials with the `monitor`, `read_pipeline`, and `manage_index_templates` privileges on your Elasticsearch cluster. Refer to the [plugin documentation](https://docs-v3-preview.elastic.dev/elastic/logstash-docs-md/tree/main/lsr/plugins-filters-elastic_integration) for details.


### Example pipeline

This pipeline receives events from Elastic Agent, runs the integration's ingest pipeline, and then splits each event on the `items` array field:
```text
input {
  elastic_agent {
    port => 5044
  }
}

filter {
  elastic_integration { 
    cloud_id => "<cloud-id>"
    api_key => "<api-key>"
  }

  split { 
    field => "items"
  }
}

output {
  elasticsearch { 
    cloud_id => "<cloud-id>"
    api_key => "<api-key>"
    data_stream => true
    ecs_compatibility => "v8"
  }
}
```