---
type: api
title: Simulate data ingestion
description: **All methods and paths for this operation:**
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-simulate-ingest
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-simulate-ingest
products:
  - Elasticsearch
---

# Simulate data ingestion


Availability: 8.12+

## Paths

- `POST` (also `GET`) `/_ingest/{index}/_simulate`
- `POST` (also `GET`) `/_ingest/_simulate`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#apikeyauth)
- [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#basicauth)
- [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#bearerauth)

#### Path Parameters

- `index` (string IndexName) — optional, leave it out to use `/_ingest/_simulate`
  The index to simulate ingesting into.
This value can be overridden by specifying an index on each document.
If you specify this parameter in the request path, it is used for any documents that do not explicitly specify an index argument.

## Description

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request.
If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would.
No data is indexed into Elasticsearch.
Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation.
The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline.
The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used.
However, you can supply substitute pipeline definitions in the body of the request.
These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

## Required authorization

* Index privileges: `index`

## Query String Parameters

- `pipeline` (string)
  The pipeline to use as the default pipeline.
This value can be used to override the default pipeline of the index.

## Request Body

`application/json`

- `docs` ([] Document) — required
  Sample documents to test in the pipeline.
  - `_id` (string Id) — optional
    Unique identifier for the document.
This ID must be unique within the `_index`.
  - `_index` (string IndexName) — optional
    Name of the index containing the document.
  - `_source` (object) — required
    JSON body for the document.
- `component_template_substitutions` (map string to ComponentTemplateNode) — optional
  A map of component template names to substitute component template definition objects.
  Map values ({} ComponentTemplateNode)
  - `template` ({} ComponentTemplateSummary) — required
    - `_meta` (Metadata) — optional
    - `version` (number VersionNumber) — optional
    - `settings` (map string to IndexSettings) — optional
    - `mappings` ({} TypeMapping) — optional
      - `all_field` ({} AllField) — optional
        - `analyzer` (string) — required
        - `enabled` (boolean) — required
        - `omit_norms` (boolean) — required
        - `search_analyzer` (string) — required
        - `similarity` (string) — required
        - `store` (boolean) — required
        - `store_term_vector_offsets` (boolean) — required
        - `store_term_vector_payloads` (boolean) — required
        - `store_term_vector_positions` (boolean) — required
        - `store_term_vectors` (boolean) — required
      - `date_detection` (boolean) — optional
      - `dynamic` (enum DynamicMapping) — optional
        Values: `strict`, `runtime`, `true`, `false`
      - `dynamic_date_formats` ([] string) — optional
      - `dynamic_templates` ([] string to DynamicTemplate) — optional
      - `_field_names` ({} FieldNamesField) — optional
        - `enabled` (boolean) — required
      - `index_field` ({} IndexField) — optional
        - `enabled` (boolean) — required
      - `_meta` (Metadata) — optional
      - `numeric_detection` (boolean) — optional
      - `properties` (map string to Property) — optional
      - `_routing` ({} RoutingField) — optional
        - `required` (boolean) — required
      - `_size` ({} SizeField) — optional
        - `enabled` (boolean) — required
      - `_source` ({} SourceField) — optional
        - `compress` (boolean) — optional
        - `compress_threshold` (string) — optional
        - `enabled` (boolean) — optional
        - `excludes` ([] string) — optional
        - `includes` ([] string) — optional
        - `mode` (enum SourceFieldMode) — optional
          

Supported values include:
  - `disabled`
  - `stored`
  - `synthetic`: Instead of storing source documents on disk exactly as you send them,
 Elasticsearch can reconstruct source content on the fly upon retrieval.
          Values: `disabled`, `stored`, `synthetic`
      - `runtime` (map string to RuntimeField) — optional
        Map values ({} RuntimeField)
        - `fields` (map string to CompositeSubField) — optional
          For type `composite`
          Map values ({} CompositeSubField)
          - `type` (enum RuntimeFieldType) — required
            Values: `boolean`, `composite`, `date`, `double`, `geo_point`, `geo_shape`, `ip`, `keyword`, `long`, `lookup`
        - `fetch_fields` ([] RuntimeFieldFetchFields) — optional
          For type `lookup`
          - `field` (string Field) — required
          - `format` (string) — optional
        - `format` (string) — optional
          A custom format for `date` type runtime fields.
        - `input_field` (string Field) — optional
          For type `lookup`
        - `target_field` (string Field) — optional
          For type `lookup`
        - `target_index` (string IndexName) — optional
          For type `lookup`
        - `script` (string Script) — optional
          Painless script executed at query time.
        - `on_script_error` (enum OnScriptError) — optional
          Values: `fail`, `continue`
        - `type` (enum RuntimeFieldType) — required
          Field type, which can be: `boolean`, `composite`, `date`, `double`, `geo_point`, `ip`,`keyword`, `long`, or `lookup`.
          Values: `boolean`, `composite`, `date`, `double`, `geo_point`, `geo_shape`, `ip`, `keyword`, `long`, `lookup`
      - `enabled` (boolean) — optional
      - `subobjects` (enum Subobjects) — optional
        Values: `true`, `false`
      - `_data_stream_timestamp` ({} DataStreamTimestamp) — optional
        - `enabled` (boolean) — required
    - `aliases` (map string to AliasDefinition) — optional
      Map values ({} AliasDefinition)
      - `filter` ({} QueryContainer) — optional
        Query used to limit documents the alias can access.
        See [QueryContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/types/_types-query_dsl-querycontainer)
      - `index_routing` (string) — optional
        Value used to route indexing operations to a specific shard.
If specified, this overwrites the `routing` value for indexing operations.
      - `is_write_index` (boolean · default: false) — optional
        If `true`, the index is the write index for the alias.
      - `routing` (string) — optional
        Value used to route indexing and search operations to a specific shard.
      - `search_routing` (string) — optional
        Value used to route search operations to a specific shard.
If specified, this overwrites the `routing` value for search operations.
      - `is_hidden` (boolean · default: false) — optional
        If `true`, the alias is hidden.
All indices for the alias must have the same `is_hidden` value.
    - `lifecycle` ({} DataStreamLifecycleWithRollover) — optional
      - `data_retention` (string Duration) — optional
        If defined, every document added to this data stream will be stored at least for this time frame.
Any time after this duration the document could be deleted.
When empty, every document in this data stream will be stored indefinitely.
      - `effective_retention` (string Duration) — optional
        The least amount of time data should be kept by elasticsearch.
      - `retention_determined_by` (enum RetentionSource) — optional
        Configuration source that can influence the retention of a data stream.
        Values: `data_stream_configuration`, `default_global_retention`, `max_global_retention`, `default_failures_retention`
      - `downsampling` ([] DownsamplingRound) — optional
        The list of downsampling rounds to execute as part of this downsampling configuration
        - `after` (string Duration) — required
          The duration since rollover when this downsampling round should execute
        - `config` ({} DownsampleConfig) — required
          The downsample configuration to execute.
          - `fixed_interval` (string) — required
            The interval at which to aggregate the original time series index.
      - `enabled` (boolean · default: true) — optional
        If defined, it turns data stream lifecycle on/off (`true`/`false`) for this data stream. A data stream lifecycle
that's disabled (enabled: `false`) will have no effect on the data stream.
      - `rollover` ({} DataStreamLifecycleRolloverConditions) — optional
        The conditions which will trigger the rollover of a backing index as configured by the cluster setting `cluster.lifecycle.default.rollover`.
This property is an implementation detail and it will only be retrieved when the query param `include_defaults` is set to true.
The contents of this field are subject to change.
        - `min_age` (string Duration) — optional
        - `max_age` (string) — optional
        - `min_docs` (number) — optional
        - `max_docs` (number) — optional
        - `min_size` (number | string ByteSize) — optional
        - `max_size` (number | string ByteSize) — optional
        - `min_primary_shard_size` (number | string ByteSize) — optional
        - `max_primary_shard_size` (number | string ByteSize) — optional
        - `min_primary_shard_docs` (number) — optional
        - `max_primary_shard_docs` (number) — optional
    - `data_stream_options` ({} DataStreamOptions) — optional
      - `failure_store` ({} DataStreamFailureStore) — optional
        If defined, it specifies configuration for the failure store of this data stream.
        - `enabled` (boolean · default: true) — optional
          If defined, it turns the failure store on/off (`true`/`false`) for this data stream. A data stream failure store
that's disabled (enabled: `false`) will redirect no new failed indices to the failure store; however, it will
not remove any existing data from the failure store.
        - `lifecycle` ({} FailureStoreLifecycle) — optional
          If defined, it specifies the lifecycle configuration for the failure store of this data stream.
          - `data_retention` (string Duration) — optional
            If defined, every document added to this data stream will be stored at least for this time frame.
Any time after this duration the document could be deleted.
When empty, every document in this data stream will be stored indefinitely.
          - `enabled` (boolean · default: true) — optional
            If defined, it turns data stream lifecycle on/off (`true`/`false`) for this data stream. A data stream lifecycle
that's disabled (enabled: `false`) will have no effect on the data stream.
  - `version` (number VersionNumber) — optional
  - `_meta` (Metadata) — optional
  - `deprecated` (boolean) — optional
- `index_template_substitutions` (map string to IndexTemplate) — optional
  A map of index template names to substitute index template definition objects.
  Map values ({} IndexTemplate)
  - `index_patterns` (string Names) — required
    Array of wildcard (`*`) expressions used to match the names of data streams and indices during creation.
  - `composed_of` ([] string Name) — required
    An ordered list of component template names.
Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
  - `template` ({} IndexTemplateSummary) — optional
    Template to be applied.
It may optionally include an `aliases`, `mappings`, or `settings` configuration.
    - `aliases` (map string to Alias) — optional
      Aliases to add.
If the index template includes a `data_stream` object, these are data stream aliases.
Otherwise, these are index aliases.
Data stream aliases ignore the `index_routing`, `routing`, and `search_routing` options.
      Map values ({} Alias)
      - `filter` ({} QueryContainer) — optional
        Query used to limit documents the alias can access.
        See [QueryContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/types/_types-query_dsl-querycontainer)
      - `index_routing` (string Routing) — optional
        Value used to route indexing operations to a specific shard.
If specified, this overwrites the `routing` value for indexing operations.
      - `is_hidden` (boolean · default: false) — optional
        If `true`, the alias is hidden.
All indices for the alias must have the same `is_hidden` value.
      - `is_write_index` (boolean · default: false) — optional
        If `true`, the index is the write index for the alias.
      - `routing` (string Routing) — optional
        Value used to route indexing and search operations to a specific shard.
      - `search_routing` (string Routing) — optional
        Value used to route search operations to a specific shard.
If specified, this overwrites the `routing` value for search operations.
    - `mappings` ({} TypeMapping) — optional
      Mapping for fields in the index.
If specified, this mapping can include field names, field data types, and mapping parameters.
      Same fields as `mappings` in `template`
    - `settings` (IndexSettings) — optional
      Configuration options for the index.
    - `lifecycle` ({} DataStreamLifecycleWithRollover) — optional
      Same fields as `lifecycle` in `template`
    - `data_stream_options` ({} DataStreamOptions) — optional
      - `failure_store` ({} DataStreamFailureStore) — optional
        If defined, it specifies configuration for the failure store of this data stream.
        - `enabled` (boolean · default: true) — optional
          If defined, it turns the failure store on/off (`true`/`false`) for this data stream. A data stream failure store
that's disabled (enabled: `false`) will redirect no new failed indices to the failure store; however, it will
not remove any existing data from the failure store.
        - `lifecycle` ({} FailureStoreLifecycle) — optional
          If defined, it specifies the lifecycle configuration for the failure store of this data stream.
          - `data_retention` (string Duration) — optional
            If defined, every document added to this data stream will be stored at least for this time frame.
Any time after this duration the document could be deleted.
When empty, every document in this data stream will be stored indefinitely.
          - `enabled` (boolean · default: true) — optional
            If defined, it turns data stream lifecycle on/off (`true`/`false`) for this data stream. A data stream lifecycle
that's disabled (enabled: `false`) will have no effect on the data stream.
  - `version` (number VersionNumber) — optional
    Version number used to manage index templates externally.
This number is not automatically generated by Elasticsearch.
  - `priority` (number) — optional
    Priority to determine index template precedence when a new data stream or index is created.
The index template with the highest priority is chosen.
If no priority is specified the template is treated as though it is of priority 0 (lowest priority).
This number is not automatically generated by Elasticsearch.
  - `_meta` (Metadata) — optional
    Optional user metadata about the index template. May have any contents.
This map is not automatically generated by Elasticsearch.
  - `allow_auto_create` (boolean) — optional
  - `data_stream` ({} IndexTemplateDataStreamConfiguration) — optional
    If this object is included, the template is used to create data streams and their backing indices.
Supports an empty object.
Data streams require a matching index template with a `data_stream` object.
    - `hidden` (boolean · default: false) — optional
      If true, the data stream is hidden.
    - `allow_custom_routing` (boolean · default: false) — optional
      If true, the data stream supports custom routing.
  - `deprecated` (boolean) — optional
    Marks this index template as deprecated.
When creating or updating a non-deprecated index template that uses deprecated components,
Elasticsearch will emit a deprecation warning.
  - `ignore_missing_component_templates` (string Names) — optional
    A list of component template names that are allowed to be absent.
- `mapping_addition` ({} TypeMapping) — optional
  Same fields as `mappings` in `template`
- `pipeline_substitutions` (map string to Pipeline) — optional
  Pipelines to test.
If you don’t specify the `pipeline` request path parameter, this parameter is required.
If you specify both this and the request path parameter, the API only uses the request path parameter.
  Map values ({} Pipeline)
  - `description` (string) — optional
    Description of the ingest pipeline.
  - `on_failure` ([] {} ProcessorContainer) — optional
    Processors to run immediately after a processor failure.
    See [ProcessorContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/types/ingest-_types-processorcontainer)
  - `processors` ([] {} ProcessorContainer) — optional
    Processors used to perform transformations on documents before indexing.
Processors run sequentially in the order specified.
    See [ProcessorContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/types/ingest-_types-processorcontainer)
  - `version` (number VersionNumber) — optional
    Version number used by external systems to track ingest pipelines.
  - `deprecated` (boolean · default: false) — optional
    Marks this ingest pipeline as deprecated.
When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
  - `_meta` (Metadata) — optional
    Arbitrary metadata about the ingest pipeline. This map is not automatically generated by Elasticsearch.

## Response

`application/json`

- `docs` ([] SimulateIngestDocumentResult)
  - `doc` ({} IngestDocumentSimulation)
    - `_id` (string Id)
      Identifier for the document.
    - `_index` (string IndexName)
      Name of the index that the document would be indexed into if this were not a simulation.
    - `_source` (map string to object)
      JSON body for the document.
    - `_version` (StringifiedVersionNumber)
    - `executed_pipelines` ([] string)
      A list of the names of the pipelines executed on this document.
    - `ignored_fields` ([] string to string)
      A list of the fields that would be ignored at the indexing step. For example, a field whose
value is larger than the allowed limit would make it through all of the pipelines, but
would not be indexed into Elasticsearch.
    - `error` ({} ErrorCause)
      Any error resulting from simulatng ingest on this doc. This can be an error generated by
executing a processor, or a mapping validation error when simulating indexing the resulting
doc.
      - `type` (string)
        The type of error
      - `reason` (union string) — nullable
        A human-readable explanation of the error, in English.
        One of: `string`
      - `stack_trace` (string)
        The server stack trace. Present only if the `error_trace=true` parameter was sent with the request.
      - `caused_by` ({} ErrorCause) — recursive
        Same fields as `error` in `doc`
      - `root_cause` ([] ErrorCause) — recursive
        Same fields as `error` in `doc`
      - `suppressed` ([] ErrorCause) — recursive
        Same fields as `error` in `doc`
## Code Examples

#### Console

```console
POST /_ingest/_simulate
{
  "docs": [
    {
      "_id": "123",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "456",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
```

#### JavaScript

```javascript
const response = await client.simulate.ingest({
  docs: [
    {
      _id: "123",
      _index: "my-index",
      _source: {
        foo: "bar",
      },
    },
    {
      _id: "456",
      _index: "my-index",
      _source: {
        foo: "rab",
      },
    },
  ],
});
```

#### Python

```python
resp = client.simulate.ingest(
    docs=[
        {
            "_id": "123",
            "_index": "my-index",
            "_source": {
                "foo": "bar"
            }
        },
        {
            "_id": "456",
            "_index": "my-index",
            "_source": {
                "foo": "rab"
            }
        }
    ],
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_ingest/_simulate" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "docs": [
      {
        "_id": "123",
        "_index": "my-index",
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_id": "456",
        "_index": "my-index",
        "_source": {
          "foo": "rab"
        }
      }
    ]
  }'
```

#### Java

```java
client.simulate().ingest(i -> i
    .docs(List.of(Document.of(d -> d
            .id("123")
            .index("my-index")
            .source(JsonData.fromJson("{\"foo\":\"bar\"}"))),Document.of(d -> d
            .id("456")
            .index("my-index")
            .source(JsonData.fromJson("{\"foo\":\"rab\"}")))))
);
```

#### C#

```c#
var response = await client.Simulate
    .IngestAsync(d1 => d1
        .Docs(d2 => d2
            .Id("123")
            .Index("my-index")
            .Source(JsonSerializer.Deserialize<JsonElement>("""
            {
              "foo": "bar"
            }
            """)), d2 => d2
            .Id("456")
            .Index("my-index")
            .Source(JsonSerializer.Deserialize<JsonElement>("""
            {
              "foo": "rab"
            }
            """))
        )
    );
```

#### PHP

```php
$resp = $client->simulate()->ingest([
    "body" => [
        "docs" => array(
            [
                "_id" => "123",
                "_index" => "my-index",
                "_source" => [
                    "foo" => "bar",
                ],
            ],
            [
                "_id" => "456",
                "_index" => "my-index",
                "_source" => [
                    "foo" => "rab",
                ],
            ],
        ),
    ],
]);
```

#### Ruby

```ruby
response = client.simulate.ingest(
  body: {
    "docs": [
      {
        "_id": "123",
        "_index": "my-index",
        "_source": {
          "foo": "bar"
        }
      },
      {
        "_id": "456",
        "_index": "my-index",
        "_source": {
          "foo": "rab"
        }
      }
    ]
  }
)
```

## Response Examples

#### Use an existing pipeline definition

A successful response when the simulation uses pipeline definitions that are already in the system.

```json
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
```

#### Use pipeline substitutions

A successful response when the simulation uses pipeline substitutions.

```json
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
```

#### Use pipeline substitutions

A successful response when the simulation uses pipeline substitutions.

```json
{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}
```