Reindex documents

POST /_reindex

Copy documents from a source to a destination. You can copy all documents to the destination index or reindex a subset of the documents. The source can be any existing index, alias, or data stream. The destination must differ from the source. For example, you cannot reindex a data stream into itself.

IMPORTANT: Reindex requires _source to be enabled for all documents in the source. The destination should be configured as wanted before calling the reindex API. Reindex does not copy the settings from the source or its associated template. Mappings, shard counts, and replicas, for example, must be configured ahead of time.

If the Elasticsearch security features are enabled, you must have the following security privileges:

The read index privilege for the source data stream, index, or alias.
The write index privilege for the destination data stream, index, or index alias.
To automatically create a data stream or index with a reindex API request, you must have the auto_configure, create_index, or manage index privilege for the destination data stream, index, or alias.
If reindexing from a remote cluster, the source.remote.user must have the monitor cluster privilege and the read index privilege for the source data stream, index, or alias.

If reindexing from a remote cluster into a cluster using Elastic Stack, you must explicitly allow the remote host using the reindex.remote.whitelist node setting on the destination cluster. If reindexing from a remote cluster into an Elastic Cloud Serverless project, only remote hosts from Elastic Cloud Hosted and Elastic Cloud Serverless are allowed. Automatic data stream creation requires a matching index template with data stream enabled.

The dest element can be configured like the index API to control optimistic concurrency control. Omitting version_type or setting it to internal causes Elasticsearch to blindly dump documents into the destination, overwriting any that happen to have the same ID.

Setting version_type to external causes Elasticsearch to preserve the version from the source, create any documents that are missing, and update any documents that have an older version in the destination than they do in the source.

Setting op_type to create causes the reindex API to create only missing documents in the destination. All existing documents will cause a version conflict.

IMPORTANT: Because data streams are append-only, any reindex request to a destination data stream must have an op_type of create. A reindex can only add new documents to a destination data stream. It cannot update existing documents in a destination data stream.

By default, version conflicts abort the reindex process. To continue reindexing if there are conflicts, set the conflicts request body property to proceed. In this case, the response includes a count of the version conflicts that were encountered. Note that the handling of other error types is unaffected by the conflicts property. Additionally, if you opt to count version conflicts, the operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.

It's recommended to reindex on indices with a green status. Reindexing can fail when a node shuts down or crashes.

When requested with wait_for_completion=true (default), the request fails if the node shuts down.
When requested with wait_for_completion=false, a task id is returned, for use with the task management APIs. The task may disappear or fail if the node shuts down. When retrying a failed reindex operation, it might be necessary to set conflicts=proceed or to first delete the partial destination index. Additionally, dry runs, checking disk space, and fetching index recovery information can help address the root cause.

Refer to the linked documentation for examples of how to reindex documents.

Required authorization

Index privileges: read,write

About reindexing indices examples

Query Parameters

refresh boolean · default: false

If true, the request refreshes affected shards to make this operation visible to search.

requests_per_second number · default: -1

The maximum number of documents to index per second, across the entire reindex operation (including slices). It can be either -1 to turn off throttling or any decimal number like 1.7 or 12 to throttle to that level.

scroll string Duration · default: 5m

The period of time that a consistent view of the index should be maintained for scrolled search. In serverless, and stack versions >= v9.5.0, we use PIT rather than scroll for pagination. We only use scroll for reindexing from remote clusters that are older than v7.10.0. Therefore, this parameter is ignored unless you are reindexing from a remote cluster that is older than v7.10.0.

slices Slices · default: 1

The number of slices this task should be divided into. It defaults to one slice, which means the task isn't sliced into subtasks.

Reindex supports sliced scroll to parallelize the reindexing process. This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.

NOTE: Reindexing from remote clusters does not support manual or automatic slicing.

If set to auto, Elasticsearch chooses the number of slices to use. This setting will use one slice per shard, up to a certain limit. If there are multiple sources, it will choose the number of slices based on the index or backing index with the smallest number of shards.

Values: auto

max_docs number

The maximum number of documents to reindex. By default, all documents are reindexed. If it is a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.

If conflicts is set to proceed, the reindex operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.

If slices is set, the max_docs limit is split evenly across the slices. If the number of documents in the source is equal to or slightly more than max_docs, this could result in slightly fewer than max_docs documents being reindexed, due to skew in the slicing.

timeout string Duration · default: 1m

The period each indexing waits for automatic index creation, dynamic mapping updates, and waiting for active shards. By default, Elasticsearch waits for at least one minute before failing. The actual wait time could be longer, particularly when multiple waits occur.

wait_for_active_shards WaitForActiveShards · default: 1

The number of shard copies that must be active before proceeding with the operation. Set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value is one, which means it waits for each primary shard to be active.

Values: all index-setting

wait_for_completion boolean · default: true

If true, the request blocks until the operation is complete. If your requested reindex operation is complex or time-consuming, it might timeout due to transport-layer limitations. While the reindex will continue to be processed by the cluster, your client will not receive updates on status automatically after timeout. Set this option true if you anticipate a long-running reindex.

require_alias boolean · default: false

If true, the destination must be an index alias.

Request

Response

application/json

Examples

Run POST _reindex to reindex from multiple sources. The index attribute in source can be a list, which enables you to copy from lots of sources in one request. This example copies documents from the my-index-000001 and my-index-000002 indices.

Request Console 1 / 8 languages

Console Kibana Dev Tools

POST _reindex
{
  "source": {
    "index": ["my-index-000001", "my-index-000002"]
  },
  "dest": {
    "index": "my-new-index-000002"
  }
}

JavaScript Elasticsearch JavaScript Client

const response = await client.reindex({
  source: {
    index: ["my-index-000001", "my-index-000002"],
  },
  dest: {
    index: "my-new-index-000002",
  },
});

Python Elasticsearch Python Client

resp = client.reindex(
    source={
        "index": [
            "my-index-000001",
            "my-index-000002"
        ]
    },
    dest={
        "index": "my-new-index-000002"
    },
)

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": {
      "index": [
        "my-index-000001",
        "my-index-000002"
      ]
    },
    "dest": {
      "index": "my-new-index-000002"
    }
  }'

Java Elasticsearch Java Client

client.reindex(r -> r
    .dest(d -> d
        .index("my-new-index-000002")
    )
    .source(s -> s
        .index(List.of("my-index-000001","my-index-000002"))
    )
);

C# Elasticsearch .NET Client

var response = await client
    .ReindexAsync(d1 => d1
        .Dest(d2 => d2
            .Index("my-new-index-000002")
        )
        .Source(d2 => d2
            .Indices(new[] { "my-index-000001", "my-index-000002" })
        )
    );

PHP Elasticsearch PHP Client

$resp = $client->reindex([
    "body" => [
        "source" => [
            "index" => array(
                "my-index-000001",
                "my-index-000002",
            ),
        ],
        "dest" => [
            "index" => "my-new-index-000002",
        ],
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.reindex(
  body: {
    "source": {
      "index": [
        "my-index-000001",
        "my-index-000002"
      ]
    },
    "dest": {
      "index": "my-new-index-000002"
    }
  }
)

Response

No example

You can use Painless to reindex daily indices to apply a new template to the existing documents. The script extracts the date from the index name and creates a new index with -1 appended. For example, all data from metricbeat-2016.05.31 will be reindexed into metricbeat-2016.05.31-1.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST /_reindex
{
  "source": {
    "index": "metricbeat-*"
  },
  "dest": {
    "index": "metricbeat"
  },
  "script": {
    "lang": "painless",
    "source": "ctx._index = 'metricbeat-' + (ctx._index.substring('metricbeat-'.length(), ctx._index.length())) + '-1'"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "metricbeat-*"
  },
  "dest": {
    "index": "metricbeat"
  },
  "script": {
    "lang": "painless",
    "source": "ctx._index = '\''metricbeat-'\'' + (ctx._index.substring('\''metricbeat-'\''.length(), ctx._index.length())) + '\''-1'\''"
  }
}'

Response

No example

Run POST _reindex to extract a random subset of the source for testing. You might need to adjust the min_score value depending on the relative amount of data extracted from source.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST _reindex
{
  "max_docs": 10,
  "source": {
    "index": "my-index-000001",
    "query": {
      "function_score" : {
        "random_score" : {},
        "min_score" : 0.9
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "max_docs": 10,
  "source": {
    "index": "my-index-000001",
    "query": {
      "function_score" : {
        "random_score" : {},
        "min_score" : 0.9
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

Run POST _reindex to modify documents during reindexing. This example bumps the version of the source document.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST _reindex
{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001",
    "version_type": "external"
  },
  "script": {
    "source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
    "lang": "painless"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001",
    "version_type": "external"
  },
  "script": {
    "source": "if (ctx._source.foo == '\''bar'\'') {ctx._version++; ctx._source.remove('\''foo'\'')}",
    "lang": "painless"
  }
}'

Response

No example

When using Elastic Cloud, you can run POST _reindex and authenticate against a remote cluster with an API key.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST /_reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "my-index-000001",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "my-index-000001",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

Run POST _reindex to slice a reindex request manually. Provide a slice ID and total number of slices to each request.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST _reindex
{
  "source": {
    "index": "my-index-000001",
    "slice": {
      "id": 0,
      "max": 2
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "my-index-000001",
    "slice": {
      "id": 0,
      "max": 2
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

Run POST _reindex?slices=5&refresh to automatically parallelize using sliced scroll to slice on _id. The slices parameter specifies the number of slices to use.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST _reindex?slices=5&refresh
{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex?slices=5&refresh" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

By default if reindex sees a document with routing then the routing is preserved unless it's changed by the script. You can set routing on the dest request to change this behavior. In this example, run POST _reindex to copy all documents from the source with the company name cat into the dest with routing set to cat.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST /_reindex
{
  "source": {
    "index": "source",
    "query": {
      "match": {
        "company": "cat"
      }
    }
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "source",
    "query": {
      "match": {
        "company": "cat"
      }
    }
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}'

Response

No example

Run POST _reindex and use the ingest pipelines feature.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
    "pipeline": "some_ingest_pipeline"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
    "pipeline": "some_ingest_pipeline"
  }
}'

Response

No example

Run POST _reindex and add a query to the source to limit the documents to reindex. For example, this request copies documents into my-new-index-000001 only if they have a user.id of kimchy.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST _reindex
{
  "source": {
    "index": "my-index-000001",
    "query": {
      "term": {
        "user.id": "kimchy"
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "my-index-000001",
    "query": {
      "term": {
        "user.id": "kimchy"
      }
    }
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

You can limit the number of processed documents by setting max_docs. For example, run POST _reindex to copy a single document from my-index-000001 to my-new-index-000001.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST /_reindex
{
  "max_docs": 1,
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "max_docs": 1,
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

You can use source filtering to reindex a subset of the fields in the original documents. For example, run POST _reindex the reindex only the user.id and _doc fields of each document.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST /_reindex
{
  "source": {
    "index": "my-index-000001",
    "_source": ["user.id", "_doc"]
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "my-index-000001",
    "_source": ["user.id", "_doc"]
  },
  "dest": {
    "index": "my-new-index-000001"
  }
}'

Response

No example

A reindex operation can build a copy of an index with renamed fields. If your index has documents with text and flag fields, you can change the latter field name to tag during the reindex.

Request Console 1 / 2 languages

Console Kibana Dev Tools

POST /_reindex
{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  },
  "script": {
    "source": "ctx._source.tag = ctx._source.remove(\"flag\")"
  }
}

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_reindex" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "source": {
    "index": "my-index-000001"
  },
  "dest": {
    "index": "my-new-index-000001"
  },
  "script": {
    "source": "ctx._source.tag = ctx._source.remove(\"flag\")"
  }
}'

Response

No example

Required authorization

Query Parameters { refresh , requests_per_second , scroll , slices , max_docs , timeout , wait_for_active_shards , wait_for_completion , require_alias , }

Request { conflicts , dest , max_docs , script , source , }

Response

Query Parameters

Request