Snapshot and restore
A snapshot is a backup of a running Elasticsearch cluster. You can use snapshots to:
- Regularly back up a cluster with no downtime
- Recover data after deletion or a hardware failure
- Transfer data between clusters
- Reduce storage costs by using searchable snapshots in the cold and frozen data tiers
Snapshots preserve more than your data. They also include the configuration and internal data of Elastic Stack features, such as ILM policies, index templates and pipelines, Kibana saved objects, alerting rules, Fleet settings and integrations, Elastic Security data, and more, depending on your use case.
Consider using snapshots to back up, at minimum, all Elasticsearch system indices and the cluster state, even if your data can be reindexed or recovered from other external sources. Without these backups, a disaster recovery scenario can result in the loss of your stack configuration and feature states, even if the underlying data can be restored.
Elasticsearch stores snapshots in an off-cluster storage location called a snapshot repository. Before you can take or restore snapshots, you must register a snapshot repository on the cluster. Elasticsearch supports different repository types depending on your deployment type:
After you register a snapshot repository, you can use snapshot lifecycle management (SLM) to automatically take and manage snapshots. You can then restore a snapshot to recover or transfer its data.
While the majority of snapshot-related operations are similar across all deployment types, Elastic Cloud Hosted, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK) offer additional capabilities, as described below.
Elastic Cloud Hosted
When you create a deployment, a default repository called found-snapshots is automatically added to the Elasticsearch cluster. This repository is specific to that cluster: the cluster ID is part of the repository’s base_path, such as /snapshots/[cluster-id].
Do not disable or delete the default cloud-snapshot-policy SLM policy, and do not change the default found-snapshots repository defined in that policy. These actions are not supported.
The default policy and repository are used when:
- Creating a new deployment from a snapshot
- Restoring a snapshot to a different deployment
- Taking automated snapshots in case of deployment changes
In Elastic Cloud Hosted, you can restore snapshots across clusters, but only within the same region.
For API-driven deployment linking of platform-managed snapshots, refer to Manage snapshot repositories in Elastic Cloud Hosted.
You can customize the snapshot retention settings in that policy to adjust them to your needs.
To use a custom snapshot repository, register a new snapshot repository and create another SLM policy.
Elastic Cloud Enterprise
To enable snapshots for your Elasticsearch clusters, you must first configure a repository at the platform level in ECE and then associate it with your deployments. Once configured, snapshots are taken every 30 minutes or at the interval you specify.
Use Kibana to manage your snapshots. In Kibana, you can:
- Set up additional repositories where snapshots are stored (other than the one managed by Elastic Cloud Enterprise)
- View and delete snapshots
- Configure a snapshot lifecycle management (SLM) policy to automate when snapshots are created and deleted
In Elastic Cloud Enterprise, you can also restore snapshots across clusters.
Elastic Cloud on Kubernetes (ECK)
On Elastic Cloud on Kubernetes, you must manually configure snapshot repositories. The system does not create Snapshot Lifecycle Management (SLM) policies or automatic snapshots by default.
For detailed configuration steps, refer to Configuring snapshots on ECK.
Snapshots back up only open indices. If you close an index, it is not included in snapshots and you will not be able to restore the data.
By default, a snapshot of a cluster contains the cluster state, all regular data streams, and all regular indices. The cluster state includes:
- Persistent cluster settings
- Index templates
- Legacy index templates
- Ingest pipelines
- ILM policies
- Stored scripts
- For snapshots taken after 7.12.0, feature states
You can also take snapshots of only specific data streams or indices in the cluster. A snapshot that includes a data stream or index automatically includes its aliases. When you restore a snapshot, you can choose whether to restore these aliases.
Snapshots don’t contain or back up:
- Transient cluster settings
- Registered snapshot repositories
- Node configuration files
- Security configuration files
A feature state contains the indices and data streams used to store configurations, history, and other data for an Elastic feature, such as Elasticsearch security, Kibana, Fleet, or Watcher. To retrieve a list of feature states, use the Features API.
GET /_features
A feature state typically includes one or more system indices or system data streams. It may also include regular indices and data streams used by the feature. For example, a feature state may include a regular index that contains the feature’s execution history. Storing this history in a regular index lets you more easily search it.
Starting with Elasticsearch 8.0 and later versions, feature states are the only way to back up and restore system indices and system data streams. Attempting to restore a system index or data stream outside its feature state is not permitted and will result in the following error:
requested system indices [.example], but system indices can only be restored as part of a feature state
Restoring system indices and data streams will require temporary elevated permissions to edit restricted indices. For more information, refer to File-based access recovery. Attempting to restore a system index or data stream without the required temporary elevated permissions will result in the following error:
Indices [.example] use and access is reserved for system operations
Snapshots are automatically deduplicated to save storage space and reduce network transfer costs. To back up an index, a snapshot makes a copy of the index’s segments and stores them in the snapshot repository. Since segments are immutable, the snapshot only needs to copy any new segments created since the repository’s last snapshot.
Each snapshot is logically independent. When you delete a snapshot, Elasticsearch only deletes the segments used exclusively by that snapshot. Elasticsearch doesn’t delete segments used by other snapshots in the repository.
A snapshot copies segments from an index’s primary shards. When you start a snapshot, Elasticsearch immediately starts copying the segments of any available primary shards. If a shard is starting or relocating, Elasticsearch will wait for these processes to complete before copying the shard’s segments. If one or more primary shards aren’t available, the snapshot attempt fails.
Once a snapshot begins copying a shard’s segments, Elasticsearch won’t move the shard to another node, even if rebalancing or shard allocation settings would typically trigger reallocation. Elasticsearch will only move the shard after the snapshot finishes copying the shard’s data.
A snapshot doesn’t represent a cluster at a precise point in time. Instead, each snapshot includes a start and end time. The snapshot represents a view of each shard’s data at some point between these two times.
To restore a snapshot to a cluster, the versions for the snapshot, cluster, and any restored indices must be compatible.
You can’t restore a snapshot to an earlier version of Elasticsearch. For example, you can’t restore a snapshot taken in 7.6.0 to a cluster running 7.5.0.
Any index you restore from a snapshot must also be compatible with the current cluster’s version. If you try to restore an index created in an incompatible version, the restore attempt fails.
The following table shows whether an index can be restored to a given cluster version. Find your index’s creation version in the left column and your cluster version across the top. For example, an index created in 6.8 can be restored to a 9.0.0–9.4.3 cluster (✅) but not to a 7.0–7.1 cluster (❌).
| Cluster version (restore target) 9.0.0–9.4.3 |
8.3–8.19 | 8.0–8.2 | 7.2–7.17 | 7.0–7.1 | 6.8 | |
| Index creation version | ||||||
| 5.0–5.6 | ✅ 1 | ✅ 1 | ❌ | ❌ | ❌ | ✅ |
| 6.0–6.7 | ✅ 1 | ✅ 1 | ❌ | ✅ | ✅ | ✅ |
| 6.8 | ✅ 1 | ✅ 1 | ❌ | ✅ | ❌ | ✅ |
| 7.0–7.1 | ✅ 1, 2 | ✅ | ✅ | ✅ | ✅ | ❌ |
| 7.2–7.17 | ✅ 1, 2 | ✅ | ✅ | ✅ | ❌ | ❌ |
| 8.0–8.19 | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
| 9.0.0–9.4.3 | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
1 Supported with archive indices.
2 Supported with searchable snapshots.
You can’t restore an index to an earlier version of Elasticsearch. For example, you can’t restore an index created in 8.18.0 to a cluster running 8.15.0.
A compatible snapshot can contain indices created in an older incompatible version. To restore these incompatible indices, you must take additional steps. For example, a snapshot of a 7.17 cluster might contain an index created in 6.8. Restoring the 6.8 index to an 8.18 cluster fails unless you use the archive functionality. To restore a 7.17 index to a 9.0 cluster, you can use the archive functionality or searchable snapshots. Keep this in mind if you take a snapshot before upgrading a cluster.
To ensure index compatibility, you can first restore the index to another cluster running the latest version of Elasticsearch that’s compatible with both the index and your current cluster. You can then use reindex-from-remote to rebuild the index on your current cluster. Reindex from remote is only possible if the index’s _source is enabled.
Reindexing from remote can take significantly longer than restoring a snapshot. Before you start, test the reindex from remote process with a subset of the data to estimate your time requirements.
Elasticsearch and Kibana apply different compatibility rules to restored snapshots. A restore that Elasticsearch accepts can still fail when Kibana starts.
Elasticsearch can typically read data from indices created in a previous major version. For example, restoring a snapshot from an 8.1.0 cluster to a 9.2.4 cluster can succeed at the Elasticsearch layer, but can fail at the Kibana layer. This might happen due to an additional check Kibana applies on startup: it inspects version aliases on the .kibana index and runs saved object migrations.
If the restored Kibana system indices, such as .kibana and .kibana_task_manager, were last written by a version older than 8.18.0, Kibana 9.x doesn't start the migrations and reports an error similar to:
FATAL Error: Kibana 8.1.0 deployment detected. Please upgrade to Kibana 8.18.0 or newer before upgrading to 9.x series.
Restoring an old snapshot directly to a 9.x cluster is not a shortcut around the normal upgrade paths. It is recommended you upgrade to the latest compatible minor release before a major upgrade, even when you use snapshots to move data between clusters.
To make Kibana state from an older 8.x snapshot compatible with a 9.x cluster, restore and migrate to an intermediate 8.x cluster first, then snapshot and restore again on the target version:
- Restore the snapshot to a cluster running Kibana 8.18.0 or later. If your target is 9.1.0 or later, use the latest 8.19.x patch release, as described in Prepare to upgrade.
- Start Kibana on the intermediate cluster and wait for startup to complete. Kibana runs saved object migrations at startup, rewriting the
.kibanasystem indices from their restored version (for example, 8.1.0) to a format that 9.x can read. Elasticsearch does not run these migrations during a snapshot restore. - Take a new snapshot of the migrated cluster, then restore that snapshot to your 9.x cluster. Kibana should start cleanly on the target version.
For example, to move state from an 8.1.0 snapshot to 9.2.4, restore the 8.1.0 snapshot to an 8.19.x cluster, let Kibana complete its startup migrations, snapshot the 8.19.x cluster, and restore that snapshot to 9.2.4.
Alternatively, if you only need to recover data and can accept a fresh Kibana setup, restore the snapshot without the kibana feature state.
Restoring a snapshot from an earlier version to run a previous Elastic Stack version is a different use case. That flow applies when you need to roll back Kibana after a failed upgrade, not when you are moving forward to a newer major version. In a rollback, you restore the kibana feature state from a snapshot taken before the failed upgrade and start Kibana on the earlier version you are rolling back to.
Taking a snapshot is the only reliable and supported way to back up a cluster. You cannot back up an Elasticsearch cluster by making copies of the data directories of its nodes. There are no supported methods to restore any data from a filesystem-level backup. If you try to restore a cluster from such a backup, it may fail with reports of corruption or missing files or other data inconsistencies, or it may appear to have succeeded having silently lost some of your data.
A copy of the data directories of a cluster’s nodes does not work as a backup because it is not a consistent representation of their contents at a single point in time. You cannot fix this by shutting down nodes while making the copies, nor by taking atomic filesystem-level snapshots, because Elasticsearch has consistency requirements that span the whole cluster. You must use the built-in snapshot functionality for cluster backups.
Don’t modify anything within the repository or run processes that might interfere with its contents. If something other than Elasticsearch modifies the contents of the repository then future snapshot or restore operations may fail, reporting corruption or other data inconsistencies, or may appear to succeed having silently lost some of your data.
You may however safely restore a repository from a backup as long as
- The repository is not registered with Elasticsearch while you are restoring its contents.
- When you have finished restoring the repository its contents are exactly as they were when you took the backup.
If you no longer need any of the snapshots in a repository, unregister it from Elasticsearch before deleting its contents from the underlying storage.
Additionally, snapshots may contain security-sensitive information, which you may wish to store in a dedicated repository.