Long Running Snapshot Task
A snapshot task is taking longer than the configured limit to finish. Snapshots read shard data and stream it to the repository, so large indices, resource contention, or slow repository connectivity extend duration.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Node |
| Domains | performance, snapshots-backup, stability, tasks |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Long running snapshot task threshold in minutes | Integer | 60 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
Summary: There are 3 tasks that ran more than the 12 minutes threshold. The longest task running start time was 2026-03-15T14:22:00 UTC time. The longest task running time is 420 ms. ACTIVE TASKS: NodeName: logs-prod-000045- TaskId: es-data-01:4521
TaskSerialNumber: 12
TaskType: logs-prod-000045
Description: bulk index to logs-prod-000045
StartTimeInMillis: 1710502920000
RunningTimeInMinutes: 48
Cancellable: true
Action: indices:data/write/bulk
Headers: {} HISTORY TASKS: NodeName:logs-prod-000045- TaskId: es-data-01:4521
TaskSerialNumber: 12
TaskType: logs-prod-000045
Description: bulk index to logs-prod-000045
StartTimeInMillis: 1710502920000
RunningTimeInMinutes: 48
Cancellable: true
Action: indices:data/write/bulk
Headers: {}
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Cancel long-running tasks
Condition: Shown when cancellable.
Cancel long-running tasks (es-data-01:4521, es-data-02:7788) that are affecting cluster stability. Use the action below.
POST _tasks/es-data-01:4521, es-data-02:7788/_cancel
Requires the manage cluster privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Fix non-cancellable snapshot tasks
Condition: Shown when not cancellable.
Review snapshot tasks that cannot be cancelled. Adjust snapshot schedule, repository performance, or cluster load so snapshots complete or can be retried safely.
Reduce snapshot scope
Condition: Shown when snapshot task is not cancellable.
Create smaller snapshots by targeting fewer indices and taking them more often so long-running snapshot tasks stay cancellable.
Impact: Long running snapshot tasks might affect the cluster performance. There can be several reasons that cause snapshot operations to run for a long time, for instance when:
- the snapshot needs to back up one or more very large indexes
- the snapshot runs on a cluster with a high load or that is short on resources (CPU, memory or disk space)
- the network between the cluster and the snapshot repository experiences high latency Snapshot operations can even get stuck in some edge cases, for instance when:
- a shard gets unassigned for some reason
- a shard gets corrupted
- a node hosting a primary shard being snapshotted goes down while the operation is in progress and there's no replica
- a circuit breaking exception or an OutOfMemoryError happens on the master node
- snapshot retention is either not configured at all or configured to retain snapshots for too long Long running snapshots can have various impacts, such as:
- degrading cluster performance by consuming too many resources
- creating incomplete or inconsistent snapshots
- increasing the risk of data loss