Long Running Delete By Query Task
A delete-by-query task has been running longer than the threshold you set. These tasks scroll matching documents and delete them in batches, keeping search contexts open for extended periods.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Node |
| Domains | performance, indexing, stability, tasks |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Long running Delete By Query task threshold in minutes | Integer | 60 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
Summary: There are 3 tasks that ran more than the 12 minutes threshold. The longest task running start time was 2026-03-15T14:22:00 UTC time. The longest task running time is 420 ms. ACTIVE TASKS: NodeName: logs-prod-000045- TaskId: es-data-01:4521
TaskSerialNumber: 12
TaskType: logs-prod-000045
Description: bulk index to logs-prod-000045
StartTimeInMillis: 1710502920000
RunningTimeInMinutes: 48
Cancellable: true
Action: indices:data/write/bulk
Headers: {} HISTORY TASKS: NodeName:logs-prod-000045- TaskId: es-data-01:4521
TaskSerialNumber: 12
TaskType: logs-prod-000045
Description: bulk index to logs-prod-000045
StartTimeInMillis: 1710502920000
RunningTimeInMinutes: 48
Cancellable: true
Action: indices:data/write/bulk
Headers: {}
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Improve long delete-by-query tasks
Condition: Shown when always included (delete-by-query tasks are not cancellable.
Use slicing to parallelize the task, narrow the query with filters to match fewer documents, and temporarily raise or turn off the refresh interval during the run. Review index mappings and analysis for optimization opportunities.
Impact: Long running delete by query tasks might affect the cluster performance. It might be perfectly ok to have delete by query operations running for a long time. For instance, you have a big index from which you need to delete a substantial amount of documents and deleting the index altogether is not an option for you, hence you decide to run a delete by query operation. However, it might be worth understanding how the delete by query operation works under the hood, and how it can affect your cluster. Delete by query uses the Scroll API to figure out the documents that need to be deleted. The Scroll API opens a search context that will be kept in memory until the delete by query task has terminated, which means that the longer the delete operation lasts, the longer the memory will be occupied by the search context. Running several delete by query operations in parallel worsens the situation even more. Even if the search context is kept alive for 5 minutes by default, it doesn't mean that the entire delete by query operation will stop within that time. It can last much longer, as each new batch of deletes will refresh that keep alive timeout for 5 more minutes. For these reasons, there are a few best practices to keep in mind:
- Specify a sufficiently restrictive search query along with your delete by query request, so as to delete as few documents as necessary
- Do not run too many delete by query requests in parallel
- Try to run delete by query requests during off-peak hours
- Deleting the index and rebuilding it might take less time and consume less resources
- Leverage slicing to better benefit from parallelism