Search queue size
The search thread pool queue on one or more nodes is backing up, so new searches wait behind work already in flight. That pattern usually means long-running or repeated expensive queries are occupying threads.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Node |
| Domains | performance, search |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Search queue threshold | Integer | 5 |
| Consecutive samples above threshold | Integer | 1 |
Raising these thresholds reduces noise but delays detection. Lowering them triggers the insight sooner but can increase alerts during minor blips.
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
AutoOps detected a high search thread pool queue on es-data-01 and es-data-02. New search requests on this node wait behind work already running, which usually means long-running or repeated expensive queries are holding threads. Indices with high search activity: logs-prod-000045 Indices with high indexing activity on the same node: logs-prod-000045 Check query logs on the affected node, tune or fix hot queries, and review capacity if the queue stays elevated.
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Increase replica count
Condition: Shown when high-searching activity is detected and if index has no replica.
Set number_of_replicas to 2 on logs-prod-000045 (currently 1) using the action below.
PUT logs-prod-000045/_settings
{
"index": {
"number_of_replicas": 2
}
}
Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Review query logs
Condition: Shown when high search activity is detected on the node and cluster version is 9.4 or above.
Review query logs on es-data-01 to find expensive searches that fill the queue. Query logs (Elasticsearch 9.4+) give structured per-query timing with less overhead than search slow logs. See Query logs.
Enable and review search slow logs
Condition: Shown when high search activity is detected on the node and cluster version is below 9.4.
Enable search slow logs with the action below, then review the slow log to find expensive queries. See Slow logs for configuration details.
PUT logs-prod-000045/_settings
{
"index.search.slowlog.threshold.query.warn": "10s",
"index.search.slowlog.threshold.query.info": "5s",
"index.search.slowlog.threshold.query.debug": "2s",
"index.search.slowlog.threshold.query.trace": "500ms",
"index.search.slowlog.threshold.fetch.warn": "1s",
"index.search.slowlog.threshold.fetch.info": "800ms",
"index.search.slowlog.threshold.fetch.debug": "500ms",
"index.search.slowlog.threshold.fetch.trace": "200ms"
}
Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Add data node
Condition: Shown when high-searching activity is detected and if index has more than twice as many pri/rep shards as available data nodes.
Add a data node to increase capacity and reduce pressure on the existing nodes.
When the search thread pool queue builds up, your searches take longer to start because they wait behind requests already running. A sustained queue backlog can trigger search rejections and make dashboards and applications feel sluggish before hard failures appear. If the queue remains elevated, review query patterns on the affected node and consider scaling capacity. Your log threshold settings control which queries are recorded — tune the threshold to capture the queries most likely driving the backlog.