Increase in shards count
The number of shards on a node jumped sharply compared with the prior observation window, faster than normal growth for that cluster. Frequent index creation, high primary counts per index, daily rollovers, extra replicas, splits, or ingest spikes are common drivers.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Index |
| Domains | index-management, shard-allocation |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Minimum shards count threshold | Integer | 500 |
| Minimum shards increase percentage | Percentage | 50 |
| Shard increase check period in hours | Integer | 12 |
| Minimum shards count increase | Integer | 50 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
An unusual increase in the number of shards was identified on node es-data-01 and es-data-02. Over the past 48 hours, the number of shards has increased by 12%. However, in the previous period of 48 hours, the number of shards had increased only by 12%.
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Add data node
Condition: Shown when shard growth exceeds cluster capacity.
Add a data node to increase capacity and reduce pressure on the existing nodes.
Reduce shard count
Condition: Shown when index has high primary shard count.
Use the shrink API to reduce primary shards on logs-prod-000045 to 12 in logs-prod-000046. See the Shrink API for prerequisites.
POST logs-prod-000045/_shrink/logs-prod-000046?pretty
{
"settings": {
"index.number_of_shards": 12
}
}
Requires the manage cluster privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Reduce shard replicas
Condition: Shown when shard growth exceeds cluster capacity.
Reduce replica count on logs-prod-000045 from 1 to 2 to free disk space. Start with indices that have more than two shard copies.
PUT logs-prod-000045/_settings
{
"index": {
"number_of_replicas": 2
}
}
Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
The sharp increase of shards can occur due to frequent index creation, high shard settings per index, daily time-based indices, rollover actions, excessive use of replicas, index splitting, increased data intake, or a lack of shard optimization or merging. A rapid, unsupervised rise in the number of shards can negatively impact performance.