Shard Too Small
The cluster holds many shards that are far smaller than the 10–50 GB range Elasticsearch guidance targets. Too many tiny shards force the master to track excess metadata and make searches fan out across more shard copies than necessary.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Index |
| Domains | performance, index-management, shard-allocation |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Minimum optimal shard size in GB | Integer | 1 |
| Exclude indices based on this pattern | List of strings | |
| Minimum indices size to match | Integer | 1 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
Total number of indices with small shards: 12 The index that contains the smallest shard is: logs-prod-000045, and the shard size is: 0. See a sample list: logs-prod-000045, logs-prod-000046.
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Set up ILM policy
Condition: Always shown for this insight.
For time-based index logs-prod-000045, create or update its ILM policy so rollovers use larger, better-sized shards.
Merge similar indices
Condition: Shown when for non-system indexes.
Merge similar indices to reduce the number of small indices and improve cluster performance and stability.
Reduce shard count
Condition: Shown when for non-system indexes.
Use the shrink API to reduce shard count from 24 to 12. See the Shrink API for prerequisites.
Impact: many small shards can slow down searches and cause cluster instability. Some indices have shards that are too small. An abundance of small shards can slow down search operations because results have to be collated from many shards. It also puts an excessive strain on all nodes because of the resources required to maintain metadata on the state of all the shards. Ideally, shard size should be between 10 and 50GB, and the number of shards on each data node is limited by default to 1,000. There’s also a limit of 1,024 shards per index.