Shard count high on frozen node
One or more frozen tier nodes host more shards than the recommended limit for stable operation. High shard counts can cause master nodes to struggle with cluster state tracking and can stress file descriptor and virtual memory limits on the affected nodes.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Node |
| Domains | shard-allocation, stability |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Early warning margin before the max shards per frozen node limit | Percentage | 10 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
The current shard count on es-data-01 and es-data-02 is 24 (limit: 3000).
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Add frozen data node
Condition: Shown when frozen node shard count is at or near limit.
Add a frozen-tier data node to host searchable snapshot indices and scale frozen storage separately.
Reduce shard count
Condition: Shown when frozen node shard count is at or near limit.
Use the shrink API to reduce shard count from 24 to 12. See the Shrink API for prerequisites.
Your cluster has more shards on one or more frozen nodes than is recommended for stable operation. Oversharding can destabilize a cluster when master nodes struggle to track many shards across nodes. High shard counts can also approach limits for virtual memory (mmap) and open file descriptors on the affected nodes. Both might need adjustment if shard counts increase further. In the version (8.15.0) you're running, the memory usage per shard is significantly higher than in more recent versions, where a pivotal change was introduced to optimize memory consumption. Consider upgrading to a more recent version. If that is not an option right now, lower the cluster.max_shards_per_node.frozen cluster setting to keep memory consumption in check.