Manage auto-follow patterns
To replicate time series indices, you configure an auto-follow pattern so that each new index in the series is replicated automatically. Whenever the name of a new index on the remote cluster matches the auto-follow pattern, a corresponding follower index is added to the local cluster.
Auto-follow patterns only match open indices on the remote cluster that have all primary shards started. Auto-follow patterns do not match indices that can’t be used for CCR such as closed indices or searchable snapshots. Avoid using an auto-follow pattern that matches indices with a read or write block. These blocks prevent follower indices from replicating such indices.
You can also create auto-follow patterns for data streams. When a new backing index is generated on a remote cluster, that index and its data stream are automatically followed if the data stream name matches an auto-follow pattern. If you create a data stream after creating the auto-follow pattern, all backing indices are followed automatically.
The data streams replicated from a remote cluster by CCR are protected from local rollovers. The promote data stream API can be used to turn these data streams into regular data streams.
Auto-follow patterns are especially useful with Index lifecycle management. Index lifecycle management detects when a managed index participates in cross-cluster replication as a leader or follower and adapts lifecycle actions that cannot run safely while replication is active.
Cross-cluster replication copies the index.lifecycle.name setting from a leader index to its follower, but it does not replicate the policy definition. To manage follower indices with index lifecycle management, define a policy with that name on the local cluster. You can use the same policy definition on both clusters or configure different lifecycle actions and retention periods.
When index lifecycle management rolls over a matching leader index, the auto-follow pattern detects the new index and creates its corresponding follower on the local cluster. When the policy on the local cluster reaches the rollover action on the previous follower, index lifecycle management waits until indexing on the leader index is complete and all operations have been replicated. It then unfollows the previous index automatically and continues its lifecycle without performing a separate rollover. The shrink and searchable snapshot actions also trigger this automatic unfollow behavior.
Cross-cluster replication does not propagate index deletions. Auto-follow patterns do not delete follower indices when their leader indices are deleted.
The index lifecycle management delete action on a leader index waits until all follower indices cease following it. To manage follower index retention, define an appropriate lifecycle policy on the local cluster.
To manage auto-follow patterns, open Kibana for the local cluster that contains the follower indices:
- Go to the Cross Cluster Replication management page in the navigation menu or use the global search field.
- Select the Auto-follow patterns tab.
From this tab, you can create, pause, resume, and delete auto-follow patterns.