Create or update auto-follow patterns

PUT /_ccr/auto_follow/{name}

Create a collection of cross-cluster replication auto-follow patterns for a remote cluster. Newly created indices on the remote cluster that match any of the patterns are automatically configured as follower indices. Indices on the remote cluster that were created before the auto-follow pattern was created will not be auto-followed even if they match the pattern.

This API can also be used to update auto-follow patterns. NOTE: Follower indices that were configured automatically before updating an auto-follow pattern will remain unchanged even if they do not match against the new patterns.

More about auto-follow patterns

Path Parameters

name string Name required: The name of the collection of auto-follow patterns.

Query Parameters

master_timeout string Duration · default: 30s: Period to wait for a connection to the master node.

Request

remote_cluster string required: The remote cluster containing the leader indices to match against.
follow_index_pattern string: The name of follower index. The template {{leader_index}} can be used to derive the name of the follower index from the name of the leader index. When following a data stream, use {{leader_index}}; CCR does not support changes to the names of a follower data stream’s backing indices.
leader_index_patterns IndexPatterns: An array of simple index patterns to match against indices in the remote cluster specified by the remote_cluster field.
leader_index_exclusion_patterns IndexPatterns: An array of simple index patterns that can be used to exclude indices from being auto-followed. Indices in the remote cluster whose names are matching one or more leader_index_patterns and one or more leader_index_exclusion_patterns won’t be followed.
max_outstanding_read_requests number · default: 12: The maximum number of outstanding reads requests from the remote cluster.
settings map string to object: Settings to override from the leader index. Note that certain settings can not be overrode (e.g., index.number_of_shards).
max_outstanding_write_requests number · default: 9: The maximum number of outstanding reads requests from the remote cluster.
read_poll_timeout string Duration · default: 1m: The maximum time to wait for new operations on the remote cluster when the follower index is synchronized with the leader index. When the timeout has elapsed, the poll for operations will return to the follower so that it can update some statistics. Then the follower will immediately attempt to read from the leader again.
max_read_request_operation_count number · default: 5120: The maximum number of operations to pull per read from the remote cluster.
max_read_request_size number | string ByteSize · default: 32mb: The maximum size in bytes of per read of a batch of operations pulled from the remote cluster.
max_retry_delay string Duration · default: 500ms: The maximum time to wait before retrying an operation that failed exceptionally. An exponential backoff strategy is employed when retrying.
max_write_buffer_count number · default: 2147483647: The maximum number of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the number of queued operations goes below the limit.
max_write_buffer_size number | string ByteSize · default: 512mb: The maximum total bytes of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the total bytes of queued operations goes below the limit.
max_write_request_operation_count number · default: 5120: The maximum number of operations per bulk write request executed on the follower.
max_write_request_size number | string ByteSize · default: 9223372036854775807b: The maximum total bytes of operations per bulk write request executed on the follower.

Response

application/json

Examples

Run PUT /_ccr/auto_follow/my_auto_follow_pattern to creates an auto-follow pattern.

Request Console 1 / 8 languages

Console Kibana Dev Tools

PUT /_ccr/auto_follow/my_auto_follow_pattern
{
  "remote_cluster" : "remote_cluster",
  "leader_index_patterns" :
  [
    "leader_index*"
  ],
  "follow_index_pattern" : "{{leader_index}}-follower",
  "settings": {
    "index.number_of_replicas": 0
  },
  "max_read_request_operation_count" : 1024,
  "max_outstanding_read_requests" : 16,
  "max_read_request_size" : "1024k",
  "max_write_request_operation_count" : 32768,
  "max_write_request_size" : "16k",
  "max_outstanding_write_requests" : 8,
  "max_write_buffer_count" : 512,
  "max_write_buffer_size" : "512k",
  "max_retry_delay" : "10s",
  "read_poll_timeout" : "30s"
}

JavaScript Elasticsearch JavaScript Client

const response = await client.ccr.putAutoFollowPattern({
  name: "my_auto_follow_pattern",
  remote_cluster: "remote_cluster",
  leader_index_patterns: ["leader_index*"],
  follow_index_pattern: "{{leader_index}}-follower",
  settings: {
    "index.number_of_replicas": 0,
  },
  max_read_request_operation_count: 1024,
  max_outstanding_read_requests: 16,
  max_read_request_size: "1024k",
  max_write_request_operation_count: 32768,
  max_write_request_size: "16k",
  max_outstanding_write_requests: 8,
  max_write_buffer_count: 512,
  max_write_buffer_size: "512k",
  max_retry_delay: "10s",
  read_poll_timeout: "30s",
});

Python Elasticsearch Python Client

resp = client.ccr.put_auto_follow_pattern(
    name="my_auto_follow_pattern",
    remote_cluster="remote_cluster",
    leader_index_patterns=[
        "leader_index*"
    ],
    follow_index_pattern="{{leader_index}}-follower",
    settings={
        "index.number_of_replicas": 0
    },
    max_read_request_operation_count=1024,
    max_outstanding_read_requests=16,
    max_read_request_size="1024k",
    max_write_request_operation_count=32768,
    max_write_request_size="16k",
    max_outstanding_write_requests=8,
    max_write_buffer_count=512,
    max_write_buffer_size="512k",
    max_retry_delay="10s",
    read_poll_timeout="30s",
)

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_ccr/auto_follow/my_auto_follow_pattern" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "remote_cluster": "remote_cluster",
    "leader_index_patterns": [
      "leader_index*"
    ],
    "follow_index_pattern": "{{leader_index}}-follower",
    "settings": {
      "index.number_of_replicas": 0
    },
    "max_read_request_operation_count": 1024,
    "max_outstanding_read_requests": 16,
    "max_read_request_size": "1024k",
    "max_write_request_operation_count": 32768,
    "max_write_request_size": "16k",
    "max_outstanding_write_requests": 8,
    "max_write_buffer_count": 512,
    "max_write_buffer_size": "512k",
    "max_retry_delay": "10s",
    "read_poll_timeout": "30s"
  }'

Java Elasticsearch Java Client

client.ccr().putAutoFollowPattern(p -> p
    .followIndexPattern("{{leader_index}}-follower")
    .leaderIndexPatterns("leader_index*")
    .maxOutstandingReadRequests(16)
    .maxOutstandingWriteRequests(8)
    .maxReadRequestOperationCount(1024)
    .maxReadRequestSize("1024k")
    .maxRetryDelay(m -> m
        .time("10s")
    )
    .maxWriteBufferCount(512)
    .maxWriteBufferSize("512k")
    .maxWriteRequestOperationCount(32768)
    .maxWriteRequestSize("16k")
    .name("my_auto_follow_pattern")
    .readPollTimeout(r -> r
        .time("30s")
    )
    .remoteCluster("remote_cluster")
    .settings("index.number_of_replicas", JsonData.fromJson("0"))
);

C# Elasticsearch .NET Client

var response = await client.CrossClusterReplication
    .PutAutoFollowPatternAsync(name: "my_auto_follow_pattern", d1 => d1
        .FollowIndexPattern("{{leader_index}}-follower")
        .LeaderIndexPatterns("leader_index*")
        .MaxOutstandingReadRequests(16)
        .MaxOutstandingWriteRequests(8)
        .MaxReadRequestOperationCount(1024)
        .MaxReadRequestSize(new ByteSize("1024k"))
        .MaxRetryDelay("10s")
        .MaxWriteBufferCount(512)
        .MaxWriteBufferSize(new ByteSize("512k"))
        .MaxWriteRequestOperationCount(32768)
        .MaxWriteRequestSize(new ByteSize("16k"))
        .ReadPollTimeout("30s")
        .RemoteCluster("remote_cluster")
        .Settings(new Dictionary<string, object>()
        {
            { "index.number_of_replicas", 0 }
        })
    );

PHP Elasticsearch PHP Client

$resp = $client->ccr()->putAutoFollowPattern([
    "name" => "my_auto_follow_pattern",
    "body" => [
        "remote_cluster" => "remote_cluster",
        "leader_index_patterns" => array(
            "leader_index*",
        ),
        "follow_index_pattern" => "{{leader_index}}-follower",
        "settings" => [
            "index.number_of_replicas" => 0,
        ],
        "max_read_request_operation_count" => 1024,
        "max_outstanding_read_requests" => 16,
        "max_read_request_size" => "1024k",
        "max_write_request_operation_count" => 32768,
        "max_write_request_size" => "16k",
        "max_outstanding_write_requests" => 8,
        "max_write_buffer_count" => 512,
        "max_write_buffer_size" => "512k",
        "max_retry_delay" => "10s",
        "read_poll_timeout" => "30s",
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.ccr.put_auto_follow_pattern(
  name: "my_auto_follow_pattern",
  body: {
    "remote_cluster": "remote_cluster",
    "leader_index_patterns": [
      "leader_index*"
    ],
    "follow_index_pattern": "{{leader_index}}-follower",
    "settings": {
      "index.number_of_replicas": 0
    },
    "max_read_request_operation_count": 1024,
    "max_outstanding_read_requests": 16,
    "max_read_request_size": "1024k",
    "max_write_request_operation_count": 32768,
    "max_write_request_size": "16k",
    "max_outstanding_write_requests": 8,
    "max_write_buffer_count": 512,
    "max_write_buffer_size": "512k",
    "max_retry_delay": "10s",
    "read_poll_timeout": "30s"
  }
)

Response

{
  "acknowledged": true
}