Loading

Connect to Elasticsearch clusters in the same ECK environment

ECK

These steps describe how to configure remote clusters between two Elasticsearch clusters that are managed by the same Elastic Cloud on Kubernetes (ECK) operator.

After the connection is established, you’ll be able to run CCS queries from Elasticsearch or set up CCR. For other remote cluster scenarios with ECK, refer to Remote clusters on Elastic Cloud on Kubernetes.

Note

The remote clusters feature requires a valid Enterprise license or Enterprise trial license. Check the license documentation for more details about managing licenses.

Note about terminology

In the case of remote clusters, the Elasticsearch cluster or deployment initiating the connection and requests is often referred to as the local cluster, while the Elasticsearch cluster or deployment receiving the requests is referred to as the remote cluster.

To create a remote cluster connection to another Elasticsearch cluster deployed in the same ECK environment, specify the remoteClusters attribute in your Elasticsearch spec.

Before you start, consider the security model that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps.

API key
For deployments based on Elastic Stack 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model uses a dedicated service endpoint, on port 9443 by default, and gives administrators fine-grained control over remote access. The API key is created on the remote cluster and defines the permissions available to all cross-cluster requests, while local user roles can further restrict, but not extend, those permissions.
TLS certificate (deprecated in Elastic Stack 9.0.0)
This model uses mutual TLS authentication over the Elasticsearch transport interface for cross-cluster operations. User authentication is performed on the local cluster and a user's role names are passed to the remote cluster for authorization. Because a superuser on the local cluster automatically gains full read access to the remote cluster, this model is only suitable for clusters within the same security domain.

Based on the selected security model, use one of the following setup procedures.

Follow these steps to configure the API key security model for remote clusters. If you run into any issues, refer to Troubleshooting.

By default, the remote cluster server interface is deactivated on ECK-managed clusters. To use the API key–based security model for cross-cluster connections, you must first enable it on the remote Elasticsearch cluster by setting spec.remoteClusterServer.enabled: true:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: <cluster-name>
  namespace: <namespace>
spec:
  version: 9.2.2
  remoteClusterServer:
    enabled: true
  nodeSets:
    - name: default
      count: 3
      ...
      ...
Note

Enabling the remote cluster server triggers a restart of the Elasticsearch cluster.

Once the remote cluster server is enabled and running on the remote cluster, you can configure the Elasticsearch reference on the local cluster and include the desired permissions for cross-cluster search and cross-cluster replication under the spec.remoteClusters field.

Permissions have to be included under the apiKey field. The API model of the Elasticsearch resource is compatible with the Elasticsearch Cross-Cluster API key API model. Fine-grained permissions can therefore be configured in both the search and replication fields.

The following example shows how to connect a local cluster to a remote cluster and specify the cross-cluster permissions under the apiKey field. This configuration is applied to the local cluster manifest:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: <local-cluster-name>
  namespace: <ns-local>
spec:
  version: 9.2.2
  remoteClusters:
  - name: <remote-cluster-name>
    elasticsearchRef:
      name: <remote-cluster-name>
      namespace: <ns-remote>
    apiKey:
      access:
        search:
          names:
            - kibana_sample_data_ecommerce
        replication:
          names:
            - kibana_sample_data_ecommerce
  nodeSets:
  - count: 3
    name: default
  ...
  ...
  1. The name and namespace of the remote Elasticsearch cluster you are connecting to.
  2. The namespace declaration can be omitted if both clusters reside in the same namespace.
  3. This example requires the Kibana sample data.

You can find a complete example in the Elastic Cloud on Kubernetes repository's recipes directory.

The following example shows how to connect a local cluster to a remote cluster using the certificate-based security model. The configuration is applied to the local cluster manifest:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: <local-cluster-name>
  namespace: <ns-local>
spec:
  version: 8.16.1
  remoteClusters:
  - name: <remote-cluster-name>
    elasticsearchRef:
      name: <remote-cluster-name>
      namespace: <ns-remote>
  nodeSets:
  - count: 3
    name: default
  ...
  ...
  1. The name and namespace of the remote Elasticsearch cluster you are connecting to.
  2. The namespace declaration can be omitted if both clusters reside in the same namespace.

If you're using the API key–based security model for cross-cluster replication or cross-cluster search, you can define user roles with remote indices privileges on the local cluster to further restrict the permissions granted by the API key. For more details, refer to Configure roles and users.