--- title: Connect Elastic Cloud Hosted deployments to Elastic Cloud on Kubernetes clusters description: These steps describe how to configure remote clusters between an Elasticsearch cluster in Elastic Cloud Hosted (ECH) and an Elasticsearch cluster running... url: https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/ec-enable-ccs-for-eck products: - Elastic Cloud Hosted applies_to: - Elastic Cloud Hosted: Generally available - Elastic Cloud on Kubernetes: Generally available --- # Connect Elastic Cloud Hosted deployments to Elastic Cloud on Kubernetes clusters These steps describe how to configure remote clusters between an Elasticsearch cluster in Elastic Cloud Hosted (ECH) and an Elasticsearch cluster running within [Elastic Cloud on Kubernetes (ECK)](https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/deploy/cloud-on-k8s). Once that’s done, you’ll be able to [run CCS queries from Elasticsearch](https://www.elastic.co/elastic/docs-builder/docs/3016/explore-analyze/cross-cluster-search) or [set up CCR](https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/tools/cross-cluster-replication/set-up-cross-cluster-replication). In the case of remote clusters, the Elasticsearch cluster or deployment initiating the connection and requests is often referred to as the **local cluster**, while the Elasticsearch cluster or deployment receiving the requests is referred to as the **remote cluster**. ## Allow the remote connection Before you start, consider the [security model](https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/security-models) that you would prefer to use for authenticating remote connections between clusters, and follow the corresponding steps. For deployments based on Elastic Stack 8.14 or later, you can use an API key to authenticate and authorize cross-cluster operations to a remote cluster. This model uses a dedicated service endpoint, on port `9443` by default, and gives administrators fine-grained control over remote access. The API key is created on the remote cluster and defines the permissions available to all cross-cluster requests, while local user roles can further restrict, but not extend, those permissions. Starting with Elastic Stack 9.3, the API key security model also supports [strong identity verification](/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/security-models#remote-cluster-strong-verification), adding an extra layer of security. With this feature, the API key can be restricted to only be usable by requests that present an allowed certificate identity, which the remote cluster validates during authentication. This model uses mutual TLS authentication over the Elasticsearch transport interface for cross-cluster operations. User authentication is performed on the local cluster and a user's role names are passed to the remote cluster for authorization. Because a superuser on the local cluster automatically gains full read access to the remote cluster, this model is only suitable for clusters within the same security domain. Follow these steps to configure the [API key security model](/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/security-models#api-key) for remote clusters. If you run into any issues, refer to [Troubleshooting](https://www.elastic.co/elastic/docs-builder/docs/3016/troubleshoot/elasticsearch/remote-clusters). ### Prerequisites and limitations - The local and remote deployments must be on Elastic Stack 8.14 or later. - Unlike the certificate-based security model, the API key model does not require mutual trust between clusters; only the local cluster is required to trust the remote cluster's certificate. ### Enable the remote cluster server interface on the remote ECK cluster By default, the remote cluster server interface is not enabled on ECK-managed clusters. To use the API key–based security model for cross-cluster connections, you must first enable it on the remote Elasticsearch cluster by setting `spec.remoteClusterServer.enabled: true`: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: namespace: spec: version: 9.3.2 remoteClusterServer: enabled: true nodeSets: - name: default count: 3 ... ... ``` Enabling the remote cluster server triggers a restart of the Elasticsearch cluster. ### Configure external access to the remote cluster server interface When the remote cluster server is enabled, ECK automatically creates a Kubernetes service named `-es-remote-cluster` that exposes the server internally on port `9443`. To allow clusters running outside your Kubernetes environment to connect to this Elasticsearch cluster, you must expose this service externally. The way to expose this service depends on your ECK version. You can customize how the remote cluster service is exposed by overriding its service specification directly under `spec.remoteClusterServer.service` in the Elasticsearch resource. By default, this service listens on port 9443. ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: namespace: spec: version: 9.2.1 remoteClusterServer: enabled: true service: spec: type: LoadBalancer nodeSets: - name: default count: 3 ... ... ``` 1. On cloud providers that support external load balancers, setting the type to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `-es-remote-cluster` through one of the Kubernetes Ingress controllers that support TCP services. You can't customize the service that ECK generates for the remote cluster interface, but you can create your own `LoadBalancer` service, `Ingress` object, or use another method available in your environment.For example, for a cluster named `quickstart`, the following command creates a separate `LoadBalancer` service named `quickstart-es-remote-cluster-lb`, pointing to the ECK-managed service `quickstart-es-remote-cluster`: ```sh kubectl expose service quickstart-es-remote-cluster \ --name=quickstart-es-remote-cluster-lb \ --type=LoadBalancer \ --port=9443 --target-port=9443 ``` 1. On cloud providers that support external load balancers, setting the type to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `-es-remote-cluster` through one of the Kubernetes Ingress controllers that support TCP services. If you change the service’s `port` to expose a different port externally, set `targetPort` explicitly to `9443`, which is the default remote cluster server listening port. Otherwise, Kubernetes uses the same value for both fields, resulting in failed connections. ### Retrieve the ECK-managed CA certificate of the remote cluster server The certificate authority (CA) used by ECK to issue certificates for the remote cluster server interface is stored in the `ca.crt` key of the secret named `-es-transport-certs-public`. If the external connections reach the Elasticsearch Pods on port `9443` without any intermediate TLS termination, you need to retrieve this CA because it is required in the local cluster configuration to establish trust. If TLS is terminated by any intermediate component and the certificate presented is not the ECK-managed one, use the CA associated with that component, or omit the CA entirely if it uses a publicly trusted certificate. To save the transport CA certificate of a cluster named `quickstart` into a local file, run the following command: ```sh kubectl get secret quickstart-es-transport-certs-public \ -o go-template='{{index .data "ca.crt" | base64decode}}' > eck_transport_ca.crt ``` ECK-managed CA certificates are automatically rotated after one year by default, but you can [configure](https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/deploy/cloud-on-k8s/configure-eck) a different validity period. When the CA certificate is rotated, ensure that this CA is updated in all environments where it's used to preserve trust. ### Create a cross-cluster API key on the remote cluster 1. On the remote cluster, use the [Elasticsearch API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-security-create-cross-cluster-api-key) or [Kibana](https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/api-keys/elasticsearch-api-keys) to create a cross-cluster API key. Configure it to include access to the indices you want to use for cross-cluster search or cross-cluster replication. 2. Copy the encoded key (`encoded` in the response) to a safe location. It is required for the local cluster configuration. ### Configure the local deployment The API key created previously is needed by the local deployment to authenticate with the corresponding set of permissions to the remote cluster. To enable this, add the API key to the local deployment's keystore.The steps to follow depend on whether the certificate authority (CA) presented by the remote cluster server, proxy, or load-balancing infrastructure is publicly trusted or private. 1. Log in to [Elastic Cloud](https://cloud.elastic.co?page=docs&placement=docs-body). 2. Find your deployment on the home page or on the **Hosted deployments** page, then select **Manage** to access its settings menus. On the **Hosted deployments** page you can narrow down your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. 3. From the navigation menu, select **Access and Security** > **Security**. 4. Locate **Remote Connections > Trust management > Connections using API keys** and select **Add API key**. 1. Fill both fields. - For the **Remote cluster name**, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - For the **Cross-cluster API key**, paste the encoded cross-cluster API key. 2. Click **Add** to save the API key. 5. Restart the local deployment to reload the new setting. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart Elasticsearch**. If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. If you need to update the remote connection with different permissions later, refer to [Change a cross-cluster API key used for a remote connection](/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/ec-edit-remove-trusted-environment#edit-remove-trusted-environment-api-key). When adding the CA certificate in the next steps, use either the ECK-managed transport CA obtained [previously](#fetch-ca-cert), or the CA of the component that terminates TLS connections to clients. 1. Log in to [Elastic Cloud](https://cloud.elastic.co?page=docs&placement=docs-body). 2. Find your deployment on the home page or on the **Hosted deployments** page, then select **Manage** to access its settings menus. On the **Hosted deployments** page you can narrow down your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list. 3. From the navigation menu, select **Access and Security** > **Security**. 4. Select **Remote Connections > Add trusted environment** and choose **Self-managed**. Then click **Next**. 5. Select **API keys** as authentication mechanism and click **Next**. 6. When asked whether the Certificate Authority (CA) of the remote environment’s proxy or load-balancing infrastructure is public, select **No, it is private**. 7. Add the API key: 1. Fill both fields. - For the **Remote cluster name**, enter the alias of your choice. You will use this alias to connect to the remote cluster later. It must be lowercase and only contain letters, numbers, dashes and underscores. - For the **Cross-cluster API key**, paste the encoded cross-cluster API key. 2. Click **Add** to save the API key. 3. Repeat these steps for each API key you want to add. For example, if you want to use several clusters of the remote environment for CCR or CCS. 8. Add the CA certificate of the remote environment. 9. Provide a name for the trusted environment. That name will appear in the trust summary of your deployment's **Security** page. 10. Select **Create trust** to complete the configuration. 11. Restart the local deployment to reload the new settings. To do that, go to the deployment's main page, locate the **Actions** menu, and select **Restart Elasticsearch**. If the local deployment runs on version 8.14 or greater, you no longer need to perform this step because the keystore is reloaded automatically with the new API keys. If you need to update the remote connection with different permissions later, refer to [Change a cross-cluster API key used for a remote connection](/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/ec-edit-remove-trusted-environment#edit-remove-trusted-environment-api-key). ### Establish mutual trust between the clustersWhen using TLS certificates-based authentication, the first step is to establish trust between the two clusters, by adding the CA certificate and trust details of each environment into the other. #### Establish trust in the ECH clusterTo configure trust in the ECH deployment: 1. Save the Elasticsearch transport CA certificate of your ECK deployment. For an Elasticsearch cluster named `quickstart`, run: ```sh kubectl get secret quickstart-es-transport-certs-public -o go-template='{{index .data "ca.crt" | base64decode}}' > eck-ca.crt ``` This command saves the certificate to `eck-ca.crt`. 2. Update the trust settings for the Elastic Cloud Hosted deployment: 1. From the **Security** menu, select **Remote Connections > Add trusted environment**, choose **Self-managed**, and click **Next**. 2. Select **Certificates** as the authentication mechanism and click **Next**. 3. In **Add trusted CA certificate**, upload the `eck-ca.crt` file retrieved in the previous step. 4. In **Select trusted clusters**, configure the following: - Select **Trust clusters whose Common Name follows the Elastic pattern**. - For **Scope ID**, enter `.es.local`, replacing `` with the namespace of your ECK cluster. - In **Trust**, select **All deployments**. 5. In **Name the environment**, enter a name for the trusted environment. That name will appear in the trust summary of your deployment’s **Security** page. 6. Select **Create trust** to complete the configuration. 7. On the confirmation screen, when prompted **Have you already set up trust from the other environment?**, select **No, I have NOT set up trust from the other environment yet**. Download both the ECH deployment CA certificate and the `trust.yml` file. These files can also be retrieved from the **Security** page of the deployment. You’ll use these files to configure trust in the ECK deployment. #### Update the downloaded `trust.yml` file for ECK compatibilityThe `trust.yml` file you downloaded from the Cloud UI includes a subject name pattern that isn't valid for your ECK cluster. Before using it in your ECK cluster, you need to update the file with the pattern that matches your cluster. Replace the line corresponding to the `Scope ID` you entered when configuring trust in the ECH deployment: ```sh "*.node.*.cluster..es.local.account" ``` Replace it with the correct subject name for your ECK cluster. The new subject name should use the following pattern: ```sh "*.node...es.local" ``` If you don’t update this entry, Elasticsearch nodes of your ECK deployment might fail to start or join the cluster due to failed trust validation. For example, the original downloaded file might contain the following: ```yaml trust.subject_name: - "*.node.2dc556bb4bd040e00d0135683b66a2f6.cluster.1075999151.account" - "*.node.*.cluster..es.local.account" ``` For an ECK cluster named `quickstart` in the `default` namespace, the updated file should look like the following: ```yaml trust.subject_name: - "*.node.2dc556bb4bd040e89d0135683b66a2f6.cluster.1075708151.account" - "*.node.quickstart.default.es.local" ``` Apply the changes and save the `trust.yml` file. #### Establish trust in the ECK clusterTo configure trust in the ECK deployment: 1. In the same namespace as your Elasticsearch cluster, upload the ECH CA certificate that you downloaded from the Cloud UI as a Kubernetes secret: ```sh kubectl create secret generic remote-ech-ca --from-file=ca.crt= -n ``` 2. In the same namespace as your Elasticsearch cluster, upload the updated `trust.yml` file as a Kubernetes config map. For a cluster named `quickstart`, run the following command: ```sh kubectl create configmap quickstart-trust-ech --from-file=trust.yml= -n ``` 3. Edit the Elasticsearch Kubernetes resource to reference the ECH CA certificate and trust.yml file. This example assumes that the Kubernetes secret and config map created in the previous steps are named `remote-ech-ca` and `quickstart-trust-ech`, respectively: Apply these changes to all `nodeSets` of your cluster. Updating this configuration will restart all Elasticsearch pods, which might take some time to complete. ```yaml spec: nodeSets: - config: xpack.security.transport.ssl.certificate_authorities: - /usr/share/elasticsearch/config/ech-ca/ca.crt xpack.security.transport.ssl.trust_restrictions.path: /usr/share/elasticsearch/config/trust-filter/trust.yml podTemplate: spec: containers: - name: elasticsearch volumeMounts: - mountPath: /usr/share/elasticsearch/config/ech-ca name: remote-ech-ca - mountPath: /usr/share/elasticsearch/config/trust-filter name: eck-ech-trust volumes: - name: remote-ech-ca secret: secretName: remote-ech-ca - name: eck-ech-trust configMap: name: quickstart-trust-ech ``` ### Configure external access to the transport interface of your ECK cluster Expose the transport service (defaults to port `9300`) of your ECK cluster to allow external Elasticsearch clusters to connect: ```yaml apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: spec: transport: service: spec: type: LoadBalancer ``` 1. On cloud providers which support external load balancers, setting the `type` field to `LoadBalancer` provisions a load balancer for your service. Alternatively, expose the service `-es-transport` through one of the Kubernetes Ingress controllers that support TCP services. If you change the service’s `port` to expose a different port externally, set `targetPort` explicitly to `9300`, which is the default transport service listening port. Otherwise, Kubernetes uses the same value for both fields, resulting in failed connections. ## Connect to the remote cluster On the local deployment, use Kibana or the Elasticsearch API to add the remote ECK cluster with the following connection settings: - **Remote address**: Use the FQDN or IP address of the LoadBalancer service or alternative resource that you created to expose the remote cluster. - For API key-based authentication, use the server interface address and port. - For TLS certificate-based authentication, use the transport interface address and port. If you haven't changed the external listening port of the kubernetes service, the port should be `9443` for API key-based authentication, or `9300` for TLS certificate-based authentication. - **TLS server name**: You can try leaving this field empty first. If the connection fails, and your environment is presenting the ECK-managed certificates during the TLS handshake, use `-es-remote-cluster..svc` as the server name. For example, for a cluster named `quickstart` in the `default` namespace, use `quickstart-es-remote-cluster.default.svc`. ### Using Kibana To add a remote cluster in Kibana: 1. Go to the **Remote Clusters** management page in the navigation menu or use the [global search field](https://www.elastic.co/elastic/docs-builder/docs/3016/explore-analyze/find-and-organize/find-apps-and-objects). 2. Select **Add a remote cluster**. 3. In **Select connection type**, choose the authentication mechanism you prepared earlier (**API keys** or **Certificates**), and click **Next**. 4. In **Add connection information**, fill in the following fields: - **Remote cluster name**: This *cluster alias* is a unique identifier that represents the connection to the remote cluster and is used to distinguish local and remote indices. When using API key authentication, this alias must match the **Remote cluster name** you configured when [adding the API key](#configure-local-cluster). - **Remote address**: Enter the endpoint of the remote cluster, including the hostname, FQDN, or IP address, and the port. Make sure you use the correct port for your authentication method: - **API keys**: Use the port configured in the remote cluster interface of the remote cluster (defaults to `9443`). - **TLS Certificates**: Use the Elasticsearch transport port (defaults to `9300`). Starting with Kibana 9.2, this field also supports IPv6 addresses. When using an IPv6 address, enclose it in square brackets followed by the port number. For example: `[2001:db8::1]:9443`. - **Configure advanced options** (optional): Expand this section if you need to customize additional settings. - **TLS server name**: Specify a value if the certificate presented by the remote cluster is signed for a different name than the remote address. - **Socket connections**: Define the number of connections to open with the remote cluster. 5. Click **Next**. 6. In **Confirm setup**, click **Add remote cluster** (you have already established trust in a previous step). ### Using the Elasticsearch API To add a remote cluster, use the [cluster update settings API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings). Configure the following fields: - **Remote cluster alias**: When using API key authentication, the cluster alias must match the one you configured when [adding the API key](#configure-local-cluster). - **mode**: `proxy` - **proxy_address**: Enter the endpoint of the remote cluster, including the hostname, FQDN, or IP address, and the port. Both IPv4 and IPv6 addresses are supported. Make sure you use the correct port for your authentication method: - **API keys**: Use the port configured in the remote cluster interface of the remote cluster (defaults to `9443`). - **TLS Certificates**: Use the Elasticsearch transport port (defaults to `9300`). When using an IPv6 address, enclose it in square brackets followed by the port number. For example: `[2001:db8::1]:9443`. - **server_name**: Specify a value if the certificate presented by the remote cluster is signed for a different name than the proxy_address. This is an example of the API call to add or update a remote cluster: ```json PUT /_cluster/settings { "persistent": { "cluster": { "remote": { "alias-for-my-remote-cluster": { "mode":"proxy", "proxy_address": ":9443", "server_name": "" } } } } } ``` For a full list of available client connection settings in proxy mode, refer to the [remote cluster settings reference](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3016/reference/elasticsearch/configuration-reference/remote-clusters#remote-cluster-proxy-settings). ## Configure roles and users If you're using the API key–based security model for cross-cluster replication or cross-cluster search, you can define user roles with [remote indices privileges](/elastic/docs-builder/docs/3016/deploy-manage/users-roles/cluster-or-deployment-auth/role-structure#roles-remote-indices-priv) on the local cluster to further restrict the permissions granted by the API key. For more details, refer to [Configure roles and users](/elastic/docs-builder/docs/3016/deploy-manage/remote-clusters/remote-clusters-api-key#remote-clusters-privileges-api-key).