Deploy Elastic OpenTelemetry for Kubernetes observability

You can use the guided onboarding or deploy all components manually.

The deployment uses different Helm values files depending on where Elasticsearch is hosted. For Elastic Cloud Serverless and Elastic Cloud Hosted, the recommended path uses the Managed OTLP endpoint, which handles data enrichment server-side. For self-managed Elastic Stack, Elastic Cloud Enterprise (ECE), and Elastic Cloud on Kubernetes (ECK), data is sent directly to Elasticsearch using the elasticsearch exporter.

The guided onboarding simplifies deploying your Kubernetes components by setting up an API key and the needed integrations in the background.

Follow these steps to use the guided onboarding:

  1. In Kibana, navigate to Observability → Add data.
  2. Select Kubernetes, then choose Kubernetes monitoring with Elastic Agent.
  3. Follow the instructions to install the OpenTelemetry Operator using the Helm chart and the provided values.yaml.

When installing the OpenTelemetry Operator:

  • Make sure the elastic_otlp_endpoint shown in the installation command is valid for your Elastic Cloud Serverless project. This is a Managed OTLP endpoint URL, not an Elasticsearch URL.
  • The elastic_api_key shown in the installation command corresponds to an API key created by Kibana when the onboarding process is initiated.
  • Make sure the elastic_otlp_endpoint shown in the installation command is valid for your Elastic Cloud Hosted deployment. This is a Managed OTLP endpoint URL, not an Elasticsearch URL.
  • The elastic_api_key shown in the installation command corresponds to an API key created by Kibana when the onboarding process is initiated.
  • Make sure the elastic_endpoint shown in the installation command is valid for your environment. If not, replace it with the correct Elasticsearch endpoint.
  • The elastic_api_key shown in the installation command corresponds to an API key created by Kibana when the onboarding process is initiated.
Note

The default installation deploys an OpenTelemetry Operator with a self-signed TLS certificate. To automatically generate and renew certificates, refer to cert-manager integrated installation for instructions on customizing the values.yaml file before running the helm install command.

Follow these steps for a manual deployment of all components.

Before installing the operator, retrieve your credentials and install the required integrations.

Follow these steps to retrieve the managed OTLP endpoint URL for your Serverless project:

  1. In Elastic Cloud Serverless, open your Observability project.
  2. Go to Add data → Application → OpenTelemetry.
  3. Select Managed OTLP Endpoint in the second step.
  4. Copy the OTLP endpoint configuration value.
  5. Select Create API Key to generate an API key.

Find your endpoint

The easiest way to get your endpoint and API key is from the Add data screen in Kibana:

  1. Go to Add data.
  2. In the Connect directly to the endpoint section, select the OpenTelemetry tab.
  3. Copy the Endpoint value.
  4. Click Create key to generate an API key with the required privileges.

Alternatively, retrieve the endpoint from the Elastic Cloud Console and create an API key manually:

  1. Log in to the Elastic Cloud Console.
  2. Find your deployment in Hosted deployments, and select Manage.
  3. Under Application endpoints, cluster and component IDs, select Managed OTLP.
  4. Copy the Direct endpoint value from the flyout.

Create an API key

Note

The Elastic Cloud Managed OTLP Endpoint validates API keys using APM application privileges. Index-level privilege scoping is not yet supported, meaning that API keys with custom index-level role descriptors return a PermissionDenied error.

Using Kibana

  1. Go to Stack Management → API keys.

  2. Click Create API key, enter a name, and enable Control security privileges.

  3. In the role descriptors box, enter the following privileges:

    {
  "otlp_writer": {
    "applications": [
      {
        "application": "apm",
        "resources": ["*"],
        "privileges": ["event:write"]
      }
    ]
  }
}

  4. Click Create API key and copy the encoded value.

Using the Elasticsearch API

Use the Create API key API:

				POST /_security/api_key
					{
  "name": "otlp-writer",
  "role_descriptors": {
    "otlp_writer": {
      "applications": [
        {
          "application": "apm",
          "resources": ["*"],
          "privileges": ["event:write"]
        }
      ]
    }
  }
}

The event:write privilege for the apm application is the minimum required to send data through the Elastic Cloud Managed OTLP Endpoint.

Create an API Key.

Install the Kubernetes OpenTelemetry Assets and System OpenTelemetry Assets integrations in Kibana.

When using the Kibana onboarding UX, the previous actions are automatically handled by Kibana.

Follow these steps to install the operator:

  1. Create the opentelemetry-operator-system Kubernetes namespace:

    kubectl create namespace opentelemetry-operator-system

Create a secret with your credentials and install the Helm chart for your deployment type:

Replace <ELASTIC_OTLP_ENDPOINT> and <ELASTIC_API_KEY> in the following command to create a secret with your credentials.

kubectl create secret generic elastic-secret-otel \
--namespace opentelemetry-operator-system \
--from-literal=elastic_otlp_endpoint='<ELASTIC_OTLP_ENDPOINT>' \
--from-literal=elastic_api_key='<ELASTIC_API_KEY>'

Install the OpenTelemetry Operator using the kube-stack Helm chart with the managed_otlp values file:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm upgrade --install --namespace opentelemetry-operator-system opentelemetry-kube-stack open-telemetry/opentelemetry-kube-stack \
--values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.5.6/deploy/helm/edot-collector/kube-stack/managed_otlp/values.yaml' \
--version '0.16.0'

For details about the pipelines, refer to Managed OTLP Endpoint.

Replace <ELASTIC_OTLP_ENDPOINT> and <ELASTIC_API_KEY> in the following command to create a secret with your credentials.

kubectl create secret generic elastic-secret-otel \
--namespace opentelemetry-operator-system \
--from-literal=elastic_otlp_endpoint='<ELASTIC_OTLP_ENDPOINT>' \
--from-literal=elastic_api_key='<ELASTIC_API_KEY>'
Note

On Windows PowerShell, replace backslashes (\) with backticks (`) for line continuation and single quotes (') with double quotes (").

Install the OpenTelemetry Operator using the kube-stack Helm chart with the managed_otlp values file:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm upgrade --install --namespace opentelemetry-operator-system opentelemetry-kube-stack open-telemetry/opentelemetry-kube-stack \
--values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.5.6/deploy/helm/edot-collector/kube-stack/managed_otlp/values.yaml' \
--version '0.16.0'

For details about the pipelines, refer to Managed OTLP Endpoint.

Create a secret in the new namespace with the following command:

kubectl create -n opentelemetry-operator-system secret generic elastic-secret-otel \
  --from-literal=elastic_endpoint='YOUR_ELASTICSEARCH_ENDPOINT' \
  --from-literal=elastic_api_key='YOUR_ELASTICSEARCH_API_KEY'

Replace:

  • YOUR_ELASTICSEARCH_ENDPOINT: Elasticsearch endpoint (with https:// prefix). For example: https://1234567.us-west2.gcp.elastic-cloud.com:443.
  • YOUR_ELASTICSEARCH_API_KEY: Elasticsearch API Key created in the previous step.

Run the following commands to deploy the opentelemetry-kube-stack Helm chart:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm upgrade --install --namespace opentelemetry-operator-system opentelemetry-kube-stack open-telemetry/opentelemetry-kube-stack \
      --values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.5.6/deploy/helm/edot-collector/kube-stack/values.yaml' \
      --version 0.16.0

This configuration includes the elasticapm connector and processor, which handle APM trace aggregations locally. When using the Managed OTLP endpoint, this processing happens server-side instead. For details about the pipelines, refer to Direct ingestion into Elasticsearch.

If you need to customize the configuration, copy the values file you used and adapt it to your needs. Refer to the compatibility matrix for a complete list of available manifests in the release branches.

On OpenShift, follow the operator installation instructions and in step 4 use the following command to apply the OpenShift values file after the base values file:

helm upgrade --install --namespace opentelemetry-operator-system opentelemetry-kube-stack open-telemetry/opentelemetry-kube-stack \
      --values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.5.6/deploy/helm/edot-collector/kube-stack/values.yaml' \
      --values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.5.6/deploy/helm/edot-collector/kube-stack/openshift/values.yaml' \
      --version 0.16.0

The OpenShift values file configures the minimum permissions required to run the chart and collect Kubernetes telemetry. It creates a custom security context constraint (SCC) for the daemon collector. Refer to the OpenShift values file for the complete configuration.

By default, the daemon collector runs as root to read host files. To run it as a non-root user, also apply the rootless values file after the OpenShift values file:

--values 'https://raw.githubusercontent.com/elastic/elastic-agent/refs/tags/v9.5.6/deploy/helm/edot-collector/kube-stack/openshift/rootless-values.yaml'

The rootless configuration uses an init container to make /var/lib/otelcol writable so that the collector can persist file log checkpoints.

If the OpenTelemetry Operator is already installed through Operator Lifecycle Manager (OLM), such as from OperatorHub, add the following flags to prevent the chart from installing another operator and its custom resource definitions (CRDs):

--set crds.installOtel=false \
--set opentelemetry-operator.enabled=false

Perform the following checks to verify that everything is running properly:

Ensure the following components are running without errors:

  • Operator Pod
  • DaemonSet Collector Pod
  • Deployment Collector Pod

Confirm that the Instrumentation object is deployed and configured with a valid endpoint.

Verify that the [OTEL][Metrics Kubernetes] Cluster Overview dashboard in Kibana is displaying data correctly.

In Discover, confirm the availability of log data. In ES|QL mode, run FROM logs-*. In classic mode, select the logs-* data view.

In Discover, ensure metrics data is available. In ES|QL mode, run FROM metrics-*. In classic mode, select the metrics-* data view.