Loading

Deploy the EDOT gateway on Kubernetes

This page extends Send data from a contrib OpenTelemetry Collector with Kubernetes-specific deployment steps. If you haven't read that page, start there for context on the architecture and components.

In Kubernetes, you deploy the EDOT gateway as a Deployment and expose it as a Service so contrib Collectors can reach it using cluster DNS.

The EDOT Collector image for standalone use is docker.elastic.co/elastic-agent/elastic-otel-collector. Unlike the full Elastic Agent image, this image's entrypoint unconditionally starts in otel mode — no extra environment variables are required.

  • A running Kubernetes cluster
  • kubectl configured to access the cluster
  • A running self-managed Elasticsearch cluster reachable from the Kubernetes cluster
  1. Create a secret for credentials

    kubectl create secret generic elastic-secret-otel \
      --from-literal=elastic_endpoint='https://your-elasticsearch:9200' \
      --from-literal=elastic_api_key='your-encoded-api-key'
    		
  2. Deploy the EDOT gateway

    Apply the following manifest. The ConfigMap holds the gateway configuration, the Deployment runs the EDOT Collector, and the Service exposes port 4317 for contrib Collectors inside the cluster.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: edot-gateway-config
    data:
      gateway.yaml: |
        receivers:
          otlp:
            protocols:
              grpc:
                endpoint: 0.0.0.0:4317
              http:
                endpoint: 0.0.0.0:4318
        connectors:
          elasticapm: {}
        processors:
          batch:
            send_batch_size: 1000
            timeout: 1s
            send_batch_max_size: 1500
          batch/metrics:
            send_batch_max_size: 0
            timeout: 1s
          elasticapm: {}
        exporters:
          elasticsearch/otel:
            endpoints:
              - ${env:ELASTIC_ENDPOINT}
            api_key: ${env:ELASTIC_API_KEY}
            mapping:
              mode: otel
        service:
          pipelines:
            traces:
              receivers: [otlp]
              processors: [batch, elasticapm]
              exporters: [elasticapm, elasticsearch/otel]
            metrics:
              receivers: [otlp]
              processors: [batch/metrics]
              exporters: [elasticsearch/otel]
            metrics/aggregated-otel-metrics:
              receivers: [elasticapm]
              processors: []
              exporters: [elasticsearch/otel]
            logs:
              receivers: [otlp]
              processors: [batch]
              exporters: [elasticsearch/otel]
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: edot-gateway
      labels:
        app: edot-gateway
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: edot-gateway
      template:
        metadata:
          labels:
            app: edot-gateway
        spec:
          containers:
            - name: edot-gateway
              image: docker.elastic.co/elastic-agent/elastic-otel-collector:{{version.edot_collector}}
              args: ["--config", "/etc/edot/gateway.yaml"]
              env:
                - name: ELASTIC_ENDPOINT
                  valueFrom:
                    secretKeyRef:
                      name: elastic-secret-otel
                      key: elastic_endpoint
                - name: ELASTIC_API_KEY
                  valueFrom:
                    secretKeyRef:
                      name: elastic-secret-otel
                      key: elastic_api_key
              ports:
                - containerPort: 4317
                - containerPort: 4318
              volumeMounts:
                - name: config
                  mountPath: /etc/edot
          volumes:
            - name: config
              configMap:
                name: edot-gateway-config
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: edot-gateway
    spec:
      selector:
        app: edot-gateway
      ports:
        - name: otlp-grpc
          port: 4317
          targetPort: 4317
        - name: otlp-http
          port: 4318
          targetPort: 4318
    		
    1. gRPC
    2. HTTP
  3. Configure the contrib Collector

    Point the contrib Collector's OTLP exporter at the gateway Service:

    exporters:
      otlp:
        endpoint: "edot-gateway:4317"
        tls:
          insecure: true
    		
    1. Set to false and configure ca_file for production
  4. Verify data in Kibana

    After the gateway pods are running and your contrib Collectors point to the edot-gateway Service, confirm that data flows in:

    1. Check your services in ObservabilityAPM.
    2. Check the traces-generic.otel-default, logs-generic.otel-default, and metrics-generic.otel-default data streams in Discover.

    If no data appears, refer to No logs, metrics, or traces visible in Kibana.

Note

For comprehensive Kubernetes observability (including host metrics, pod logs, Kubernetes events, and cluster metrics), use the opentelemetry-kube-stack Helm chart with the Elastic values instead. Refer to Kubernetes observability for more information.