Monitoring Logstash with OpenTelemetry

Logstash can export metrics to any OpenTelemetry Protocol (OTLP) compatible backend, like Elastic, Prometheus, and others.

The OpenTelemetry metrics exporter sends Logstash runtime metrics directly using the OpenTelemetry Protocol (OTLP). This provides a standardized way to collect and export metrics without requiring an intermediate collector, though you can also route metrics through an OpenTelemetry Collector if needed.

To enable OpenTelemetry metrics export, add the following settings to your logstash.yml file:

otel.metrics.enabled: true
otel.exporter.otlp.endpoint: "http://localhost:4317"
otel.metric.export.interval: "10s"
otel.exporter.otlp.protocol: "grpc"
		

OpenTelemetry settings can be configured using Java system properties or by editing the logstash.yml file. The resolution order is:

1. Java system properties (highest priority). For example, -Dotel.service.name=my-service
2. logstash.yml (lowest priority)

Supported system properties and their Logstash setting equivalents:

To send metrics directly to the Elastic Cloud Managed OTLP Endpoint:

1. Get your Elastic Cloud OTLP endpoint from your deployment's settings.
2. Create an API key with appropriate permissions.
3. Configure Logstash:

otel.metrics.enabled: true
otel.exporter.otlp.endpoint: "https://your-deployment.apm.us-central1.gcp.cloud.es.io:443"
otel.exporter.otlp.protocol: "http"
otel.exporter.otlp.headers: "Authorization=ApiKey your-base64-encoded-api-key"
		

You can also send metrics to an OpenTelemetry Collector, which can then forward them to multiple backends:

otel.metrics.enabled: true
otel.exporter.otlp.endpoint: "http://otel-collector:4317"
otel.exporter.otlp.protocol: "grpc"
		

Example OpenTelemetry Collector configuration to forward to Elasticsearch:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:

exporters:
  elasticsearch:
    endpoints: ["https://your-elasticsearch-host:9200"]
    api_key: "your-api-key"
    mapping:
      mode: otel

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [elasticsearch]
		

Logstash exports the following metrics via OpenTelemetry:

Pipeline metrics include a pipeline.id attribute to identify the pipeline.

These metrics are available when using persistent queues (queue.type: persisted).

Plugin metrics include pipeline.id, plugin.type, and plugin.id attributes.

These metrics are available when running on Linux with cgroups enabled, for example in containers.

The following resource attributes are automatically added to all metrics:

Additional resource attributes can be added using the otel.resource.attributes setting.

When sending metrics to Elastic Cloud via the native OTLP endpoint, metrics are stored in APM data streams (.ds-metrics-apm.app.logstash-*). You can view them in:

1. Find your Logstash service under Observability → APM → Services.
2. Query metrics directly in Observability → Metrics Explorer.
3. Search the metrics-apm.app.logstash-* data view using Discover.

When using an OpenTelemetry Collector with the Elasticsearch exporter, create a data view matching your configured index pattern. For example, metrics-otel-*.

To see detailed OpenTelemetry SDK logs, add the following to config/log4j2.properties:

logger.otel.name = io.opentelemetry
logger.otel.level = debug
		

Connection refused errors

Verify the endpoint is accessible:

• For gRPC (default): Port 4317
• For HTTP: Port 4318 with /v1/metrics path automatically appended
• If no port is specified, defaults are used: 80 for http://, 443 for https://

Authentication errors

Ensure the otel.exporter.otlp.headers is correctly formatted:

• For API keys: ApiKey base64-encoded-key
• For Bearer tokens: Bearer your-token

Metrics not appearing

• Check that otel.metrics.enabled is set to true
• Verify the export interval hasn't been set too high
• Check Logstash logs for export errors