LLM observability with Elastic OpenTelemetry

Applications make more and more use of Generative Artificial Intelligence (GenAI). Telemetry data in terms of spans, metrics and logs when communicating with the GenAI APIs becomes more and more important to operate the application in production and understand the application's behavior and health state.

Elastic currently supports LLM observability through the Elastic OpenTelemetry. The Elastic OTel Java, Elastic OTel Node.js, and Elastic OTel Python distributions support LLM observability as a tech preview.

Supported technologies

The following LLM platforms are supported:

Technology	Elastic OTel Java	Elastic OTel Node.js	Elastic OTel Python
OpenAI Client	✅	✅	✅
AWS Bedrock	❌	❌	✅
Google Vertex AI	❌	❌	✅

See the Supported Technologies section in the corresponding Elastic OTel SDK for detailed information on supported versions.

Quickstart

Follow these steps to instrument LLMs using Elastic OpenTelemetry.

Instrument your LLM application

Select the environment and target system from the quick start overview and follow the setup instructions to instrument your LLM application. Instrumentation for the supported technologies is enabled by default.

Configuration

See the Configuration section in the corresponding Elastic OTel SDK to turn on or off specific instrumentations and check which instrumentation is active by default.

When you complete the setup and configuration of the Elastic OTel SDK and there is a workload on your application, start checking for telemetry data in Kibana. If there's no data showing up, see the troubleshooting of the corresponding Elastic OTel SDK.