---
title: Context Engine concepts
description: Understand AI indices, sources, automations, Knowledge Indicators, and agent access in Context Engine.
url: https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/concepts
products:
  - Elastic Documentation
  - Elastic Observability
  - Elastic Security
  - Elasticsearch
  - Kibana
applies_to:
  - Elastic Cloud Serverless: Experimental
  - Elastic Stack: Planned
---

# Context Engine concepts
<important>
  This page is currently hidden from the documentation navigation.The following Context Engine pages are live:
  - [Landing page](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine)
    - [Context Engine quickstart](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/quickstart)
  - [Context Engine concepts](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/concepts)
  - [Build and maintain an AI index](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/build-and-maintain-ai-index)
    - [Create and manage AI indices](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/create-and-manage-ai-indices)
  - [Add and manage sources](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/add-and-manage-sources)
  - [Create and manage automations](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/create-and-manage-automations)
  - [Evaluate and improve KIs](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/evaluate-and-improve-knowledge-indicators)
  - [Use an AI index](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/use-an-ai-index)
    - [Retrieve context](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/retrieve-context-from-ai-index)
  - [Configure agents to use an AI index](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/use-context-engine-with-agents)
    - [Use Context Engine with Agent Builder](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/use-context-engine-with-agent-builder)
  - [Query AI indices from LangChain](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/langchain-integration)
  - [Context Engine APIs](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/context-engine-api)
</important>

The following concepts help you understand how Context Engine turns source data into context for agents and applications:
- [AI indices](#ai-indices) group the configuration and generated context for a particular purpose.
- [Sources](#sources) identify the raw data available to automations.
- [Automations and workflows](#automations-and-workflows) analyze sources and generate or refresh KIs.
- [Knowledge Indicators (KIs)](#knowledge-indicators) are documents of reusable context generated from source data.
- [Agents and applications](#agents-and-applications) retrieve and apply the generated context.
- [Tools, system instructions, and skills](#tools-system-instructions-and-skills) cover the operations agents can perform and the guidance for using them.
- [Agent traces](#agent-traces) provide feedback about how well the context supports real questions.


## AI indices

An AI index groups the metadata, [sources](#sources), [automations](#automations-and-workflows), optional [agent traces](#agent-traces), and Elasticsearch storage for a particular body of context. Its name and description help an [agent](#agents-and-applications) determine whether that context is relevant to a question. Its generated [Knowledge Indicators (KIs)](#knowledge-indicators) provide the context that agents and applications query.
Creating an AI index does not ingest source data or generate knowledge. Sources define the data that its automations can analyze, and KIs are stored as the generated context. To create, update, or delete an AI index, refer to [Create and manage AI indices](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/create-and-manage-ai-indices). To plan and maintain one, refer to [Build and maintain an AI index](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/build-and-maintain-ai-index). For the complete UI-led example, follow the [Context Engine quickstart](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/quickstart).

### Managed AI indices

A managed AI index is a built-in AI index supplied by an Elastic integration. The owning Elastic feature defines its configuration and supplies its [KIs](#knowledge-indicators), giving [agents](#agents-and-applications) access to product-provided context without requiring you to configure its [sources](#sources) or [automations](#automations-and-workflows).
Managed AI indices have a **Managed** badge in **Context**. Agents can discover and query them in the same way as user-created AI indices, subject to the same access checks. You can open a managed AI index and inspect its KIs, but you cannot edit its description, sources, automations, or [agent traces](#agent-traces), or delete it. To build context for your own use case, create a user-created AI index instead.

## Sources

A source identifies data that an [automation](#automations-and-workflows) can use to generate [Knowledge Indicators (KIs)](#knowledge-indicators).
You can add source data in the following ways:
- **Elasticsearch index, data stream, or alias:** Select an Elasticsearch data object. Context Engine creates a source query in the form `FROM <name>`.
- **Connector:** Select a configured connector to an external system, such as Google Drive or GitHub.
- **Advanced ES|QL:** Write an ES|QL query when you need to select data from multiple Elasticsearch indices or narrow the records available to the automation.

Internally, both an Elasticsearch selection and an advanced query are stored as ES|QL sources. How an automation analyzes any source depends on its workflow.
To add or update source data, refer to [Add and manage sources](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/add-and-manage-sources). For an end-to-end example, follow the [Context Engine quickstart](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/quickstart).

## Automations and workflows

An automation generates or refreshes [Knowledge Indicators (KIs)](#knowledge-indicators) from an [AI index's](#ai-indices) [sources](#sources). Context Engine implements each automation as a workflow in [Elastic Workflows](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/workflows). The workflow defines the operations and [instructions](#tools-system-instructions-and-skills) used to analyze the source and write the result.
The workflow can retrieve mappings, run ES|QL samples and aggregations, generate structured content with an AI prompt, assemble the KI, verify generated queries, and write the KI when verification passes. You can inspect the workflow to verify the scope of its queries and the evidence behind its output. Refer to [Anatomy of a workflow](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/workflows/authoring-techniques/anatomy) to understand its triggers, constants, steps, and execution lifecycle.
Passing syntax and runtime checks means that a generated query parses and runs. It does not confirm that the query's fields, grouping, or calculations answer the intended question. To create, review, run, and schedule an automation, refer to [Create and manage automations](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/create-and-manage-automations). For an end-to-end example, follow the [Context Engine quickstart](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/quickstart).

## Knowledge Indicators

A Knowledge Indicator (KI) is a document that records reusable context derived from [raw source data](#sources). [Automations](#automations-and-workflows) generate KIs and store them in an [AI index](#ai-indices), where [agents](#agents-and-applications) retrieve them to answer recurring questions without repeatedly finding and interpreting the same information. This can reduce response time and model token use. Depending on the use case, a KI can contain business meaning, derived findings, source limitations, or verified query patterns for retrieving current details.
Unlike a cached answer to one request, a KI can support different questions and multiple agents. When evaluation reveals a gap, improve the source or automation and regenerate the KI so the correction is available to future retrievals.
The [generation strategy](/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/build-and-maintain-ai-index#select-a-generation-strategy) determines how an automation divides source data into KIs and how much information each KI contains. To examine a generated KI and evaluate its usefulness, refer to [Evaluate and improve Knowledge Indicators](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/evaluate-and-improve-knowledge-indicators).

## Agents and applications

Agents and applications retrieve [KIs](#knowledge-indicators) from an [AI index](#ai-indices) as context. In Elastic Agent Builder, you make an AI index available by adding it to an agent's configuration. Other agent frameworks and applications can use supported Context Engine operations through Model Context Protocol (MCP) [tools](#tools-system-instructions-and-skills) or APIs.
At retrieval time, an agent or application lists the AI indices it can access, describes a relevant AI index, and queries it for KIs. For the available query modes and complete retrieval workflow, refer to [Retrieve context from an AI index](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/retrieve-context-from-ai-index).
To choose a retrieval or integration route, refer to [Use an AI index](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/use-an-ai-index). For the built-in integration, follow [Use Context Engine with Elastic Agent Builder](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/context-engine/use-context-engine-with-agent-builder).

## Tools, system instructions, and skills

A tool is an operation an [agent](#agents-and-applications) can perform. MCP exposes tools to compatible clients, while APIs let applications call supported operations directly or wrap them as tools.
System instructions and skills guide an agent in deciding when and how to use those operations. A skill packages reusable instructions and can associate them with tools and reference content, but it is not itself a tool, connection mechanism, or agent framework. Refer to the [Context Engine entries in the built-in skills reference](/elastic/docs-builder/docs/4302/explore-analyze/ai-features/agent-builder/builtin-skills-reference#agent-builder-context-engine-skills) for the skills that support context generation, evaluation, and retrieval.

## Agent traces

Agent traces record how an [agent](#agents-and-applications) runs, including its model calls and tool calls. An [AI index](#ai-indices) can reference traces from an Elastic Agent Builder agent or from a data stream that contains OpenTelemetry generative AI spans.
Context Engine can analyze these traces to identify query errors, empty retrievals, and cases where an agent queries raw data because its [KIs](#knowledge-indicators) do not cover the question. This feedback can expose gaps in the index's [sources](#sources), [automations](#automations-and-workflows), or generated context. Agent traces are feedback about context use. They are not sources from which automations generate KIs.
Agent traces do not update KIs by themselves. Use their evidence to refine sources or automations, then regenerate and retest the KIs. This process turns repeated agent work into improvements to shared context instead of one-off changes to individual agent instructions.
For information about trace collection, contents, privacy, and access, refer to [Collect Elastic Agent Builder traces](https://www.elastic.co/elastic/docs-builder/docs/4302/explore-analyze/ai-features/agent-builder/collect-traces).