Context Engine

Important

This page is currently hidden from the documentation navigation.

The following Context Engine pages are live:

Context Engine enables you to distill raw source data into context optimized for retrieval by agents and applications. Without reusable context, agents can spend each run rediscovering which data to use, what it means, and how to query it. Building this context up front reduces repeated source data scanning and interpretation, helping agents respond faster and use fewer model tokens.

Automations distill source data into Knowledge Indicators, which agents and applications retrieve from an AI index:

Sources feed automations that generate Knowledge Indicators in an AI index. Agents retrieve the Knowledge Indicators and can use their guidance to query source data directly for more detailed or up-to-date information. Agent traces can reveal where the context needs improvement.

Context Engine is useful when agents repeatedly need to interpret large, complex, or changing bodies of data. For example, you can:

  • Turn technical documentation, policies, cases, or runbooks into reusable explanations, procedures, and limitations.
  • Give agents business definitions and verified query patterns for working with structured data.
  • Build context that develops across records, such as profiles of services, systems, accounts, or projects.
  • Surface significant findings or conditions without making every agent analyze all source records.

These use cases share the same advantage: move recurring interpretation into an automation, reuse the resulting context across questions and agents, and improve it as agent traces reveal gaps.

For example, consider an agent asked to summarize the issues affecting your largest customers. The agent still needs access to the account, support, and contract data that contains the answer. Context Engine changes how much discovery and interpretation the agent must perform before it can use that data:

Without Context Engine With Context Engine
Discover the relevant sources and inspect their structure during the agent run. Retrieve KIs that describe the relevant sources and when to use them.
Infer business meaning, relationships between customers, and known limitations. Reuse context that automations generated from the source data.
Develop and test queries before addressing the task. Follow verified query guidance when source data must be queried directly.
Repeat the same interpretation in later runs or other agents. Refine the automation and regenerate its KIs so future runs can use the improvement.

To build and use context with Context Engine:

  1. Create an AI index

    Create an AI index for a defined area of context.

  2. Add source data

    Add one or more sources that contain the relevant data.

  3. Generate Knowledge Indicators

    Configure automations, implemented as Elastic Workflows, to generate and refresh KIs from those sources.

  4. Make the context available

    Make the AI index available to an agent or application.

  5. Configure context retrieval

    Configure the agent or application with the appropriate tools and instructions to retrieve KIs as context and use KI guidance to query source data directly when more detailed or up-to-date information is required.

  6. Evaluate and improve the context

    Review KIs and agent traces to identify missing, misleading, or underused context. Refine the sources or automations, regenerate the KIs, and repeat as questions and source data change.

Follow the Context Engine quickstart to create an AI index from existing Elasticsearch data, generate your first KI, and test how an Elastic Agent Builder agent uses it.

Learn how AI indices, sources, automations, KIs, and agent access fit together in Context Engine concepts.

Learn how to choose source data, select a KI generation strategy, review automations, and maintain useful context in Build and maintain an AI index.

Learn how to retrieve context directly or through an agent or application in Use an AI index.

Use the Context Engine API guide to find the available operations and links to the complete Kibana API reference.