Retrieve context from an AI index

Query an AI index to retrieve its Knowledge Indicators (KIs) as context for an agent or application. You can call the Context Engine APIs directly. Agent integrations can expose tools that perform the list, describe, and query sequence.

To query an AI index, you need:

• The contextEngine:enabled advanced setting turned on in the Kibana space you want to query.
• At least one AI index that contains KIs.
• The Context Engine Read privilege in that space.
• The Elasticsearch read and view_index_metadata privileges for the AI index's backing indices.

For direct API access, you also need credentials for the caller. The examples on this page use an API key.

Choose a query mode based on whether your code or an agent controls the retrieval sequence:

Run the requests from Kibana Console or use cURL from an application environment. Console uses your current session and Kibana space.

For cURL, set the Kibana URL and encoded API key used by the examples:

export KIBANA_URL="https://my-deployment.kb.us-east-1.aws.elastic.cloud"
export KIBANA_API_KEY="VnVhQ2ZHY0JDZGJrU..."
		

For a non-default space, include /s/<space-id> at the end of KIBANA_URL in the cURL examples.

Use the list AI indices API to retrieve the AI indices that the caller can read:

				GET kbn:/api/context_engine/ai_index
		

curl -X GET "${KIBANA_URL}/api/context_engine/ai_index" \
  -H "Authorization: ApiKey ${KIBANA_API_KEY}" \
  -H "elastic-api-version: 2023-10-31"
		

Use each entry's description to determine whether the AI index is relevant. Keep its id for the describe request. The response also contains dest.value, which is the exact ES|QL target for the AI index.

An AI index is omitted when the caller cannot read its backing index. It can still appear before an automation has written any KIs to its backing index.

Use the describe AI index API before constructing a query:

				GET kbn:/api/context_engine/ai_index/customer_support/_describe
		

export AI_INDEX_ID="customer_support"

curl -X GET "${KIBANA_URL}/api/context_engine/ai_index/${AI_INDEX_ID}/_describe" \
  -H "Authorization: ApiKey ${KIBANA_API_KEY}" \
  -H "elastic-api-version: 2023-10-31"
		

The response field contains a context block with the AI index's purpose, ES|QL target, fields, semantic fields, KI types, tags, and example queries. Use its target and field names verbatim. Fields can differ between AI indices.

Construct an ES|QL query from the describe response and submit it with the query AI indices API. The following example filters an AI index for KIs of type faq:

				POST kbn:/api/context_engine/ai_index/_query
					{
  "query": "FROM ai-index-ds-customer_support | WHERE type == ?type | KEEP title, description, content, type, tags | LIMIT 10",
  "params": { "type": "faq" },
  "limit": 10
}
		

curl -X POST "${KIBANA_URL}/api/context_engine/ai_index/_query" \
  -H "Authorization: ApiKey ${KIBANA_API_KEY}" \
  -H "elastic-api-version: 2023-10-31" \
  -H "kbn-xsrf: true" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "FROM ai-index-ds-customer_support | WHERE type == ?type | KEEP title, description, content, type, tags | LIMIT 10",
    "params": { "type": "faq" },
    "limit": 10
  }'
		

Replace ai-index-ds-customer_support and the selected fields with values returned by the describe operation. Pass user-provided values as named parameters instead of adding them directly to the query string.

The query operation applies the current Kibana space to the request and returns only documents visible in that space. Do not add a space condition to the ES|QL query.

By default, the query operation returns only active, unexpired KIs. For an AI index backed by a data stream, it also returns only the latest revision of each KI. To query deleted or expired KIs, add conditions that reference governance.lifecycle.status or expires_at. When a query references either field, the operation does not apply the corresponding default filter.

An agent can perform the retrieval sequence when its integration provides access to the list, describe, and query operations. How those operations become available depends on the integration:

• With Elastic Agent Builder, assign an AI index to an agent. Elastic Agent Builder automatically adds the three tools and retrieval instructions.
• With LangChain, wrap the three Context Engine API operations as LangChain tools, then provide them to the agent with instructions that establish their order.

In each case, the agent uses the AI index description to select relevant context, constructs ES|QL from the describe response, and uses the returned KIs to complete the task.

Confirm the following details across the three operations:

• The list response includes the expected AI index and the description matches the question or task.
• The describe response supplies the expected ES|QL target, fields, KI types, tags, and example queries.
• The query response contains columns and values that represent relevant KIs from the selected AI index.

When an agent runs the sequence, inspect its tool calls to confirm that it selected the expected AI index, used the target and fields from the describe response, and retrieved relevant context.

Use the following table to resolve common retrieval problems: