Loading

ES|QL metadata fields

ES|QL can access metadata fields.

To access these fields, use the METADATA directive with the FROM source command. For example:

FROM index METADATA _index, _id

The following metadata fields are available in ES|QL:

Metadata field Type Description
_id keyword Unique document ID.
_ignored keyword Names every field in a document that was ignored when the document was indexed.
_index keyword Index name.
_index_mode keyword Index mode. For example: standard, lookup, or logsdb.
_score float Query relevance score (when enabled). Scores are updated when using full text search functions.
_size integer Size in bytes of the original _source field when the mapper-size plugin is enabled.
_source Special _source type Original JSON document body passed at index time (or a reconstructed version if synthetic _source is enabled).
_version long Document version number
  • Metadata fields are only available when the data source is an index
  • The _source type is not supported by functions
  • Only the FROM command supports the METADATA directive
  • Once enabled, metadata fields work like regular index fields

Once enabled, metadata fields are available to subsequent processing commands, just like other index fields:

FROM ul_logs, apps METADATA _index, _version
| WHERE id IN (13, 14) AND _version == 1
| EVAL key = CONCAT(_index, "_", TO_STR(id))
| SORT id, _index
| KEEP id, _index, _version, key
id:long _index:keyword _version:long key:keyword
13 apps 1 apps_13
13 ul_logs 1 ul_logs_13
14 apps 1 apps_14
14 ul_logs 1 ul_logs_14

Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field:

FROM employees METADATA _index, _id
| STATS max = MAX(emp_no) BY _index
max:integer _index:keyword
10100 employees 
FROM products METADATA _score
| WHERE MATCH(description, "wireless headphones")
| SORT _score DESC
| KEEP name, description, _score
Tip

Refer to ES|QL for search for more information on relevance scoring and how to use _score in your queries.

Retrieving _source

Using _source is useful when you want to retrieve most or all fields from a document.

You should consider retrieving _source instead of individual fields when:

  • You need several fields from a document, and most of them are text fields.
  • You have long text fields or geoshape fields in your documents.
  • You want the original document instead of the indexed values for your fields.
  • You have nested objects or arrays that you want to preserve in their original structure.

You might want to avoid retrieving _source when:

  • Your fields are stored as doc_values. doc_values access is faster than _source.
  • Your index uses synthetic source. Accessing _source in synthetic source mode has a performance penalty.
  • The text fields you need are stored fields.

Using _source or selecting fields are both valid options, but can have performance and data format implications that you should consider based on your use case.