ES|QL for search

Elastic Stack Technical Preview Serverless Technical Preview

This page provides an overview of how to use ES|QL for search use cases.

The following table summarizes the key search features available in ES|QL and when they were introduced.

ES|QL can be used for both simple filtering and relevance-based searching:

• Filtering removes non-matching documents without calculating relevance scores
• Searching both filters documents and ranks them by how well they match the query

Note that filtering is faster than searching, because it doesn't require score calculations.

To get the most relevant results first, you need to use METADATA _score and sort by score. For example:

FROM books METADATA _score
| WHERE match(title, "Shakespeare") OR match(plot, "Shakespeare")
| SORT _score DESC

When working with relevance scoring in ES|QL:

• If you don't include METADATA _score in your query, this only performs filtering operations with no relevance calculation.
• When you include METADATA _score, any search function included in WHERE conditions contribute to the relevance score. This means that every occurrence of MATCH, QSTR and KQL will affect the score.
• Filtering operations that are not search functions, like range conditions and exact matches, don't affect the score.
• Including METADATA _score doesn't automatically sort your results by relevance. You must explicitly use SORT _score DESC or SORT _score ASC to order your results by relevance.

ES|QL offers two syntax options for match, which replicate the functionality of match queries in Query DSL.

Use the compact operator syntax (:) for simple text matching with default parameters.

FROM logs | WHERE message: "connection error"

Use the match() function syntax when you need to pass additional parameters:

FROM products | WHERE match(name, "laptop", { "boost": 2.0 })

These full-text functions address several key limitations that existed for text filtering in ES|QL:

• They work directly on multivalued fields, returning results when any value in a multivalued field matches the query
• They leverage analyzers, ensuring the query is analyzed with the same process as the indexed data (enabling case-insensitive matching, ASCII folding, stopword removal, and synonym support)
• They are highly performant, using Lucene index structures rather than pattern matching or regular expressions to locate terms in your data

Refer to this blog for more context: Introducing full text filtering in ES|QL.

The qstr function provides the same functionality as the Query DSL's query_string query. This is for advanced use cases, such as wildcard searches, searches across multiple fields, and more.

FROM articles METADATA _score
| WHERE QSTR("(new york city) OR (big apple)")
| SORT _score DESC
| LIMIT 10

For complete details, refer to the Query DSL query_string docs.

Use the KQL function to use the Kibana Query Language in your ES|QL queries:

FROM logs*
| WHERE KQL("http.request.method:GET AND agent.type:filebeat")

The kql function is useful when transitioning queries from Kibana's Discover, Dashboard, or other interfaces that use KQL. This will allow you to gradually migrate queries to ES|QL without needing to rewrite them all at once.

You can perform semantic searches over semantic_text field types using the same match syntax as full-text search.

This example uses the match operator ::

FROM articles METADATA _score
| WHERE semantic_content: "What are the impacts of climate change on agriculture?"
| SORT _score DESC

This example uses the match function:

FROM articles METADATA _score
| WHERE match(semantic_content, "What are the impacts of climate change on agriculture?")
| SORT _score DESC

Hybrid search combines lexical and semantic search with custom weights:

FROM books METADATA _score
| WHERE match(semantic_title, "fantasy adventure", { "boost": 0.75 }) 
    OR match(title, "fantasy adventure", { "boost": 0.25 })
| SORT _score DESC

Refer to ES|QL limitations for a list of known limitations.

• Search and filter with ES|QL: Hands-on tutorial for getting started with search tools in {esql}
• Semantic search with semantic_text: Learn how to use the semantic_text field type

• Search functions: Complete reference for all search functions
• Limitations: Current limitations for search in ES|QL

• Analysis: Learn how text is processed for full-text search
• Semantic search: Get an overview of semantic search in Elasticsearch
• Query vs filter context: Understand the difference between query and filter contexts in Elasticsearch

• Introducing full text filtering in ES|QL: Overview of ES|QL's text filtering capabilities