---
title: ES|QL rules
description: Create detection rules using Elasticsearch Query Language (ESQL) with aggregation and pipeline processing.
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/esql
products:
- Elastic Cloud Serverless
- Elastic Security
applies_to:
- Serverless Security projects: Generally available
- Elastic Stack: Generally available
---
# ES|QL rules
## Overview
ES|QL rules use [Elasticsearch Query Language (ES|QL)](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql) to query source events and aggregate or transform data using a pipeline syntax. Query results are returned as a table where each row becomes an alert. ES|QL rules combine the flexibility of a full query pipeline with the detection capabilities of Elastic Security.
### When to use an ES|QL rule
ES|QL rules are the right fit when:
- You need **aggregation, transformation, or enrichment** within the query itself, such as computing statistics, renaming fields, or filtering on calculated values.
- The detection logic requires **pipe-based processing** that KQL and EQL cannot express, such as `STATS...BY` followed by `WHERE` to filter aggregated results.
- You want to create **new computed fields** (using `EVAL`) and alert on values derived from source data rather than raw field values.
ES|QL rules are **not** the best fit when:
- A field-value match is sufficient. Use a [custom query rule](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/custom-query) instead.
- You need to detect ordered event sequences. Use an [EQL rule](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/eql) instead.
- You want anomaly detection without explicit query logic. Use a [machine learning rule](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/machine-learning) instead.
### Data requirements
ES|QL rules query Elasticsearch indices directly using the `FROM` command. The indices must be accessible to the user who creates or last edits the rule.
### Alert deduplication and `_id` metadata
For **non-aggregating** queries (queries that do not use `STATS...BY`), the detection engine relies on the document `_id` metadata field to avoid creating duplicate alerts for the same source event across rule executions.
You don't need to add `METADATA _id` in the rule query for deduplication. The query in the editor is saved exactly as you enter it. You can paste from Discover or from AI-assisted tools without adding metadata clauses.If `_id` will be missing from the query results (for example, because of `DROP _id`, `RENAME _id AS …`, or `EVAL _id = …`), the query editor shows a non-blocking warning that you might get duplicate alerts. You can still save the rule after confirming the **Save with errors** dialog, but might get duplicate alerts until you adjust the query.
You must add `METADATA _id` to the `FROM` command yourself for non-aggregating queries if you want deduplication across executions. Without it, the same source event can generate duplicate alerts.
You can still include `METADATA` explicitly—for example `METADATA _id, _index, _version`—when you need additional [metadata fields](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-metadata-fields) in the query or for clarity.
## Annotated examples
The following examples use the [detections API](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/using-the-api) request format to show how ES|QL rules are defined. Each example is followed by a breakdown of the ES|QL-specific fields. For common fields like `name`, `severity`, and `interval`, refer to the [detections API documentation](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-detection-engine-rules-api).
### Aggregating query
This rule counts failed login attempts per user and alerts when any user exceeds 20 failures within the query window.
```json
{
"type": "esql",
"language": "esql",
"name": "High failed login count per user",
"description": "Detects users with more than 20 failed login attempts in the query window.",
"query": "FROM logs-* | WHERE event.category == \"authentication\" AND event.outcome == \"failure\" | STATS failed_count = COUNT(*) BY user.name | WHERE failed_count > 20",
"severity": "high",
"risk_score": 73,
"interval": "5m",
"from": "now-6m"
}
```
| Field | Value | Purpose |
|---------------------|-------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `type` / `language` | `"esql"` / `"esql"` | Both must be `"esql"`. |
| `query` | `FROM logs-* \| WHERE ... \| STATS ... BY ... \| WHERE ...` | An aggregating query pipeline. `FROM` specifies the source indices. `STATS...BY` groups failed logins by `user.name` and counts them. The final `WHERE` filters to users exceeding 20 failures. Each result row becomes an alert containing only the `BY` fields and computed values. |
ES|QL rules don't use a separate `index` field. Source indices are specified in the `FROM` command within the query.
### Non-aggregating query with deduplication
This rule detects process-start events with suspicious encoded arguments. The query omits `METADATA _id` in the `FROM` clause; in Elastic Stack 9.4 and later, the detection engine injects `METADATA _id` at execution time for alert deduplication. On Elastic Stack 9.3 and earlier, add `METADATA _id` (and optionally `_index`, `_version`) to the `FROM` command yourself.
```json
{
"type": "esql",
"language": "esql",
"name": "Process execution with encoded arguments",
"description": "Detects process start events where the command line contains encoded content.",
"query": "FROM logs-endpoint.events.* | WHERE event.category == \"process\" AND event.type == \"start\" AND process.command_line LIKE \"*-encoded*\" | LIMIT 100",
"severity": "medium",
"risk_score": 47,
"interval": "5m",
"from": "now-6m"
}
```
| Field | Value | Purpose |
|---------|--------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `query` | `FROM ... \| WHERE ... \| LIMIT 100` | A non-aggregating query. Each matching row becomes an alert. Elastic Stack: Planned For deduplication across executions, `METADATA _id` is automatically added if missing, but you must ensure that `_id` appears in the execution results. Commands that restrict or remove fields (such as `DROP _id` or `KEEP agent.*` which retains only `agent.*` fields) will exclude `_id` from results and prevent deduplication. Elastic Stack: Generally available from 9.0 to 9.3 In earlier versions, include `METADATA _id` (and optionally other metadata fields) after `FROM`. |
| `LIMIT` | `100` | Limits the number of results per execution. Interacts with the **Max alerts per run** setting, and the rule uses the lower of the two values. |
## ES|QL rule field reference
The following settings appear in the **Define rule** section when creating an ES|QL rule. For settings shared across all rule types, refer to [Rule settings reference](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/common-rule-settings).
The [ES|QL query](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql) that defines the detection logic. Can be aggregating (with `STATS...BY`) or non-aggregating. Each row in the query result becomes an alert. For non-aggregating queries, validation may show a non-blocking warning if `_id` is absent from the results (for example after `DROP _id`); you can still save the rule after confirming **Save with errors**.
Reduce repeated or duplicate alerts by grouping them on one or more fields. For details, refer to [Alert suppression](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/6103/solutions/security/detect-and-alert/alert-suppression).