How the experimental alerting system works

This page walks through what happens at each step after a rule runs on its schedule. Both paths begin the same way: Kibana writes a rule event for each matching row. The rule's configuration determines whether those events belong to an alert episode. Use this page to understand how the different components of the experimental alerting system interact.

When Kibana tracks matches as an alert episode, it writes each match as a rule event (type: alert) with episode.* fields. Events that share episode.id belong to the same alert episode. Each new event can advance the alert episode's lifecycle state. An action policy sits between the alert episode and a workflow, deciding whether and when to invoke it.

Step Actor Action
1 Rule Runs on schedule and evaluates ES|QL against your data
2 Kibana Query returns results → Writes one rule event per matching row to .rule-events (type: alert)
3 Kibana Opens an alert episode in pending and advances it to active once the activation threshold is met
4 Action policy Evaluates the alert episode against its conditions (eligibility, match conditions, and frequency)
5 Action policy If conditions are met, invokes a workflow
6 Workflow Sends notification or runs automation
7 Kibana Condition clears → Writes a new rule event → The alert episode moves to recovering → inactive
8 Action policy Evaluates the recovery event and invokes a workflow if conditions are met
9 Workflow Sends the recovery notification
Note

Steps 4–6 and 8–9 run on a separate background process that polls roughly every 5 seconds. Action policy evaluation is not triggered synchronously by the rule's own execution. There is always at least one dispatcher polling cycle between a rule run and any resulting notification.

An SRE team wants to know when checkout service latency degrades, and notify the on-call team when it does. The team creates a rule that opens an alert episode:

  1. The rule runs an ES|QL query every five minutes, checking p95 checkout service latency.
  2. The first check where p95 exceeds 2 seconds opens an alert episode in pending. A second consecutive breach moves it to active.
  3. An action policy with a rule.tags: "checkout" matcher invokes an on-call workflow that sends a Slack message.

The engineer investigates, fixes a slow query, and the alert episode recovers automatically.

Kibana writes a rule event (type: signal) to .rule-events for each match. These events stay in .rule-events. They don't appear on Alerts and aren't evaluated by action policies or lifecycle triggers. They accumulate over time and are immediately queryable in Discover for incident investigation, or as inputs to a follow-on rule that opens an alert episode. For query examples, dashboards, and correlation patterns, refer to Query rule events.

Step Actor Action
1 Rule Runs on schedule and evaluates ES|QL against your data
2 Kibana Query returns results → Writes one rule event per matching row to .rule-events (type: signal)
3 Kibana Keeps the event available for later analysis. It doesn't go to a policy or workflow

A security team wants to track calls to a rarely-used administrator API endpoint, but individual calls aren't suspicious enough to page anyone. To start collecting data without generating noise, the team creates a rule that records matches without opening an alert episode:

  1. The rule runs an ES|QL query on a schedule, checking for calls to the administrator API endpoint.
  2. Each time the query returns results, Kibana writes a rule event (type: signal) to .rule-events.
  3. The events accumulate silently and are immediately queryable in Discover.

After a few weeks, the accumulated events become useful in two ways. The team can write a follow-on rule that opens an alert episode and combines admin API calls with other events (such as a spike in error rates) to catch correlated activity that neither source would surface on its own. When an outage happens, the team can query that history as evidence directly in Discover, without reconstructing the original query or worrying that the source data has become stale.

  • Get started: Enable the experimental alerting system and create your first rule.
  • Rules: What rules detect, how action policies invoke workflows, and how to choose a creation path.
  • Notifications and actions: Set up action policies that invoke workflows when an alert episode matches.