Loading

Machine learning rules

Machine learning rules generate alerts when a machine learning anomaly detection job discovers an anomaly that exceeds a defined score threshold. Unlike other rule types, machine learning rules do not require you to write a query. Instead, they rely on machine learning jobs that continuously model normal behavior and flag deviations.

machine learning rules are the right fit when:

  • You want to detect behavioral anomalies that are difficult to express as static queries, such as unusual login times, atypical data transfer volumes, or rare process executions for a given user or host.
  • A machine learning job is already active (or you plan to enable one) that models the relevant behavior.
  • You want adaptive detection that automatically adjusts to changing baselines without manual threshold tuning.

machine learning rules are not the best fit when:

Prerequisites

To create or edit machine learning rules, you need:

For an overview of using machine learning with Elastic Security, refer to Anomaly detection.

Select one or more machine learning jobs that model the behavior you want to detect. Elastic Security ships with prebuilt jobs covering common use cases:

  • Unusual login activity. Detects logins at unusual times or from unusual locations.
  • Rare process execution. Surfaces processes that rarely execute on a given host.
  • DNS tunneling. Identifies unusually high DNS query volumes or rare query patterns.

If a selected job is not currently active, it starts automatically when you enable the rule.

The anomaly score ranges from 0 to 100, where higher scores indicate stronger deviations from normal behavior. Guidelines for setting the threshold:

Threshold range Effect
25-50 Casts a wide net. Generates more alerts, including moderate anomalies. Suitable for initial exploration of a new job.
50-75 Balanced. Surfaces significant anomalies while filtering out low-confidence results. A good starting point for most rules.
75-100 High confidence only. Generates fewer alerts but each one represents a strong deviation. Best for mature jobs with well-understood baselines.

machine learning rules can be noisy when a job is newly deployed or the underlying data shifts. Use these techniques to reduce false positives:

  • Raise the anomaly score threshold if the job is generating too many low-confidence alerts.
  • Add rule exceptions to suppress alerts from known-benign anomalies, such as scheduled maintenance windows or expected batch processes.
  • Allow the job time to learn. Most jobs need at least two weeks of data before their baselines stabilize. Anomalies flagged during the initial learning period are less reliable.

machine learning alerts differ from other alert types:

  • Alerts are generated from anomaly results, not raw source events. The alert contains anomaly metadata (score, job ID, influencers, bucket time) rather than the original event fields.
  • When configuring alert suppression, only anomaly fields are available because source event fields are not present in the anomaly results.
  • Severity and risk score overrides based on source event fields are not applicable. Consider mapping the anomaly score to severity using severity override on anomaly-specific fields.
Tip

See it in practice. These prebuilt rules use machine learning detection:

  • Anomalous Process For a Linux Population. Uses a rare-process machine learning job to surface processes that are unusual across a fleet of Linux hosts.
  • Unusual Login Activity. Triggers when a user logs in at unusual times or from unusual source IPs, based on a login-activity machine learning job.
  • Spike in Network Traffic To a Country. Detects sudden increases in outbound network traffic to a specific country, useful for identifying data exfiltration.

The following settings are specific to machine learning rules. For settings shared across all rule types, refer to Rule settings reference.

Machine learning jobs
The anomaly detection jobs whose results the rule evaluates. Select one or more jobs. If a selected job is not active, it starts automatically when the rule is enabled.
Anomaly score threshold
The minimum anomaly score (0-100) above which the rule generates alerts. Only anomalies that meet or exceed this score trigger alerts.
Suppress alerts by (optional)
Reduce repeated or duplicate alerts by grouping them on one or more fields. Only anomaly fields are available for suppression because machine learning alerts do not contain source event fields. For details, refer to Alert suppression.
Related integrations (optional)
Associate the rule with one or more Elastic integrations to indicate data dependencies and allow users to verify each integration's installation status.