Get anomaly records for an anomaly detection job

  • POST also GET /_ml/anomaly_detectors/{job_id}/results/records
Auth

Records contain the detailed analytical results. They describe the anomalous activity that has been identified in the input data based on the detector configuration. There can be many anomaly records depending on the characteristics and size of the input data. In practice, there are often too many to be able to manually process them. The machine learning features therefore perform a sophisticated aggregation of the anomaly records into buckets. The number of record results depends on the number of anomalies found in each bucket, which relates to the number of time series being modeled and the number of detectors.

Required authorization

Cluster privileges monitor_ml

Path parameters

job_id string required

Identifier for the anomaly detection job.

Query parameters

desc boolean · default: false

If true, the results are sorted in descending order.

end union oneOf · default: -1

Returns records with timestamps earlier than this time. The default value means results are not limited to specific timestamps.

One of: string or number
exclude_interim boolean · default: false

If true, the output excludes interim results.

from number · default: 0

Skips the specified number of records.

record_score number · default: 0

Returns records with anomaly scores greater or equal than this value.

size number · default: 100

Specifies the maximum number of records to obtain.

sort string · default: record_score

Specifies the sort field for the requested records.

start union oneOf · default: -1

Returns records with timestamps after this time. The default value means results are not limited to specific timestamps.

One of: string or number

Request body

desc boolean · default: false

Refer to the description for the desc query parameter.

end union oneOf · default: -1

Refer to the description for the end query parameter.

One of: string or number
exclude_interim boolean · default: false

Refer to the description for the exclude_interim query parameter.

page object
from number · default: 0

Skips the specified number of items.

size number · default: 10000

Specifies the maximum number of items to obtain.

record_score number · default: 0

Refer to the description for the record_score query parameter.

sort string · default: record_score

Refer to the description for the sort query parameter.

start union oneOf · default: -1

Refer to the description for the start query parameter.

One of: string or number

Response

application/json
count number required
records object[] required
actual number[]

The actual value for the bucket.

anomaly_score_explanation object

Information about the factors impacting the initial anomaly score.

anomaly_characteristics_impact number

Impact from the duration and magnitude of the detected anomaly relative to the historical average.

anomaly_length number

Length of the detected anomaly in the number of buckets.

anomaly_type string

Type of the detected anomaly: spike or dip.

high_variance_penalty boolean

Indicates reduction of anomaly score for the bucket with large confidence intervals. If a bucket has large confidence intervals, the score is reduced.

incomplete_bucket_penalty boolean

If the bucket contains fewer samples than expected, the score is reduced.

lower_confidence_bound number

Lower bound of the 95% confidence interval.

multi_bucket_impact number

Impact of the deviation between actual and typical values in the past 12 buckets.

single_bucket_impact number

Impact of the deviation between actual and typical values in the current bucket.

typical_value number

Typical (expected) value for this bucket.

upper_confidence_bound number

Upper bound of the 95% confidence interval.

bucket_span number required

The length of the bucket in seconds. This value matches the bucket_span that is specified in the job.

by_field_name string

The field used to split the data. In particular, this property is used for analyzing the splits with respect to their own history. It is used for finding unusual values in the context of the split.

by_field_value string

The value of by_field_name.

causes object[]

For population analysis, an over field must be specified in the detector. This property contains an array of anomaly records that are the causes for the anomaly that has been identified for the over field. This sub-resource contains the most anomalous records for the over_field_name. For scalability reasons, a maximum of the 10 most significant causes of the anomaly are returned. As part of the core analytical modeling, these low-level anomaly records are aggregated for their parent over field record. The causes resource contains similar elements to the record resource, namely actual, typical, geo_results.actual_point, geo_results.typical_point, *_field_name and *_field_value. Probability and scores are not applicable to causes.

actual number[]
by_field_name string
by_field_value string
correlated_by_field_value string
field_name string
function string
function_description string
geo_results object
actual_point string

The actual value for the bucket formatted as a geo_point.

typical_point string

The typical value for the bucket formatted as a geo_point.

influencers object[]
influencer_field_name string required
influencer_field_values string[] required
over_field_name string
over_field_value string
partition_field_name string
partition_field_value string
probability number required
typical number[]
detector_index number required

A unique identifier for the detector.

field_name string

Certain functions require a field to operate on, for example, sum(). For those functions, this value is the name of the field to be analyzed.

function string

The function in which the anomaly occurs, as specified in the detector configuration. For example, max.

function_description string

The description of the function in which the anomaly occurs, as specified in the detector configuration.

geo_results object

If the detector function is lat_long, this object contains comma delimited strings for the latitude and longitude of the actual and typical values.

actual_point string

The actual value for the bucket formatted as a geo_point.

typical_point string

The typical value for the bucket formatted as a geo_point.

influencers object[]

If influencers were specified in the detector configuration, this array contains influencers that contributed to or were to blame for an anomaly.

influencer_field_name string required
influencer_field_values string[] required
initial_record_score number required

A normalized score between 0-100, which is based on the probability of the anomalousness of this record. This is the initial value that was calculated at the time the bucket was processed.

is_interim boolean required

If true, this is an interim result. In other words, the results are calculated based on partial input data.

job_id string required

Identifier for the anomaly detection job.

over_field_name string

The field used to split the data. In particular, this property is used for analyzing the splits with respect to the history of all splits. It is used for finding unusual values in the population of all splits.

over_field_value string

The value of over_field_name.

partition_field_name string

The field used to segment the analysis. When you use this property, you have completely independent baselines for each value of this field.

partition_field_value string

The value of partition_field_name.

probability number required

The probability of the individual anomaly occurring, in the range 0 to 1. For example, 0.0000772031. This value can be held to a high precision of over 300 decimal places, so the record_score is provided as a human-readable and friendly interpretation of this.

record_score number required

A normalized score between 0-100, which is based on the probability of the anomalousness of this record. Unlike initial_record_score, this value will be updated by a re-normalization process as new data is analyzed.

result_type string required

Internal. This is always set to record.

timestamp number required

The start time of the bucket for which these results were calculated.

typical number[]

The typical value for the bucket, according to analytical modeling.