---
type: api
title: Evaluate data frame analytics
description: The API packages together commonly used evaluation metrics for various types
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/operation/operation-ml-evaluate-data-frame
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/operation/operation-ml-evaluate-data-frame
products:
  - Elasticsearch
---

# Evaluate data frame analytics

Availability: 7.3+

## Paths

- `POST` `/_ml/data_frame/_evaluate`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#apikeyauth)
- [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#basicauth)
- [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#bearerauth)

## Description

The API packages together commonly used evaluation metrics for various types
of machine learning features. This has been designed for use on indexes
created by data frame analytics. Evaluation requires both a ground truth
field and an analytics result field to be present.

## Required authorization

* Cluster privileges: `monitor_ml`

## Request Body

`application/json`

- `evaluation` ({} DataframeEvaluationContainer) — required
  Defines the type of evaluation you want to perform.
  - `classification` ({} DataframeEvaluationClassification) — optional
    Classification evaluation evaluates the results of a classification analysis which outputs a prediction that identifies to which of the classes each document belongs.
    - `actual_field` (string Field) — required
      The field of the index which contains the ground truth. The data type of this field can be boolean or integer. If the data type is integer, the value has to be either 0 (false) or 1 (true).
    - `predicted_field` (string Field) — optional
      The field in the index which contains the predicted value, in other words the results of the classification analysis.
    - `top_classes_field` (string Field) — optional
      The field of the index which is an array of documents of the form { "class_name": XXX, "class_probability": YYY }. This field must be defined as nested in the mappings.
    - `metrics` ({} DataframeEvaluationClassificationMetrics) — optional
      Specifies the metrics that are used for the evaluation.
      - `auc_roc` ({} DataframeEvaluationClassificationMetricsAucRoc) — optional
        The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve. It is calculated for a specific class (provided as "class_name") treated as positive.
        - `class_name` (string Name) — optional
          Name of the only class that is treated as positive during AUC ROC calculation. Other classes are treated as negative ("one-vs-all" strategy). All the evaluated documents must have class_name in the list of their top classes.
        - `include_curve` (boolean) — optional
          Whether or not the curve should be returned in addition to the score. Default value is false.
      - `precision` (map string to object) — optional
        Precision of predictions (per-class and average).
      - `recall` (map string to object) — optional
        Recall of predictions (per-class and average).
      - `accuracy` (map string to object) — optional
        Accuracy of predictions (per-class and overall).
      - `multiclass_confusion_matrix` (map string to object) — optional
        Multiclass confusion matrix.
  - `outlier_detection` ({} DataframeEvaluationOutlierDetection) — optional
    Outlier detection evaluates the results of an outlier detection analysis which outputs the probability that each document is an outlier.
    - `actual_field` (string Field) — required
      The field of the index which contains the ground truth. The data type of this field can be boolean or integer. If the data type is integer, the value has to be either 0 (false) or 1 (true).
    - `predicted_probability_field` (string Field) — required
      The field of the index that defines the probability of whether the item belongs to the class in question or not. It’s the field that contains the results of the analysis.
    - `metrics` ({} DataframeEvaluationOutlierDetectionMetrics) — optional
      Specifies the metrics that are used for the evaluation.
      - `auc_roc` ({} DataframeEvaluationClassificationMetricsAucRoc) — optional
        The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve. It is calculated for a specific class (provided as "class_name") treated as positive.
        - `class_name` (string Name) — optional
          Name of the only class that is treated as positive during AUC ROC calculation. Other classes are treated as negative ("one-vs-all" strategy). All the evaluated documents must have class_name in the list of their top classes.
        - `include_curve` (boolean) — optional
          Whether or not the curve should be returned in addition to the score. Default value is false.
      - `precision` (map string to object) — optional
        Precision of predictions (per-class and average).
      - `recall` (map string to object) — optional
        Recall of predictions (per-class and average).
      - `confusion_matrix` (map string to object) — optional
        Accuracy of predictions (per-class and overall).
  - `regression` ({} DataframeEvaluationRegression) — optional
    Regression evaluation evaluates the results of a regression analysis which outputs a prediction of values.
    - `actual_field` (string Field) — required
      The field of the index which contains the ground truth. The data type of this field must be numerical.
    - `predicted_field` (string Field) — required
      The field in the index that contains the predicted value, in other words the results of the regression analysis.
    - `metrics` ({} DataframeEvaluationRegressionMetrics) — optional
      Specifies the metrics that are used for the evaluation. For more information on mse, msle, and huber, consult the Jupyter notebook on regression loss functions.
      - `mse` (map string to object) — optional
        Average squared difference between the predicted values and the actual (ground truth) value. For more information, read this wiki article.
      - `msle` ({} DataframeEvaluationRegressionMetricsMsle) — optional
        Average squared difference between the logarithm of the predicted values and the logarithm of the actual (ground truth) value.
        - `offset` (number) — optional
          Defines the transition point at which you switch from minimizing quadratic error to minimizing quadratic log error. Defaults to 1.
      - `huber` ({} DataframeEvaluationRegressionMetricsHuber) — optional
        Pseudo Huber loss function.
        - `delta` (number) — optional
          Approximates 1/2 (prediction - actual)2 for values much less than delta and approximates a straight line with slope delta for values much larger than delta. Defaults to 1. Delta needs to be greater than 0.
      - `r_squared` (map string to object) — optional
        Proportion of the variance in the dependent variable that is predictable from the independent variables.
- `index` (string IndexName) — required
  Defines the `index` in which the evaluation will be performed.
- `query` ({} QueryContainer) — optional
  A query clause that retrieves a subset of data from the source index.
  See [QueryContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/types/_types-query_dsl-querycontainer)

## Response

`application/json`

- `classification` ({} DataframeClassificationSummary)
  Evaluation results for a classification analysis.
It outputs a prediction that identifies to which of the classes each document belongs.
  - `auc_roc` ({} DataframeEvaluationSummaryAucRoc)
    The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve.
It is calculated for a specific class (provided as "class_name") treated as positive.
    - `value` (number)
    - `curve` ([] DataframeEvaluationSummaryAucRocCurveItem)
      - `tpr` (number)
      - `fpr` (number)
      - `threshold` (number)
  - `accuracy` ({} DataframeClassificationSummaryAccuracy)
    Accuracy of predictions (per-class and overall).
    - `classes` ([] DataframeEvaluationClass)
      - `value` (number)
      - `class_name` (string Name)
    - `overall_accuracy` (number)
  - `multiclass_confusion_matrix` ({} DataframeClassificationSummaryMulticlassConfusionMatrix)
    Multiclass confusion matrix.
    - `confusion_matrix` ([] ConfusionMatrixItem)
      - `actual_class` (string Name)
      - `actual_class_doc_count` (number)
      - `predicted_classes` ([] ConfusionMatrixPrediction)
        - `predicted_class` (string Name)
        - `count` (number)
      - `other_predicted_class_doc_count` (number)
    - `other_actual_class_count` (number)
  - `precision` ({} DataframeClassificationSummaryPrecision)
    Precision of predictions (per-class and average).
    - `classes` ([] DataframeEvaluationClass)
      - `value` (number)
      - `class_name` (string Name)
    - `avg_precision` (number)
  - `recall` ({} DataframeClassificationSummaryRecall)
    Recall of predictions (per-class and average).
    - `classes` ([] DataframeEvaluationClass)
      - `value` (number)
      - `class_name` (string Name)
    - `avg_recall` (number)
- `outlier_detection` ({} DataframeOutlierDetectionSummary)
  Evaluation results for an outlier detection analysis.
It outputs the probability that each document is an outlier.
  - `auc_roc` ({} DataframeEvaluationSummaryAucRoc · default: {"include_curve": false})
    The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve.
    - `value` (number)
    - `curve` ([] DataframeEvaluationSummaryAucRocCurveItem)
      - `tpr` (number)
      - `fpr` (number)
      - `threshold` (number)
  - `precision` (map string to number)
    Set the different thresholds of the outlier score at where the metric is calculated.
  - `recall` (map string to number)
    Set the different thresholds of the outlier score at where the metric is calculated.
  - `confusion_matrix` (map string to ConfusionMatrixThreshold)
    Set the different thresholds of the outlier score at where the metrics (`tp` - true positive, `fp` - false positive, `tn` - true negative, `fn` - false negative) are calculated.
    Map values ({} ConfusionMatrixThreshold)
    - `tp` (number)
      True Positive
    - `fp` (number)
      False Positive
    - `tn` (number)
      True Negative
    - `fn` (number)
      False Negative
- `regression` ({} DataframeRegressionSummary)
  Evaluation results for a regression analysis which outputs a prediction of values.
  - `huber` ({} DataframeEvaluationValue)
    Pseudo Huber loss function.
    - `value` (number)
  - `mse` ({} DataframeEvaluationValue)
    Average squared difference between the predicted values and the actual (`ground truth`) value.
    - `value` (number)
  - `msle` ({} DataframeEvaluationValue)
    Average squared difference between the logarithm of the predicted values and the logarithm of the actual (`ground truth`) value.
    - `value` (number)
  - `r_squared` ({} DataframeEvaluationValue)
    Proportion of the variance in the dependent variable that is predictable from the independent variables.
    - `value` (number)
## Code Examples

#### Console

```console
POST _ml/data_frame/_evaluate
{
  "index": "animal_classification",
  "evaluation": {
    "classification": {
      "actual_field": "animal_class",
      "predicted_field": "ml.animal_class_prediction",
      "metrics": {
        "multiclass_confusion_matrix": {}
      }
    }
  }
}
```

#### JavaScript

```javascript
const response = await client.ml.evaluateDataFrame({
  index: "animal_classification",
  evaluation: {
    classification: {
      actual_field: "animal_class",
      predicted_field: "ml.animal_class_prediction",
      metrics: {
        multiclass_confusion_matrix: {},
      },
    },
  },
});
```

#### Python

```python
resp = client.ml.evaluate_data_frame(
    index="animal_classification",
    evaluation={
        "classification": {
            "actual_field": "animal_class",
            "predicted_field": "ml.animal_class_prediction",
            "metrics": {
                "multiclass_confusion_matrix": {}
            }
        }
    },
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_ml/data_frame/_evaluate" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "index": "animal_classification",
    "evaluation": {
      "classification": {
        "actual_field": "animal_class",
        "predicted_field": "ml.animal_class_prediction",
        "metrics": {
          "multiclass_confusion_matrix": {}
        }
      }
    }
  }'
```

#### Java

```java
client.ml().evaluateDataFrame(e -> e
    .evaluation(ev -> ev
        .classification(c -> c
            .actualField("animal_class")
            .predictedField("ml.animal_class_prediction")
            .metrics(m -> m
                .multiclassConfusionMatrix(Map.of())
            )
        )
    )
    .index("animal_classification")
);
```

#### C#

```c#
var response = await client.MachineLearning
    .EvaluateDataFrameAsync(d1 => d1
        .Evaluation(d2 => d2
            .Classification(d3 => d3
                .ActualField(x => x.AnimalClass)
                .Metrics(d4 => d4
                    .MulticlassConfusionMatrix(new Dictionary<string, object>() { })
                )
                .PredictedField(x => x.Ml.AnimalClassPrediction)
            )
        )
        .Index("animal_classification")
    );
```

#### PHP

```php
$resp = $client->ml()->evaluateDataFrame([
    "body" => [
        "index" => "animal_classification",
        "evaluation" => [
            "classification" => [
                "actual_field" => "animal_class",
                "predicted_field" => "ml.animal_class_prediction",
                "metrics" => [
                    "multiclass_confusion_matrix" => new ArrayObject([]),
                ],
            ],
        ],
    ],
]);
```

#### Ruby

```ruby
response = client.ml.evaluate_data_frame(
  body: {
    "index": "animal_classification",
    "evaluation": {
      "classification": {
        "actual_field": "animal_class",
        "predicted_field": "ml.animal_class_prediction",
        "metrics": {
          "multiclass_confusion_matrix": {}
        }
      }
    }
  }
)
```

## Response Examples

#### Classification example 1

A succesful response from `POST _ml/data_frame/_evaluate` to evaluate a classification analysis job for an annotated index. The `actual_class` contains the name of the class the analysis tried to predict. The `actual_class_doc_count` is the number of documents in the index belonging to the `actual_class`. The `predicted_classes` object contains the list of the predicted classes and the number of predictions associated with the class.

```json
{
  "classification": {
    "multiclass_confusion_matrix": {
      "confusion_matrix": [
        {
          "actual_class": "cat",
          "actual_class_doc_count": 12,
          "predicted_classes": [
            {
              "predicted_class": "cat",
              "count": 12
            },
            {
              "predicted_class": "dog",
              "count": 0
            }
          ],
          "other_predicted_class_doc_count": 0
        },
        {
          "actual_class": "dog",
          "actual_class_doc_count": 11,
          "predicted_classes": [
            {
              "predicted_class": "dog",
              "count": 7
            },
            {
              "predicted_class": "cat",
              "count": 4
            }
          ],
          "other_predicted_class_doc_count": 0
        }
      ],
      "other_actual_class_count": 0
    }
  }
}
```

#### Classification example 2

A succesful response from `POST _ml/data_frame/_evaluate` to evaluate a classification analysis job with the AUC ROC metrics for an annotated index.

```json
{
  "classification": {
    "auc_roc": {
      "value": 0.8941788639536681
    }
  }
}
```

#### Outlier detection

A successful response from `POST _ml/data_frame/_evaluate` to evaluate an outlier detection job.

```json
{
  "outlier_detection": {
    "auc_roc": {
      "value": 0.9258475774641445
    },
    "confusion_matrix": {
      "0.25": {
        "tp": 5,
        "fp": 9,
        "tn": 204,
        "fn": 5
      },
      "0.5": {
        "tp": 1,
        "fp": 5,
        "tn": 208,
        "fn": 9
      },
      "0.75": {
        "tp": 0,
        "fp": 4,
        "tn": 209,
        "fn": 10
      }
    },
    "precision": {
      "0.25": 0.35714285714285715,
      "0.5": 0.16666666666666666,
      "0.75": 0
    },
    "recall": {
      "0.25": 0.5,
      "0.5": 0.1,
      "0.75": 0
    }
  }
}
```