Evaluate data frame analytics

  • POST /_ml/data_frame/_evaluate
Auth

The API packages together commonly used evaluation metrics for various types of machine learning features. This has been designed for use on indexes created by data frame analytics. Evaluation requires both a ground truth field and an analytics result field to be present.

Required authorization

Cluster privileges monitor_ml

Request body

evaluation object required

Defines the type of evaluation you want to perform.

classification object

Classification evaluation evaluates the results of a classification analysis which outputs a prediction that identifies to which of the classes each document belongs.

actual_field string required

The field of the index which contains the ground truth. The data type of this field can be boolean or integer. If the data type is integer, the value has to be either 0 (false) or 1 (true).

predicted_field string

The field in the index which contains the predicted value, in other words the results of the classification analysis.

top_classes_field string

The field of the index which is an array of documents of the form { "class_name": XXX, "class_probability": YYY }. This field must be defined as nested in the mappings.

metrics object

Specifies the metrics that are used for the evaluation.

auc_roc object

The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve. It is calculated for a specific class (provided as "class_name") treated as positive.

class_name string

Name of the only class that is treated as positive during AUC ROC calculation. Other classes are treated as negative ("one-vs-all" strategy). All the evaluated documents must have class_name in the list of their top classes.

include_curve boolean

Whether or not the curve should be returned in addition to the score. Default value is false.

precision map string to object

Precision of predictions (per-class and average).

recall map string to object

Recall of predictions (per-class and average).

accuracy map string to object

Accuracy of predictions (per-class and overall).

multiclass_confusion_matrix map string to object

Multiclass confusion matrix.

outlier_detection object

Outlier detection evaluates the results of an outlier detection analysis which outputs the probability that each document is an outlier.

actual_field string required

The field of the index which contains the ground truth. The data type of this field can be boolean or integer. If the data type is integer, the value has to be either 0 (false) or 1 (true).

predicted_probability_field string required

The field of the index that defines the probability of whether the item belongs to the class in question or not. It’s the field that contains the results of the analysis.

metrics object

Specifies the metrics that are used for the evaluation.

auc_roc object

The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve. It is calculated for a specific class (provided as "class_name") treated as positive.

class_name string

Name of the only class that is treated as positive during AUC ROC calculation. Other classes are treated as negative ("one-vs-all" strategy). All the evaluated documents must have class_name in the list of their top classes.

include_curve boolean

Whether or not the curve should be returned in addition to the score. Default value is false.

precision map string to object

Precision of predictions (per-class and average).

recall map string to object

Recall of predictions (per-class and average).

confusion_matrix map string to object

Accuracy of predictions (per-class and overall).

regression object

Regression evaluation evaluates the results of a regression analysis which outputs a prediction of values.

actual_field string required

The field of the index which contains the ground truth. The data type of this field must be numerical.

predicted_field string required

The field in the index that contains the predicted value, in other words the results of the regression analysis.

metrics object

Specifies the metrics that are used for the evaluation. For more information on mse, msle, and huber, consult the Jupyter notebook on regression loss functions.

mse map string to object

Average squared difference between the predicted values and the actual (ground truth) value. For more information, read this wiki article.

msle object

Average squared difference between the logarithm of the predicted values and the logarithm of the actual (ground truth) value.

offset number

Defines the transition point at which you switch from minimizing quadratic error to minimizing quadratic log error. Defaults to 1.

huber object

Pseudo Huber loss function.

delta number

Approximates 1/2 (prediction - actual)2 for values much less than delta and approximates a straight line with slope delta for values much larger than delta. Defaults to 1. Delta needs to be greater than 0.

r_squared map string to object

Proportion of the variance in the dependent variable that is predictable from the independent variables.

index string required

Defines the index in which the evaluation will be performed.

query {} Query

A query clause that retrieves a subset of data from the source index.

Response

application/json
classification object

Evaluation results for a classification analysis. It outputs a prediction that identifies to which of the classes each document belongs.

auc_roc object

The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve. It is calculated for a specific class (provided as "class_name") treated as positive.

value number required
curve object[]
tpr number required
fpr number required
threshold number required
accuracy object

Accuracy of predictions (per-class and overall).

classes object[] required
value number required
class_name string required
overall_accuracy number required
multiclass_confusion_matrix object

Multiclass confusion matrix.

confusion_matrix object[] required
actual_class string required
actual_class_doc_count number required
predicted_classes object[] required
predicted_class string required
count number required
other_predicted_class_doc_count number required
other_actual_class_count number required
precision object

Precision of predictions (per-class and average).

classes object[] required
value number required
class_name string required
avg_precision number required
recall object

Recall of predictions (per-class and average).

classes object[] required
value number required
class_name string required
avg_recall number required
outlier_detection object

Evaluation results for an outlier detection analysis. It outputs the probability that each document is an outlier.

auc_roc object · default: {"include_curve": false}

The AUC ROC (area under the curve of the receiver operating characteristic) score and optionally the curve.

value number required
curve object[]
tpr number required
fpr number required
threshold number required
precision map string to number

Set the different thresholds of the outlier score at where the metric is calculated.

recall map string to number

Set the different thresholds of the outlier score at where the metric is calculated.

confusion_matrix map string to object

Set the different thresholds of the outlier score at where the metrics (tp - true positive, fp - false positive, tn - true negative, fn - false negative) are calculated.

<string> object
tp number required

True Positive

fp number required

False Positive

tn number required

True Negative

fn number required

False Negative

regression object

Evaluation results for a regression analysis which outputs a prediction of values.

huber object

Pseudo Huber loss function.

value number required
mse object

Average squared difference between the predicted values and the actual (ground truth) value.

value number required
msle object

Average squared difference between the logarithm of the predicted values and the logarithm of the actual (ground truth) value.

value number required
r_squared object

Proportion of the variance in the dependent variable that is predictable from the independent variables.

value number required