Evaluate ranked search results
- POST also GET /{index}/_rank_eval
- POST also GET /_rank_eval
Evaluate the quality of ranked search results over a set of typical search queries.
- Index privileges:
read
Path Parameters
-
indexIndexName | IndexNameoptional -
A comma-separated list of data streams, indices, and index aliases used to limit the request. Wildcard (
*) expressions are supported. To target all data streams and indices in a cluster, omit this parameter or use_allor*.Leave it out to use
/_rank_eval.
Query Parameters
-
allow_no_indicesboolean · default: true If
false, the request returns an error if any wildcard expression, index alias, or_allvalue targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targetingfoo*,bar*returns an error if an index starts withfoobut no index starts withbar.-
expand_wildcardsstring ExpandWildcards · default: open Whether to expand wildcard expression to concrete indices that are open, closed or both.
Supported values include:
all: Match any data stream or index, including hidden ones.open: Match open, non-hidden indices. Also matches any non-hidden data stream.closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.hidden: Match hidden data streams and hidden indices. Must be combined withopen,closed, orboth.none: Wildcard expressions are not accepted.
-
Values:
allopenclosedhiddennone If
true, missing or closed indices are not included in the response.-
search_typestring Search operation type
Request
-
requestsRankEvalRequestItemrequired -
A set of typical search requests, together with their provided ratings.
-
idstring Idrequired -
The search request’s ID, used to group result details later.
-
ratingsDocumentRatingrequired -
List of document ratings
-
template_idstring Id -
The search template Id
-
paramsmap string to object -
The search template parameters.
-
metricRankEvalMetric -
Definition of the evaluation metric to calculate.
-
precisionRankEvalMetricPrecision -
knumber · default: 10 -
Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
-
relevant_rating_thresholdnumber · default: 1 -
Sets the rating threshold above which documents are considered to be "relevant".
-
ignore_unlabeledboolean · default: false -
Controls how unlabeled documents in the search results are counted. If set to true, unlabeled documents are ignored and neither count as relevant or irrelevant. Set to false (the default), they are treated as irrelevant.
-
recallRankEvalMetricRecall -
knumber · default: 10 -
Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
-
relevant_rating_thresholdnumber · default: 1 -
Sets the rating threshold above which documents are considered to be "relevant".
-
mean_reciprocal_rankRankEvalMetricMeanReciprocalRank -
knumber · default: 10 -
Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
-
relevant_rating_thresholdnumber · default: 1 -
Sets the rating threshold above which documents are considered to be "relevant".
-
dcgRankEvalMetricDiscountedCumulativeGain -
expected_reciprocal_rankRankEvalMetricExpectedReciprocalRank -
knumber · default: 10 -
Sets the maximum number of documents retrieved per query. This value will act in place of the usual size parameter in the query.
-
maximum_relevancenumberrequired -
The highest relevance grade used in the user-supplied relevance judgments.
Response
-
metric_scorenumberrequired -
The overall evaluation quality calculated by the defined metric
-
detailsmap string to RankEvalMetricDetailrequired -
The details section contains one entry for every query in the original requests section, keyed by the search request id
-
<string>RankEvalMetricDetail -
metric_scorenumberrequired -
The metric_score in the details section shows the contribution of this query to the global quality metric score
-
unrated_docsUnratedDocumentrequired -
The unrated_docs section contains an _index and _id entry for each document in the search result for this query that didn’t have a ratings value. This can be used to ask the user to supply ratings for these documents
-
hitsRankEvalHitItemrequired -
The hits section shows a grouping of the search results with their supplied ratings
-
metric_detailsmap string to string to objectrequired -
The metric_details give additional information about the calculated quality metric (e.g. how many of the retrieved documents were relevant). The content varies for each metric but allows for better interpretation of the results
-
failuresmap string to objectrequired