Get tokens from text analysis

POST also GET /{index}/_analyze
POST also GET /_analyze

The analyze API performs analysis on a text string and returns the resulting tokens.

Generating excessive amount of tokens may cause a node to run out of memory. The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced. If more than this limit of tokens gets generated, an error occurs. The _analyze endpoint without a specified index will always use 10000 as its limit.

Read the reference documentation

Required authorization

Authentication

Api key

Authorization: <value>

Elasticsearch APIs support key-based authentication. You must create an API key and use the encoded value in the request header. For example:

		curl -X GET "${ES_URL}/_cat/indices?v=true" \
  -H "Authorization: ApiKey ${API_KEY}"

To get API keys, use the /_security/api_key APIs.

Basic

Authorization: Basic <credentials>

Bearer

Authorization: Bearer <token>

Elasticsearch APIs support the use of bearer tokens in the Authorization HTTP header to authenticate with the API. For examples, refer to Token-based authentication services

Path parameters

Query parameters

Request body

Response

application/json

Examples

Request Console 1 / 8 languages

You can apply any of the built-in analyzers to the text string without specifying an index.

Console Kibana Dev Tools

POST /_analyze
{
  "analyzer": "standard",
  "text": "this is a test"
}

JavaScript Elasticsearch JavaScript Client

const response = await client.indices.analyze({
  analyzer: "standard",
  text: "this is a test",
});

Python Elasticsearch Python Client

resp = client.indices.analyze(
    analyzer="standard",
    text="this is a test",
)

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "analyzer": "standard",
    "text": "this is a test"
  }'

Java Elasticsearch Java Client

client.indices().analyze(a -> a
    .analyzer("standard")
    .text("this is a test")
);

C# Elasticsearch .NET Client

var response = await client.Indices
    .AnalyzeAsync(d1 => d1
        .Analyzer("standard")
        .Text("this is a test")
    );

PHP Elasticsearch PHP Client

$resp = $client->indices()->analyze([
    "body" => [
        "analyzer" => "standard",
        "text" => "this is a test",
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.indices.analyze(
  body: {
    "analyzer": "standard",
    "text": "this is a test"
  }
)

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Request Console 1 / 2 languages

If the text parameter is provided as array of strings, it is analyzed as a multi-value field.

Console Kibana Dev Tools

POST /_analyze
{
  "analyzer": "standard",
  "text": [
    "this is a test",
    "the second text"
  ]
}

Generated from the API spec

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "analyzer": "standard",
    "text": [
      "this is a test",
      "the second text"
    ]
  }'

Generated from the API spec

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Request Console 1 / 2 languages

You can test a custom transient analyzer built from tokenizers, token filters, and char filters. Token filters use the filter parameter.

Console Kibana Dev Tools

POST /_analyze
{
  "tokenizer": "keyword",
  "filter": [
    "lowercase"
  ],
  "char_filter": [
    "html_strip"
  ],
  "text": "this is a <b>test</b>"
}

Generated from the API spec

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer": "keyword",
    "filter": [
      "lowercase"
    ],
    "char_filter": [
      "html_strip"
    ],
    "text": "this is a <b>test</b>"
  }'

Generated from the API spec

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Request Console 1 / 2 languages

Custom tokenizers, token filters, and character filters can be specified in the request body.

Console Kibana Dev Tools

POST /_analyze
{
  "tokenizer": "whitespace",
  "filter": [
    "lowercase",
    {
      "type": "stop",
      "stopwords": [
        "a",
        "is",
        "this"
      ]
    }
  ],
  "text": "this is a test"
}

Generated from the API spec

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer": "whitespace",
    "filter": [
      "lowercase",
      {
        "type": "stop",
        "stopwords": [
          "a",
          "is",
          "this"
        ]
      }
    ],
    "text": "this is a test"
  }'

Generated from the API spec

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Request Console 1 / 2 languages

Run POST /analyze_sample/_analyze to run an analysis on the text using the default index analyzer associated with the analyze_sample index. Alternatively, the analyzer can be derived based on a field mapping.

Console Kibana Dev Tools

POST /analyze_sample/_analyze
{
  "field": "obj1.field1",
  "text": "this is a test"
}

Generated from the API spec

curl Shell

curl -X POST "$ELASTICSEARCH_URL/analyze_sample/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "field": "obj1.field1",
    "text": "this is a test"
  }'

Generated from the API spec

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Request Console 1 / 2 languages

Run POST /analyze_sample/_analyze and supply a normalizer for a keyword field if there is a normalizer associated with the specified index.

Console Kibana Dev Tools

POST /analyze_sample/_analyze
{
  "normalizer": "my_normalizer",
  "text": "BaR"
}

Generated from the API spec

curl Shell

curl -X POST "$ELASTICSEARCH_URL/analyze_sample/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "normalizer": "my_normalizer",
    "text": "BaR"
  }'

Generated from the API spec

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Request Console 1 / 2 languages

If you want to get more advanced details, set explain to true. It will output all token attributes for each token. You can filter token attributes you want to output by setting the attributes option. NOTE: The format of the additional detail information is labelled as experimental in Lucene and it may change in the future.

Console Kibana Dev Tools

POST /_analyze
{
  "tokenizer": "standard",
  "filter": [
    "snowball"
  ],
  "text": "detailed output",
  "explain": true,
  "attributes": [
    "keyword"
  ]
}

Generated from the API spec

curl Shell

curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer": "standard",
    "filter": [
      "snowball"
    ],
    "text": "detailed output",
    "explain": true,
    "attributes": [
      "keyword"
    ]
  }'

Generated from the API spec

Response

A successful response for an analysis with explain set to true.

{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Required authorization

Authentication

Path parameters

Query parameters

Request body { }

Response

Request body