---
type: api
title: Get tokens from text analysis
description: **All methods and paths for this operation:**
url: https://www.elastic.co/elastic/docs-builder/docs/4641/api/doc/elasticsearch-serverless/operation/operation-indices-analyze
resource: https://www.elastic.co/elastic/docs-builder/docs/4641/api/doc/elasticsearch-serverless/operation/operation-indices-analyze
products:
  - Elasticsearch Serverless
---

# Get tokens from text analysis

Authentication: required

## Paths

- `POST` (also `GET`) `/{index}/_analyze`
- `POST` (also `GET`) `/_analyze`

#### Path parameters

- `index` (string) — optional, leave it out to use `/_analyze`
  Index used to derive the analyzer.
If specified, the `analyzer` or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

## Description

The analyze API performs analysis on a text string and returns the resulting tokens.

Generating excessive amount of tokens may cause a node to run out of memory.
The `index.analyze.max_token_count` setting enables you to limit the number of tokens that can be produced.
If more than this limit of tokens gets generated, an error occurs.
The `_analyze` endpoint without a specified index will always use `10000` as its limit.

[Learn more about text analysis](https://www.elastic.co/docs/manage-data/data-store/text-analysis)

## Required authorization

- Index privileges: `index`

## Query string parameters

- `index` (string) — optional
  Index used to derive the analyzer.
If specified, the `analyzer` or field parameter overrides this value.
If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

## Request body

`application/json`

- `analyzer` (string) — optional
  The name of the analyzer that should be applied to the provided `text`.
This could be a built-in analyzer, or an analyzer that’s been configured in the index.
- `attributes` (string[]) — optional
  Array of token attributes used to filter the output of the `explain` parameter.
- `char_filter` (object[]) — optional
  Array of character filters used to preprocess characters before the tokenizer.
- `explain` (boolean · default: false) — optional
  If `true`, the response includes token attributes and additional details.
- `field` (string) — optional
  Field used to derive the analyzer.
To use this parameter, you must specify an index.
If specified, the `analyzer` parameter overrides this value.
- `filter` (object[]) — optional
  Array of token filters used to apply after the tokenizer.
- `normalizer` (string) — optional
  Normalizer to use to convert text into a single token.
- `text` (union oneOf) — optional
  Text to analyze.
If an array of strings is provided, it is analyzed as a multi-value field.
  One of: `string[]` or `string`
- `tokenizer` (object) — optional
  Tokenizer to use to convert text into tokens.

## Response

`application/json`

- `detail` (object)
  - `analyzer` (object)
    - `name` (string)
    - `tokens` (object[])
      - `bytes` (string)
      - `end_offset` (number)
      - `keyword` (boolean)
      - `position` (number)
      - `positionLength` (number)
      - `start_offset` (number)
      - `termFrequency` (number)
      - `token` (string)
      - `type` (string)
  - `charfilters` (object[])
    - `filtered_text` (string[])
    - `name` (string)
  - `custom_analyzer` (boolean)
  - `tokenfilters` (object[])
    - `name` (string)
    - `tokens` (object[])
      - `bytes` (string)
      - `end_offset` (number)
      - `keyword` (boolean)
      - `position` (number)
      - `positionLength` (number)
      - `start_offset` (number)
      - `termFrequency` (number)
      - `token` (string)
      - `type` (string)
  - `tokenizer` (object)
    Same fields as `tokenfilters` in `detail`
- `tokens` (object[])
  - `end_offset` (number)
  - `position` (number)
  - `positionLength` (number)
  - `start_offset` (number)
  - `token` (string)
  - `type` (string)
## Code examples

### No index specified

#### Console

```console
POST /_analyze
{
  "analyzer": "standard",
  "text": "this is a test"
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  analyzer: "standard",
  text: "this is a test",
});
```

#### Python

```python
resp = client.indices.analyze(
    analyzer="standard",
    text="this is a test",
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "analyzer": "standard",
    "text": "this is a test"
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .analyzer("standard")
    .text("this is a test")
);
```

#### C#

```c#
var response = await client.Indices
    .AnalyzeAsync(d1 => d1
        .Analyzer("standard")
        .Text("this is a test")
    );
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "body" => [
        "analyzer" => "standard",
        "text" => "this is a test",
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  body: {
    "analyzer": "standard",
    "text": "this is a test"
  }
)
```

### An array of text strings

#### Console

```console
POST /_analyze
{
  "analyzer": "standard",
  "text": [
    "this is a test",
    "the second text"
  ]
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  analyzer: "standard",
  text: ["this is a test", "the second text"],
});
```

#### Python

```python
resp = client.indices.analyze(
    analyzer="standard",
    text=[
        "this is a test",
        "the second text"
    ],
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "analyzer": "standard",
    "text": [
      "this is a test",
      "the second text"
    ]
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .analyzer("standard")
    .text(List.of("this is a test","the second text"))
);
```

#### C#

```c#
var response = await client.Indices
    .AnalyzeAsync(d1 => d1
        .Analyzer("standard")
        .Text("this is a test", "the second text")
    );
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "body" => [
        "analyzer" => "standard",
        "text" => array(
            "this is a test",
            "the second text",
        ),
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  body: {
    "analyzer": "standard",
    "text": [
      "this is a test",
      "the second text"
    ]
  }
)
```

### Custom analyzer example 1

#### Console

```console
POST /_analyze
{
  "tokenizer": "keyword",
  "filter": [
    "lowercase"
  ],
  "char_filter": [
    "html_strip"
  ],
  "text": "this is a <b>test</b>"
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  tokenizer: "keyword",
  filter: ["lowercase"],
  char_filter: ["html_strip"],
  text: "this is a test</b>",
});
```

#### Python

```python
resp = client.indices.analyze(
    tokenizer="keyword",
    filter=[
        "lowercase"
    ],
    char_filter=[
        "html_strip"
    ],
    text="this is a test</b>",
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer": "keyword",
    "filter": [
      "lowercase"
    ],
    "char_filter": [
      "html_strip"
    ],
    "text": "this is a test\u003C/b\u003E"
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .charFilter(c -> c
        .name("html_strip")
    )
    .filter(f -> f
        .name("lowercase")
    )
    .text("this is a test</b>")
    .tokenizer(t -> t
        .name("keyword")
    )
);
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "body" => [
        "tokenizer" => "keyword",
        "filter" => array(
            "lowercase",
        ),
        "char_filter" => array(
            "html_strip",
        ),
        "text" => "this is a test</b>",
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  body: {
    "tokenizer": "keyword",
    "filter": [
      "lowercase"
    ],
    "char_filter": [
      "html_strip"
    ],
    "text": "this is a test</b>"
  }
)
```

### Custom analyzer example 2

#### Console

```console
POST /_analyze
{
  "tokenizer": "whitespace",
  "filter": [
    "lowercase",
    {
      "type": "stop",
      "stopwords": [
        "a",
        "is",
        "this"
      ]
    }
  ],
  "text": "this is a test"
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  tokenizer: "whitespace",
  filter: [
    "lowercase",
    {
      type: "stop",
      stopwords: ["a", "is", "this"],
    },
  ],
  text: "this is a test",
});
```

#### Python

```python
resp = client.indices.analyze(
    tokenizer="whitespace",
    filter=[
        "lowercase",
        {
            "type": "stop",
            "stopwords": [
                "a",
                "is",
                "this"
            ]
        }
    ],
    text="this is a test",
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer": "whitespace",
    "filter": [
      "lowercase",
      {
        "type": "stop",
        "stopwords": [
          "a",
          "is",
          "this"
        ]
      }
    ],
    "text": "this is a test"
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .filter(List.of(TokenFilter.of(t -> t
            .name("lowercase")
        ),TokenFilter.of(to -> to
            .definition(d -> d
                .stop(s -> s
                    .stopwords(List.of("a","is","this"))
                )
            )
        )))
    .text("this is a test")
    .tokenizer(tok -> tok
        .name("whitespace")
    )
);
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "body" => [
        "tokenizer" => "whitespace",
        "filter" => array(
            "lowercase",
            [
                "type" => "stop",
                "stopwords" => array(
                    "a",
                    "is",
                    "this",
                ),
            ],
        ),
        "text" => "this is a test",
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  body: {
    "tokenizer": "whitespace",
    "filter": [
      "lowercase",
      {
        "type": "stop",
        "stopwords": [
          "a",
          "is",
          "this"
        ]
      }
    ],
    "text": "this is a test"
  }
)
```

### Derive analyzer from field mapping

#### Console

```console
POST /analyze_sample/_analyze
{
  "field": "obj1.field1",
  "text": "this is a test"
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  index: "analyze_sample",
  field: "obj1.field1",
  text: "this is a test",
});
```

#### Python

```python
resp = client.indices.analyze(
    index="analyze_sample",
    field="obj1.field1",
    text="this is a test",
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/analyze_sample/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "field": "obj1.field1",
    "text": "this is a test"
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .field("obj1.field1")
    .index("analyze_sample")
    .text("this is a test")
);
```

#### C#

```c#
var response = await client.Indices
    .AnalyzeAsync(d1 => d1
        .Index("analyze_sample")
        .Field(x => x.Obj1.Field1)
        .Text("this is a test")
    );
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "index" => "analyze_sample",
    "body" => [
        "field" => "obj1.field1",
        "text" => "this is a test",
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  index: "analyze_sample",
  body: {
    "field": "obj1.field1",
    "text": "this is a test"
  }
)
```

### Normalizer

#### Console

```console
POST /analyze_sample/_analyze
{
  "normalizer": "my_normalizer",
  "text": "BaR"
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  index: "analyze_sample",
  normalizer: "my_normalizer",
  text: "BaR",
});
```

#### Python

```python
resp = client.indices.analyze(
    index="analyze_sample",
    normalizer="my_normalizer",
    text="BaR",
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/analyze_sample/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "normalizer": "my_normalizer",
    "text": "BaR"
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .index("analyze_sample")
    .normalizer("my_normalizer")
    .text("BaR")
);
```

#### C#

```c#
var response = await client.Indices
    .AnalyzeAsync(d1 => d1
        .Index("analyze_sample")
        .Normalizer("my_normalizer")
        .Text("BaR")
    );
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "index" => "analyze_sample",
    "body" => [
        "normalizer" => "my_normalizer",
        "text" => "BaR",
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  index: "analyze_sample",
  body: {
    "normalizer": "my_normalizer",
    "text": "BaR"
  }
)
```

### Explain analysis

#### Console

```console
POST /_analyze
{
  "tokenizer": "standard",
  "filter": [
    "snowball"
  ],
  "text": "detailed output",
  "explain": true,
  "attributes": [
    "keyword"
  ]
}
```

#### JavaScript

```javascript
const response = await client.indices.analyze({
  tokenizer: "standard",
  filter: ["snowball"],
  text: "detailed output",
  explain: true,
  attributes: ["keyword"],
});
```

#### Python

```python
resp = client.indices.analyze(
    tokenizer="standard",
    filter=[
        "snowball"
    ],
    text="detailed output",
    explain=True,
    attributes=[
        "keyword"
    ],
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_analyze" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tokenizer": "standard",
    "filter": [
      "snowball"
    ],
    "text": "detailed output",
    "explain": true,
    "attributes": [
      "keyword"
    ]
  }'
```

#### Java

```java
client.indices().analyze(a -> a
    .attributes("keyword")
    .explain(true)
    .filter(f -> f
        .name("snowball")
    )
    .text("detailed output")
    .tokenizer(t -> t
        .name("standard")
    )
);
```

#### PHP

```php
$resp = $client->indices()->analyze([
    "body" => [
        "tokenizer" => "standard",
        "filter" => array(
            "snowball",
        ),
        "text" => "detailed output",
        "explain" => true,
        "attributes" => array(
            "keyword",
        ),
    ],
]);
```

#### Ruby

```ruby
response = client.indices.analyze(
  body: {
    "tokenizer": "standard",
    "filter": [
      "snowball"
    ],
    "text": "detailed output",
    "explain": true,
    "attributes": [
      "keyword"
    ]
  }
)
```

## Response examples

#### Indices analyze response example7

`application/json`

A successful response for an analysis with `explain` set to `true`.

```json
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}
```