Create an Elasticsearch inference endpoint

Auth

Create an inference endpoint to perform an inference task with the elasticsearch service.

Note

Your Elasticsearch deployment contains preconfigured ELSER and E5 inference endpoints, you only need to create the enpoints using the API if you want to customize the settings.

If you use the ELSER or the E5 model through the elasticsearch service, the API request will automatically download and deploy the model if it isn't downloaded yet.

Note

You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.

After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Required authorization

Cluster privileges manage_inference

Path parameters

task_type enum required

The type of the inference task that the model will perform.

Values: rerank sparse_embedding text_embedding
elasticsearch_inference_id string required

The unique identifier of the inference endpoint. The must not match the model_id.

Query parameters

timeout string · default: 30s

Specifies the amount of time to wait for the inference endpoint to be created.

Request body

chunking_settings object

The chunking configuration object. Applies only to the sparse_embedding and text_embedding task types. Not applicable to the rerank task type.

Reference
max_chunk_size number · default: 250

The maximum size of a chunk in words. This value cannot be lower than 20 (for sentence strategy) or 10 (for word strategy). This value should not exceed the window size for the associated model.

overlap number · default: 100

The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

sentence_overlap number · default: 1

The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

separator_group string

Only applicable to the recursive strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be markdown or plaintext.

Using this parameter is an alternative to manually specifying a custom separators list.

separators string[]

Only applicable to the recursive strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the max_chunk_size limit, to reduce the total number of chunks generated.

strategy string · default: sentence

The chunking strategy: sentence, word, none or recursive.

  • If strategy is set to recursive, you must also specify:
  • max_chunk_size
  • either separators orseparator_group

Learn more about different chunking strategies in the linked documentation.

Reference
service enum required

The type of service supported for the specified task type. In this case, elasticsearch.

Values: elasticsearch
service_settings object required

Settings used to install the inference model. These settings are specific to the elasticsearch service.

adaptive_allocations object

Adaptive allocations configuration details. If enabled is true, the number of allocations of the model is set based on the current load the process gets. When the load is high, a new model allocation is automatically created, respecting the value of max_number_of_allocations if it's set. When the load is low, a model allocation is automatically removed, respecting the value of min_number_of_allocations if it's set. If enabled is true, do not set the number of allocations manually.

enabled boolean · default: false

Turn on adaptive_allocations.

max_number_of_allocations number

The maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.

min_number_of_allocations number

The minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.

deployment_id string

The deployment identifier for a trained model deployment. When deployment_id is used the model_id is optional.

model_id string required

The name of the model to use for the inference task. It can be the ID of a built-in model (for example, .multilingual-e5-small for E5) or a text embedding model that was uploaded by using the Eland client.

Reference
num_allocations number

The total number of allocations that are assigned to the model across machine learning nodes. Increasing this value generally increases the throughput. If adaptive allocations are enabled, do not set this value because it's automatically set.

num_threads number required

The number of threads used by each model allocation during inference. This setting generally increases the speed per inference request. The inference process is a compute-bound process; threads_per_allocations must not exceed the number of available allocated processors per node. The value must be a power of 2. The maximum value is 32.

long_document_strategy string

Available only for the rerank task type using the Elastic reranker model. Controls the strategy used for processing long documents during inference.

Possible values:

  • truncate (default): Processes only the beginning of each document.
  • chunk: Splits long documents into smaller parts (chunks) before inference.

When long_document_strategy is set to chunk, Elasticsearch splits each document into smaller parts but still returns a single score per document. That score reflects the highest relevance score among all chunks.

max_chunks_per_doc number

Only for the rerank task type. Limits the number of chunks per document that are sent for inference when chunking is enabled. If not set, all chunks generated for the document are processed.

task_settings object

Settings to configure the inference task. These settings are specific to the task type you specified.

return_documents boolean · default: true

For a rerank task, return the document instead of only the index.

Response

application/json
chunking_settings object

The chunking configuration object. Applies only to the embedding, sparse_embedding and text_embedding task types. Not applicable to the rerank, completion, or chat_completion task types.

max_chunk_size number · default: 250

The maximum size of a chunk in words. This value cannot be lower than 20 (for sentence strategy) or 10 (for word strategy). This value should not exceed the window size for the associated model.

overlap number · default: 100

The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

sentence_overlap number · default: 1

The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

separator_group string

Only applicable to the recursive strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be markdown or plaintext.

Using this parameter is an alternative to manually specifying a custom separators list.

separators string[]

Only applicable to the recursive strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the max_chunk_size limit, to reduce the total number of chunks generated.

strategy string · default: sentence

The chunking strategy: sentence, word, none or recursive.

  • If strategy is set to recursive, you must also specify:
  • max_chunk_size
  • either separators orseparator_group

Learn more about different chunking strategies in the linked documentation.

Reference
service string required

The service type

service_settings object required

Settings specific to the service

task_settings object

Task settings specific to the service and task type

inference_id string required

The inference Id

task_type enum required

The task type

Values: sparse_embedding text_embedding rerank