---
type: api
title: Create a custom inference endpoint
description: The custom service gives more control over how to interact with external inference services that aren't explicitly supported through dedicated integrations.
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/operation/operation-inference-put-custom
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/operation/operation-inference-put-custom
products:
  - Elasticsearch
---

# Create a custom inference endpoint

Availability: 8.19+

## Paths

- `PUT` `/_inference/{task_type}/{custom_inference_id}`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/authentication#apikeyauth)
- [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/authentication#basicauth)
- [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/authentication#bearerauth)

#### Path Parameters

- `task_type` (enum CustomTaskType)
  The type of the inference task that the model will perform.
  Values: `text_embedding`, `sparse_embedding`, `rerank`, `completion`
- `custom_inference_id` (string Id)
  The unique identifier of the inference endpoint.

## Description

The custom service gives more control over how to interact with external inference services that aren't explicitly supported through dedicated integrations.
The custom service gives you the ability to define the headers, url, query parameters, request body, and secrets.
The custom service supports the template replacement functionality, which enables you to define a template that can be replaced with the value associated with that key.
Templates are portions of a string that start with `${` and end with `}`.
The parameters `secret_parameters` and `task_settings` are checked for keys for template replacement. Template replacement is supported in the `request`, `headers`, `url`, and `query_parameters`.
If the definition (key) is not found for a template, an error message is returned.
In case of an endpoint definition like the following:
```
PUT _inference/text_embedding/test-text-embedding
{
  "service": "custom",
  "service_settings": {
     "secret_parameters": {
          "api_key": "<some api key>"
     },
     "url": "...endpoints.huggingface.cloud/v1/embeddings",
     "headers": {
         "Authorization": "Bearer ${api_key}",
         "Content-Type": "application/json"
     },
     "request": "{\"input\": ${input}}",
     "response": {
         "json_parser": {
             "text_embeddings":"$.data[*].embedding[*]"
         }
     }
  }
}
```
To replace `${api_key}` the `secret_parameters` and `task_settings` are checked for a key named `api_key`.

> info
> Templates should not be surrounded by quotes.

Pre-defined templates:
* `${input}` refers to the array of input strings that comes from the `input` field of the subsequent inference requests.
* `${input_type}` refers to the input type translation values.
* `${query}` refers to the query field used specifically for reranking tasks.
* `${top_n}` refers to the `top_n` field available when performing rerank requests.
* `${return_documents}` refers to the `return_documents` field available when performing rerank requests.

## Required authorization

* Cluster privileges: `manage_inference`

## Request Body

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings) — optional
  The chunking configuration object.
Applies only to the `sparse_embedding` or `text_embedding` task types.
Not applicable to the `rerank` or `completion` task types.
  - `max_chunk_size` (number · default: 250) — optional
    The maximum size of a chunk in words.
This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
This value should not exceed the window size for the associated model.
  - `overlap` (number · default: 100) — optional
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1) — optional
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `separator_group` (string) — optional
    Only applicable to the `recursive` strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type.
Values can be `markdown` or `plaintext`.

Using this parameter is an alternative to manually specifying a custom `separators` list.
  - `separators` ([] string) — optional
    Only applicable to the `recursive` strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern.
The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
the `max_chunk_size` limit, to reduce the total number of chunks generated.
  - `strategy` (string · default: sentence) — optional
    The chunking strategy: `sentence`, `word`, `none` or `recursive`.

 * If `strategy` is set to `recursive`, you must also specify:

- `max_chunk_size`
- either `separators` or`separator_group`

Learn more about different chunking strategies in the linked documentation.
- `service` (enum CustomServiceType) — required
  The type of service supported for the specified task type. In this case, `custom`.
  Values: `custom`
- `service_settings` ({} CustomServiceSettings) — required
  Settings used to install the inference model.
These settings are specific to the `custom` service.
  - `batch_size` (number) — optional
    Specifies the batch size used for the semantic_text field. If the field is not provided, the default is 10.
The batch size is the maximum number of inputs in a single request to the upstream service.
The chunk within the batch are controlled by the selected chunking strategy for the semantic_text field.
  - `headers` (map string to string) — optional
    Specifies the HTTP header parameters – such as `Authorization` or `Content-Type` – that are required to access the custom service.
For example:
```
"headers": {
  "Authorization": "Bearer ${api_key}",
  "Content-Type": "application/json;charset=utf-8"
}
```
  - `input_type` (map string to string) — optional
    Specifies the input type translation values that are used to replace the `${input_type}` template in the request body.
For example:
```
"input_type": {
  "translation": {
    "ingest": "do_ingest",
    "search": "do_search"
  },
  "default": "a_default"
},
```
If the subsequent inference requests come from a search context, the `search` key will be used and the template will be replaced with `do_search`.
If it comes from the ingest context `do_ingest` is used. If it's a different context that is not specified, the default value will be used. If no default is specified an empty string is used.
`translation` can be:
* `classification`
* `clustering`
* `ingest`
* `search`
  - `query_parameters` ([] CustomServiceQueryParameter) — optional
    Specifies the query parameters as a list of tuples. The arrays inside the `query_parameters` must have two items, a key and a value.
For example:
```
"query_parameters":[
  ["param_key", "some_value"],
  ["param_key", "another_value"],
  ["other_key", "other_value"]
]
```
If the base url is `https://www.elastic.co` it results in: `https://www.elastic.co?param_key=some_value&param_key=another_value&other_key=other_value`.
  - `request` ({} CustomRequestParams) — required
    The request configuration object.
    - `content` (string) — required
      The body structure of the request. It requires passing in the string-escaped result of the JSON format HTTP request body.
For example:
```
"request": "{\"input\":${input}}"
```
> info
> The content string needs to be a single line except when using the Kibana console.
  - `response` ({} CustomResponseParams) — required
    The response configuration object.
    - `json_parser` (map string to string) — required
      Specifies the JSON parser that is used to parse the response from the custom service.
Different task types require different json_parser parameters.
For example:
```
# text_embedding
# For a response like this:

{
 "object": "list",
 "data": [
     {
       "object": "embedding",
       "index": 0,
       "embedding": [
           0.014539449,
           -0.015288644
       ]
     }
 ],
 "model": "text-embedding-ada-002-v2",
 "usage": {
     "prompt_tokens": 8,
     "total_tokens": 8
 }
}

# the json_parser definition should look like this:

"response":{
  "json_parser":{
    "text_embeddings":"$.data[*].embedding[*]"
  }
}

# Elasticsearch supports the following embedding types:
* float
* byte
* bit (or binary)

To specify the embedding type for the response, the `embedding_type`
field should be added in the `json_parser` object. Here's an example:
"response":{
  "json_parser":{
    "text_embeddings":"$.data[*].embedding[*]",
    "embedding_type":"bit"
  }
}

If `embedding_type` is not specified, it defaults to `float`.

# sparse_embedding
# For a response like this:

{
  "request_id": "75C50B5B-E79E-4930-****-F48DBB392231",
  "latency": 22,
  "usage": {
     "token_count": 11
  },
  "result": {
     "sparse_embeddings": [
        {
          "index": 0,
          "embedding": [
            {
              "token_id": 6,
              "weight": 0.101
            },
            {
              "token_id": 163040,
              "weight": 0.28417
            }
          ]
        }
     ]
  }
}

# the json_parser definition should look like this:

"response":{
  "json_parser":{
    "token_path":"$.result.sparse_embeddings[*].embedding[*].token_id",
    "weight_path":"$.result.sparse_embeddings[*].embedding[*].weight"
  }
}

# rerank
# For a response like this:

{
  "results": [
    {
      "index": 3,
      "relevance_score": 0.999071,
      "document": "abc"
    },
    {
      "index": 4,
      "relevance_score": 0.7867867,
      "document": "123"
    },
    {
      "index": 0,
      "relevance_score": 0.32713068,
      "document": "super"
    }
  ],
}

# the json_parser definition should look like this:

"response":{
  "json_parser":{
    "reranked_index":"$.result.scores[*].index",    // optional
    "relevance_score":"$.result.scores[*].score",
    "document_text":"xxx"    // optional
  }
}

# completion
# For a response like this:

{
 "id": "chatcmpl-B9MBs8CjcvOU2jLn4n570S5qMJKcT",
 "object": "chat.completion",
 "created": 1741569952,
 "model": "gpt-4.1-2025-04-14",
 "choices": [
   {
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I assist you today?",
      "refusal": null,
      "annotations": []
    },
    "logprobs": null,
    "finish_reason": "stop"
  }
 ]
}

# the json_parser definition should look like this:

"response":{
  "json_parser":{
    "completion_result":"$.choices[*].message.content"
  }
}
  - `secret_parameters` (map string to string) — required
    Specifies secret parameters, like `api_key` or `api_token`, that are required to access the custom service.
For example:
```
"secret_parameters":{
  "api_key":"<api_key>"
}
```
  - `url` (string) — optional
    The URL endpoint to use for the requests.
- `task_settings` ({} CustomTaskSettings) — optional
  Settings to configure the inference task.
These settings are specific to the task type you specified.
  - `parameters` (map string to boolean | number | string) — optional
    Specifies parameters that are required to run the custom service. The parameters depend on the model your custom service uses.
For example:
```
"task_settings":{
  "parameters":{
    "input_type":"query",
    "return_token":true
  }
}
```

> warn
> The `task_settings.parameters` cannot contain the same keys as `secret_parameters`. If they do, an error will be returned. This applies to PUT requests and POST requests.

## Response

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings)
  The chunking configuration object.
Applies only to the `embedding`, `sparse_embedding` and `text_embedding` task types.
Not applicable to the `rerank`, `completion`, or `chat_completion` task types.
  - `max_chunk_size` (number · default: 250)
    The maximum size of a chunk in words.
This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
This value should not exceed the window size for the associated model.
  - `overlap` (number · default: 100)
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1)
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `separator_group` (string)
    Only applicable to the `recursive` strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type.
Values can be `markdown` or `plaintext`.

Using this parameter is an alternative to manually specifying a custom `separators` list.
  - `separators` ([] string)
    Only applicable to the `recursive` strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern.
The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
the `max_chunk_size` limit, to reduce the total number of chunks generated.
  - `strategy` (string · default: sentence)
    The chunking strategy: `sentence`, `word`, `none` or `recursive`.

 * If `strategy` is set to `recursive`, you must also specify:

- `max_chunk_size`
- either `separators` or`separator_group`

Learn more about different chunking strategies in the linked documentation.
- `service` (string)
  The service type
- `service_settings` (ServiceSettings)
  Settings specific to the service
- `task_settings` (TaskSettings)
  Task settings specific to the service and task type
- `inference_id` (string)
  The inference Id
- `task_type` (enum TaskTypeCustom)
  The task type
  Values: `text_embedding`, `sparse_embedding`, `rerank`, `completion`
## Code Examples

#### Console

```console
PUT _inference/text_embedding/custom-embeddings
{
    "service": "custom",
    "service_settings": {
        "secret_parameters": {
            "api_key": "<api key>"
        },
        "url": "https://api.openai.com/v1/embeddings",
        "headers": {
            "Authorization": "Bearer ${api_key}",
            "Content-Type": "application/json;charset=utf-8"
        },
        "request": "{\"input\": ${input}, \"model\": \"text-embedding-3-small\"}",
        "response": {
            "json_parser": {
                "text_embeddings": "$.data[*].embedding[*]"
            }
        }
    }
}
```

#### JavaScript

```javascript
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "custom-embeddings",
  inference_config: {
    service: "custom",
    service_settings: {
      secret_parameters: {
        api_key: "<api key>",
      },
      url: "https://api.openai.com/v1/embeddings",
      headers: {
        Authorization: "Bearer ${api_key}",
        "Content-Type": "application/json;charset=utf-8",
      },
      request: '{"input": ${input}, "model": "text-embedding-3-small"}',
      response: {
        json_parser: {
          text_embeddings: "$.data[*].embedding[*]",
        },
      },
    },
  },
});
```

#### Python

```python
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="custom-embeddings",
    inference_config={
        "service": "custom",
        "service_settings": {
            "secret_parameters": {
                "api_key": "<api key>"
            },
            "url": "https://api.openai.com/v1/embeddings",
            "headers": {
                "Authorization": "Bearer ${api_key}",
                "Content-Type": "application/json;charset=utf-8"
            },
            "request": "{\"input\": ${input}, \"model\": \"text-embedding-3-small\"}",
            "response": {
                "json_parser": {
                    "text_embeddings": "$.data[*].embedding[*]"
                }
            }
        }
    },
)
```

#### curl

```curl
curl -X PUT "$ELASTICSEARCH_URL/_inference/text_embedding/custom-embeddings" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "custom",
    "service_settings": {
      "secret_parameters": {
        "api_key": "\u003Capi key\u003E"
      },
      "url": "https://api.openai.com/v1/embeddings",
      "headers": {
        "Authorization": "Bearer ${api_key}",
        "Content-Type": "application/json;charset=utf-8"
      },
      "request": "{\u0022input\u0022: ${input}, \u0022model\u0022: \u0022text-embedding-3-small\u0022}",
      "response": {
        "json_parser": {
          "text_embeddings": "$.data[*].embedding[*]"
        }
      }
    }
  }'
```

#### Java

```java
client.inference().put(p -> p
    .inferenceId("custom-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("custom")
        .serviceSettings(JsonData.fromJson("""
{"secret_parameters":{"api_key":"<api key>"},"url":"https://api.openai.com/v1/embeddings","headers":{"Authorization":"Bearer ${api_key}","Content-Type":"application/json;charset=utf-8"},"request":"{\"input\": ${input}, \"model\": \"text-embedding-3-small\"}","response":{"json_parser":{"text_embeddings":"$.data[*].embedding[*]"}}}
"""))
    )
);
```

#### C#

```c#
var response = await client.Inference
    .PutAsync(new PutInferenceRequest()
    {
        InferenceId = "custom-embeddings",
        TaskType = TaskType.TextEmbedding,
        InferenceConfig = new()
        {
            Service = "custom",
            ServiceSettings = JsonSerializer.Deserialize<JsonElement>("""
            {
              "secret_parameters": {
                "api_key": "\u003Capi key\u003E"
              },
              "url": "https://api.openai.com/v1/embeddings",
              "headers": {
                "Authorization": "Bearer ${api_key}",
                "Content-Type": "application/json;charset=utf-8"
              },
              "request": "{\u0022input\u0022: ${input}, \u0022model\u0022: \u0022text-embedding-3-small\u0022}",
              "response": {
                "json_parser": {
                  "text_embeddings": "$.data[*].embedding[*]"
                }
              }
            }
            """)
        }
    });
```

#### PHP

```php
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "custom-embeddings",
    "body" => [
        "service" => "custom",
        "service_settings" => [
            "secret_parameters" => [
                "api_key" => "<api key>",
            ],
            "url" => "https://api.openai.com/v1/embeddings",
            "headers" => [
                "Authorization" => "Bearer \${api_key}",
                "Content-Type" => "application/json;charset=utf-8",
            ],
            "request" => "{\"input\": \${input}, \"model\": \"text-embedding-3-small\"}",
            "response" => [
                "json_parser" => [
                    "text_embeddings" => "$.data[*].embedding[*]",
                ],
            ],
        ],
    ],
]);
```

#### Ruby

```ruby
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "custom-embeddings",
  body: {
    "service": "custom",
    "service_settings": {
      "secret_parameters": {
        "api_key": "<api key>"
      },
      "url": "https://api.openai.com/v1/embeddings",
      "headers": {
        "Authorization": "Bearer ${api_key}",
        "Content-Type": "application/json;charset=utf-8"
      },
      "request": "{\"input\": ${input}, \"model\": \"text-embedding-3-small\"}",
      "response": {
        "json_parser": {
          "text_embeddings": "$.data[*].embedding[*]"
        }
      }
    }
  }
)
```