---
type: api
title: Create an OpenAI inference endpoint
description: Create an inference endpoint to perform an inference task with the `openai` service or `openai` compatible APIs.
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch-serverless/operation/operation-inference-put-openai
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch-serverless/operation/operation-inference-put-openai
products:
  - Elasticsearch Serverless
---

# Create an OpenAI inference endpoint

Availability:

## Paths

- `PUT` `/_inference/{task_type}/{openai_inference_id}`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch-serverless/authentication#apikeyauth)

#### Path Parameters

- `task_type` (enum OpenAITaskType)
  The type of the inference task that the model will perform.
NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.
  Values: `chat_completion`, `completion`, `text_embedding`, `embedding`
- `openai_inference_id` (string Id)
  The unique identifier of the inference endpoint.

## Description

Create an inference endpoint to perform an inference task with the `openai` service or `openai` compatible APIs.

## Required authorization

* Cluster privileges: `manage_inference`

## Query String Parameters

- `timeout` (string Duration · default: 30s)
  Specifies the amount of time to wait for the inference endpoint to be created.

## Request Body

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings) — optional
  The chunking configuration object.
Applies only to the `text_embedding` task type.
Not applicable to the `completion` or `chat_completion` task types.
  - `max_chunk_size` (number · default: 250) — optional
    The maximum size of a chunk in words.
This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
This value should not exceed the window size for the associated model.
  - `overlap` (number · default: 100) — optional
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1) — optional
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `separator_group` (string) — optional
    Only applicable to the `recursive` strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type.
Values can be `markdown` or `plaintext`.

Using this parameter is an alternative to manually specifying a custom `separators` list.
  - `separators` ([] string) — optional
    Only applicable to the `recursive` strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern.
The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
the `max_chunk_size` limit, to reduce the total number of chunks generated.
  - `strategy` (string · default: sentence) — optional
    The chunking strategy: `sentence`, `word`, `none` or `recursive`.

 * If `strategy` is set to `recursive`, you must also specify:

- `max_chunk_size`
- either `separators` or`separator_group`

Learn more about different chunking strategies in the linked documentation.
- `service` (enum OpenAIServiceType) — required
  The type of service supported for the specified task type. In this case, `openai`.
  Values: `openai`
- `service_settings` ({} OpenAIServiceSettings) — required
  Settings used to install the inference model. These settings are specific to the `openai` service.
  - `api_key` (string) — optional
    A valid API key of your OpenAI account.
You can find your OpenAI API keys in your OpenAI account under the API keys section.

IMPORTANT: You must specify either `api_key` or `client_secret`.
If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.
  - `client_id` (string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The application ID that's assigned to your app.

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `client_secret` (string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The application secret that you created for your app.

IMPORTANT: You must specify either `api_key` or `client_secret`.
If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `dimensions` (number) — optional
    For a `text_embedding` or `embedding` task, the number of dimensions the resulting output embeddings should have.
It is supported only in `text-embedding-3` and later models.
If it is not set, the OpenAI defined default for the model is used.
  - `model_id` (string) — required
    The name of the model to use for the inference task.
Refer to the OpenAI documentation for the list of available text embedding models.
  - `organization_id` (string) — optional
    The unique identifier for your organization.
You can find the Organization ID in your OpenAI account under *Settings > Organizations*.
  - `rate_limit` ({} RateLimitSetting) — optional
    This setting helps to minimize the number of rate limit errors returned from OpenAI.
The `openai` service sets a default number of requests allowed per minute depending on the task type.
For `text_embedding` and `embedding`, it is set to `3000`.
For `completion` and `chat_completion`, it is set to `500`.
    - `requests_per_minute` (number) — optional
      The number of requests allowed per minute.
By default, the number of requests allowed per minute is set by each service as follows:

* `alibabacloud-ai-search` service: `1000`
* `amazonbedrock` service: `240`
* `anthropic` service: `50`
* `azureaistudio` service: `240`
* `azureopenai` service and task type `text_embedding`: `1440`
* `azureopenai` service and task types `completion` or `chat_completion`: `120`
* `cohere` service: `10000`
* `contextualai` service: `1000`
* `elastic` service and task type `chat_completion`: `240`
* `fireworksai` service: `6000`
* `googleaistudio` service: `360`
* `googlevertexai` service: `30000`
* `hugging_face` service: `3000`
* `jinaai` service: `2000`
* `llama` service: `3000`
* `mistral` service: `240`
* `openai` service and task type `text_embedding`: `3000`
* `openai` service and task type `completion`: `500`
* `openshift_ai` service: `3000`
* `voyageai` service: `2000`
* `watsonxai` service: `120`
  - `scopes` ([] string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The resource identifier of the resource you want.
For example:
```
"scopes": [
  "scope1",
  "scope2"
]
```

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `similarity` (enum OpenAISimilarityType) — optional
    For a `text_embedding` or `embedding` task, the similarity measure. One of `cosine`, `dot_product`, `l2_norm`. Defaults to `dot_product`.
    Values: `cosine`, `dot_product`, `l2_norm`
  - `token_url` (string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
An OAuth2 token endpoint where Elasticsearch sends a request to exchange client credentials for an access token.

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `token_url`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `url` (string) — optional
    The URL endpoint to use for the requests.
It can be changed for testing purposes.
Default value is `https://api.openai.com/v1/embeddings` for a `text_embedding` or `embedding` task,
`https://api.openai.com/v1/chat/completions` for a `completion` or `chat_completion` task.
- `task_settings` ({} OpenAITaskSettings) — optional
  Settings to configure the inference task.
These settings are specific to the task type you specified.
  - `user` (string) — optional
    Specifies the user issuing the request.
This information can be used for abuse detection.
  - `headers` (map string to string) — optional
    Specifies custom HTTP header parameters.
For example:
```
"headers": {
  "Custom-Header": "Some-Value",
  "Another-Custom-Header": "Another-Value"
}
```

## Response

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings)
  The chunking configuration object.
Applies only to the `embedding`, `sparse_embedding` and `text_embedding` task types.
Not applicable to the `rerank`, `completion`, or `chat_completion` task types.
  - `max_chunk_size` (number · default: 250)
    The maximum size of a chunk in words.
This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
This value should not exceed the window size for the associated model.
  - `overlap` (number · default: 100)
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1)
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `separator_group` (string)
    Only applicable to the `recursive` strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type.
Values can be `markdown` or `plaintext`.

Using this parameter is an alternative to manually specifying a custom `separators` list.
  - `separators` ([] string)
    Only applicable to the `recursive` strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern.
The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
the `max_chunk_size` limit, to reduce the total number of chunks generated.
  - `strategy` (string · default: sentence)
    The chunking strategy: `sentence`, `word`, `none` or `recursive`.

 * If `strategy` is set to `recursive`, you must also specify:

- `max_chunk_size`
- either `separators` or`separator_group`

Learn more about different chunking strategies in the linked documentation.
- `service` (string)
  The service type
- `service_settings` (ServiceSettings)
  Settings specific to the service
- `task_settings` (TaskSettings)
  Task settings specific to the service and task type
- `inference_id` (string)
  The inference Id
- `task_type` (enum TaskTypeOpenAI)
  The task type
  Values: `text_embedding`, `chat_completion`, `completion`, `embedding`
## Code Examples

#### Console

```console
PUT _inference/text_embedding/openai-embeddings
{
    "service": "openai",
    "service_settings": {
        "api_key": "OpenAI-API-Key",
        "model_id": "text-embedding-3-small",
        "dimensions": 128
    }
}
```

#### JavaScript

```javascript
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "openai-embeddings",
  inference_config: {
    service: "openai",
    service_settings: {
      api_key: "OpenAI-API-Key",
      model_id: "text-embedding-3-small",
      dimensions: 128,
    },
  },
});
```

#### Python

```python
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="openai-embeddings",
    inference_config={
        "service": "openai",
        "service_settings": {
            "api_key": "OpenAI-API-Key",
            "model_id": "text-embedding-3-small",
            "dimensions": 128
        }
    },
)
```

#### curl

```curl
curl -X PUT "$ELASTICSEARCH_URL/_inference/text_embedding/openai-embeddings" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "openai",
    "service_settings": {
      "api_key": "OpenAI-API-Key",
      "model_id": "text-embedding-3-small",
      "dimensions": 128
    }
  }'
```

#### Java

```java
client.inference().put(p -> p
    .inferenceId("openai-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("openai")
        .serviceSettings(JsonData.fromJson("""
{"api_key":"OpenAI-API-Key","model_id":"text-embedding-3-small","dimensions":128}
"""))
    )
);
```

#### C#

```c#
var response = await client.Inference
    .PutAsync(new PutInferenceRequest()
    {
        InferenceId = "openai-embeddings",
        TaskType = TaskType.TextEmbedding,
        InferenceConfig = new()
        {
            Service = "openai",
            ServiceSettings = JsonSerializer.Deserialize<JsonElement>("""
            {
              "api_key": "OpenAI-API-Key",
              "model_id": "text-embedding-3-small",
              "dimensions": 128
            }
            """)
        }
    });
```

#### PHP

```php
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "openai-embeddings",
    "body" => [
        "service" => "openai",
        "service_settings" => [
            "api_key" => "OpenAI-API-Key",
            "model_id" => "text-embedding-3-small",
            "dimensions" => 128,
        ],
    ],
]);
```

#### Ruby

```ruby
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "openai-embeddings",
  body: {
    "service": "openai",
    "service_settings": {
      "api_key": "OpenAI-API-Key",
      "model_id": "text-embedding-3-small",
      "dimensions": 128
    }
  }
)
```