---
type: api
title: Create an Azure OpenAI inference endpoint
description: Create an inference endpoint to perform an inference task with the `azureopenai` service.
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/operation/operation-inference-put-azureopenai
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/operation/operation-inference-put-azureopenai
products:
  - Elasticsearch
---

# Create an Azure OpenAI inference endpoint

Availability: 8.14+

## Paths

- `PUT` `/_inference/{task_type}/{azureopenai_inference_id}`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#apikeyauth)
- [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#basicauth)
- [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#bearerauth)

#### Path Parameters

- `task_type` (enum AzureOpenAITaskType)
  The type of the inference task that the model will perform.
NOTE: The `chat_completion` task type only supports streaming and only through the _stream API.
  Values: `completion`, `chat_completion`, `text_embedding`
- `azureopenai_inference_id` (string Id)
  The unique identifier of the inference endpoint.

## Description

Create an inference endpoint to perform an inference task with the `azureopenai` service.

The list of chat completion models that you can choose from in your Azure OpenAI deployment include:

* [GPT-4 and GPT-4 Turbo models](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#gpt-4-and-gpt-4-turbo-models)
* [GPT-3.5](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#gpt-35)

The list of embeddings models that you can choose from in your deployment can be found in the [Azure models documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#embeddings).

## Required authorization

* Cluster privileges: `manage_inference`

## Query String Parameters

- `timeout` (string Duration · default: 30s)
  Specifies the amount of time to wait for the inference endpoint to be created.

## Request Body

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings) — optional
  The chunking configuration object.
Applies only to the `text_embedding` task type.
Not applicable to the `completion` and `chat_completion` task types.
  - `max_chunk_size` (number · default: 250) — optional
    The maximum size of a chunk in words.
This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
This value should not exceed the window size for the associated model.
  - `overlap` (number · default: 100) — optional
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1) — optional
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `separator_group` (string) — optional
    Only applicable to the `recursive` strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type.
Values can be `markdown` or `plaintext`.

Using this parameter is an alternative to manually specifying a custom `separators` list.
  - `separators` ([] string) — optional
    Only applicable to the `recursive` strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern.
The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
the `max_chunk_size` limit, to reduce the total number of chunks generated.
  - `strategy` (string · default: sentence) — optional
    The chunking strategy: `sentence`, `word`, `none` or `recursive`.

 * If `strategy` is set to `recursive`, you must also specify:

- `max_chunk_size`
- either `separators` or`separator_group`

Learn more about different chunking strategies in the linked documentation.
- `service` (enum AzureOpenAIServiceType) — required
  The type of service supported for the specified task type. In this case, `azureopenai`.
  Values: `azureopenai`
- `service_settings` ({} AzureOpenAIServiceSettings) — required
  Settings used to install the inference model. These settings are specific to the `azureopenai` service.
  - `api_key` (string) — optional
    A valid API key for your Azure OpenAI account.

IMPORTANT: You must specify either `api_key`, `entra_id`, or `client_secret`.
If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.
  - `api_version` (string) — required
    The Azure API version ID to use.
It is recommended to use the latest supported non-preview version.
  - `client_id` (string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The application ID that's assigned to your app.

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `client_secret` (string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The application secret that you created in the Microsoft app registration portal for your app.

IMPORTANT: You must specify either `api_key`, `entra_id`, or `client_secret`.
If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `deployment_id` (string) — required
    The deployment name of your deployed models.
Your Azure OpenAI deployments can be found though the Azure OpenAI Studio portal that is linked to your subscription.
  - `entra_id` (string) — optional
    A valid Microsoft Entra token.

IMPORTANT: You must specify either `api_key`, `entra_id`, or `client_secret`.
If you do not provide one or you provide more than one of them, you will receive an error when you try to create your endpoint.
  - `rate_limit` ({} RateLimitSetting) — optional
    This setting helps to minimize the number of rate limit errors returned from Azure.
The `azureopenai` service sets a default number of requests allowed per minute depending on the task type.
For `text_embedding`, it is set to `1440`.
For `completion` and `chat_completion`, it is set to `120`.
    - `requests_per_minute` (number) — optional
      The number of requests allowed per minute.
By default, the number of requests allowed per minute is set by each service as follows:

* `alibabacloud-ai-search` service: `1000`
* `amazonbedrock` service: `240`
* `anthropic` service: `50`
* `azureaistudio` service: `240`
* `azureopenai` service and task type `text_embedding`: `1440`
* `azureopenai` service and task types `completion` or `chat_completion`: `120`
* `cohere` service: `10000`
* `contextualai` service: `1000`
* `elastic` service and task type `chat_completion`: `240`
* `fireworksai` service: `6000`
* `googleaistudio` service: `360`
* `googlevertexai` service: `30000`
* `hugging_face` service: `3000`
* `jinaai` service: `2000`
* `llama` service: `3000`
* `mistral` service: `240`
* `openai` service and task type `text_embedding`: `3000`
* `openai` service and task type `completion`: `500`
* `openshift_ai` service: `3000`
* `voyageai` service: `2000`
* `watsonxai` service: `120`
  - `resource_name` (string) — required
    The name of your Azure OpenAI resource.
You can find this from the list of resources in the Azure Portal for your subscription.
  - `scopes` ([] string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The resource identifier (application ID URI) of the resource you want, suffixed with .default
For example:
```
"scopes": [
  "https://cognitiveservices.azure.com/.default"
]
```

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
  - `tenant_id` (string) — optional
    For OAuth 2.0 authorization using the client credentials grant flow.
The directory tenant the application plans to operate against.

IMPORTANT: To configure OAuth 2.0, you must specify `client_id`, `scopes`, `tenant_id`, and `client_secret` together.
If one of the fields is missing, you will receive an error when you try to create your endpoint.
- `task_settings` ({} AzureOpenAITaskSettings) — optional
  Settings to configure the inference task.
These settings are specific to the task type you specified.
  - `user` (string) — optional
    Specifies the user issuing the request.
This information can be used for abuse detection.
  - `headers` (map string to string) — optional
    Specifies custom HTTP header parameters.
For example:
```
"headers": {
  "Custom-Header": "Some-Value",
  "Another-Custom-Header": "Another-Value"
}
```

## Response

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings)
  The chunking configuration object.
Applies only to the `embedding`, `sparse_embedding` and `text_embedding` task types.
Not applicable to the `rerank`, `completion`, or `chat_completion` task types.
  - `max_chunk_size` (number · default: 250)
    The maximum size of a chunk in words.
This value cannot be lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
This value should not exceed the window size for the associated model.
  - `overlap` (number · default: 100)
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1)
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `separator_group` (string)
    Only applicable to the `recursive` strategy and required when using it.

Sets a predefined list of separators in the saved chunking settings based on the selected text type.
Values can be `markdown` or `plaintext`.

Using this parameter is an alternative to manually specifying a custom `separators` list.
  - `separators` ([] string)
    Only applicable to the `recursive` strategy and required when using it.

A list of strings used as possible split points when chunking text.

Each string can be a plain string or a regular expression (regex) pattern.
The system tries each separator in order to split the text, starting from the first item in the list.

After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
the `max_chunk_size` limit, to reduce the total number of chunks generated.
  - `strategy` (string · default: sentence)
    The chunking strategy: `sentence`, `word`, `none` or `recursive`.

 * If `strategy` is set to `recursive`, you must also specify:

- `max_chunk_size`
- either `separators` or`separator_group`

Learn more about different chunking strategies in the linked documentation.
- `service` (string)
  The service type
- `service_settings` (ServiceSettings)
  Settings specific to the service
- `task_settings` (TaskSettings)
  Task settings specific to the service and task type
- `inference_id` (string)
  The inference Id
- `task_type` (enum TaskTypeAzureOpenAI)
  The task type
  Values: `text_embedding`, `completion`, `chat_completion`
## Code Examples

#### Console

```console
PUT _inference/text_embedding/azure_openai_embeddings
{
    "service": "azureopenai",
    "service_settings": {
        "api_key": "Api-Key",
        "resource_name": "Resource-name",
        "deployment_id": "Deployment-id",
        "api_version": "2024-02-01"
    }
}
```

#### JavaScript

```javascript
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "azure_openai_embeddings",
  inference_config: {
    service: "azureopenai",
    service_settings: {
      api_key: "Api-Key",
      resource_name: "Resource-name",
      deployment_id: "Deployment-id",
      api_version: "2024-02-01",
    },
  },
});
```

#### Python

```python
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="azure_openai_embeddings",
    inference_config={
        "service": "azureopenai",
        "service_settings": {
            "api_key": "Api-Key",
            "resource_name": "Resource-name",
            "deployment_id": "Deployment-id",
            "api_version": "2024-02-01"
        }
    },
)
```

#### curl

```curl
curl -X PUT "$ELASTICSEARCH_URL/_inference/text_embedding/azure_openai_embeddings" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "azureopenai",
    "service_settings": {
      "api_key": "Api-Key",
      "resource_name": "Resource-name",
      "deployment_id": "Deployment-id",
      "api_version": "2024-02-01"
    }
  }'
```

#### Java

```java
client.inference().put(p -> p
    .inferenceId("azure_openai_embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("azureopenai")
        .serviceSettings(JsonData.fromJson("""
{"api_key":"Api-Key","resource_name":"Resource-name","deployment_id":"Deployment-id","api_version":"2024-02-01"}
"""))
    )
);
```

#### C#

```c#
var response = await client.Inference
    .PutAsync(new PutInferenceRequest()
    {
        InferenceId = "azure_openai_embeddings",
        TaskType = TaskType.TextEmbedding,
        InferenceConfig = new()
        {
            Service = "azureopenai",
            ServiceSettings = JsonSerializer.Deserialize<JsonElement>("""
            {
              "api_key": "Api-Key",
              "resource_name": "Resource-name",
              "deployment_id": "Deployment-id",
              "api_version": "2024-02-01"
            }
            """)
        }
    });
```

#### PHP

```php
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "azure_openai_embeddings",
    "body" => [
        "service" => "azureopenai",
        "service_settings" => [
            "api_key" => "Api-Key",
            "resource_name" => "Resource-name",
            "deployment_id" => "Deployment-id",
            "api_version" => "2024-02-01",
        ],
    ],
]);
```

#### Ruby

```ruby
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "azure_openai_embeddings",
  body: {
    "service": "azureopenai",
    "service_settings": {
      "api_key": "Api-Key",
      "resource_name": "Resource-name",
      "deployment_id": "Deployment-id",
      "api_version": "2024-02-01"
    }
  }
)
```

## Response Examples

#### A text embedding task

A successful response when creating an Azure OpenAI `text_embedding` inference endpoint.

```json
{
  "inference_id": "azure_openai_embeddings",
  "task_type": "text_embedding",
  "service": "azureopenai",
  "service_settings": {
    "resource_name": "Resource-name",
    "deployment_id": "Deployment-id",
    "api_version": "2024-02-01",
    "rate_limit": {
      "requests_per_minute": 1140
    },
    "dimensions": 1536,
    "similarity": "dot_product"
  },
  "chunking_settings": {
    "strategy": "sentence",
    "max_chunk_size": 250,
    "sentence_overlap": 1
  }
}
```

#### A completion task

A successful response when creating an Azure OpenAI `completion` inference endpoint.

```json
{
  "inference_id": "azure_openai_completion",
  "task_type": "completion",
  "service": "azureopenai",
  "service_settings": {
    "resource_name": "Resource-name",
    "deployment_id": "Deployment-id",
    "api_version": "2024-02-01",
    "rate_limit": {
      "requests_per_minute": 120
    }
  }
}
```

#### A chat completion task

A successful response when creating an Azure OpenAI `chat_completion` inference endpoint.

```json
{
  "inference_id": "azure_openai_chat_completion",
  "task_type": "chat_completion",
  "service": "azureopenai",
  "service_settings": {
    "resource_name": "Resource-name",
    "deployment_id": "Deployment-id",
    "api_version": "2024-02-01",
    "rate_limit": {
      "requests_per_minute": 120
    }
  }
}
```