---
type: api
title: Perform chat completion inference
description: The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation.
url: https://www.elastic.co/elastic/docs-builder/docs/4641/api/doc/elasticsearch/v8/operation/operation-inference-chat-completion-unified
resource: https://www.elastic.co/elastic/docs-builder/docs/4641/api/doc/elasticsearch/v8/operation/operation-inference-chat-completion-unified
products:
  - Elasticsearch
---

# Perform chat completion inference

Authentication: required

Availability: Generally available since 8.18

## Paths

- `POST` `/_inference/chat_completion/{inference_id}/_stream`

#### Path parameters

- `inference_id` (string) — required
  The inference Id

## Description

The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation.
It only works with the `chat_completion` task type.

NOTE: The `chat_completion` task type is only available within the _stream API and only supports streaming.
The Chat completion inference API and the Stream inference API differ in their response structure and capabilities.
The Chat completion inference API provides more comprehensive customization options through more fields and function calling support.
To determine whether a given inference service supports this task type, please see the page for that service.

## Query string parameters

- `timeout` (string · default: 30s) — optional
  Specifies the amount of time to wait for the inference request to complete.

## Request body

`application/json`

- `messages` (object[]) — required
  A list of objects representing the conversation.
Requests should generally only add new messages from the user (role `user`).
The other message roles (`assistant`, `system`, or `tool`) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.
  - `content` (union oneOf) — optional
    The content of the message.

String example:
```
{
   "content": "Some string"
}
```

Object example:
```
{
  "content": [
      {
       "text": "Some text",
       "type": "text"
      }
   ]
}
```
    - **string**
    - **[]ContentObject**
      An object style representation of a single portion of a conversation.
      - `text` (string) — required
        The text content.
      - `type` (string) — required
        The type of content.
  - `role` (string) — required
    The role of the message author. Valid values are `user`, `assistant`, `system`, and `tool`.
  - `tool_call_id` (string) — optional
    Only for `tool` role messages. The tool call that this message is responding to.
  - `tool_calls` (object[]) — optional
    Only for `assistant` role messages. The tool calls generated by the model. If it's specified, the `content` field is optional.
Example:
```
{
  "tool_calls": [
      {
          "id": "call_KcAjWtAww20AihPHphUh46Gd",
          "type": "function",
          "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Boston, MA\"}"
          }
      }
  ]
}
```
    - `id` (string) — required
      The identifier of the tool call.
    - `function` (object) — required
      The function that the model called.
      - `arguments` (string) — required
        The arguments to call the function with in JSON format.
      - `name` (string) — required
        The name of the function to call.
    - `type` (string) — required
      The type of the tool call.
- `model` (string) — optional
  The ID of the model to use.
- `max_completion_tokens` (number) — optional
  The upper bound limit for the number of tokens that can be generated for a completion request.
- `stop` (string[]) — optional
  A sequence of strings to control when the model should stop generating additional tokens.
- `temperature` (number) — optional
  The sampling temperature to use.
- `tool_choice` (union oneOf) — optional
  Controls which tool is called by the model.
String representation: One of `auto`, `none`, or `requrired`. `auto` allows the model to choose between calling tools and generating a message. `none` causes the model to not call any tools. `required` forces the model to call one or more tools.
Example (object representation):
```
{
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_weather"
      }
  }
}
```
  - **string**
  - **CompletionToolChoice**
    Controls which tool is called by the model.
    - `type` (string) — required
      The type of the tool.
    - `function` (object) — required
      The tool choice function.
      - `name` (string) — required
        The name of the function to call.
- `tools` (object[]) — optional
  A list of tools that the model can call.
Example:
```
{
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_price_of_item",
              "description": "Get the current price of an item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "12345"
                      },
                      "unit": {
                          "type": "currency"
                      }
                  }
              }
          }
      }
  ]
}
```
  - `type` (string) — required
    The type of tool.
  - `function` (object) — required
    The function definition.
    - `description` (string) — optional
      A description of what the function does.
This is used by the model to choose when and how to call the function.
    - `name` (string) — required
      The name of the function.
    - `parameters` (object) — optional
      The parameters the functional accepts. This should be formatted as a JSON object.
    - `strict` (boolean) — optional
      Whether to enable schema adherence when generating the function call.
- `top_p` (number) — optional
  Nucleus sampling, an alternative to sampling with temperature.

## Response

`application/json`

Type: `object`

## Code examples

### A chat completion task

#### Console

```console
POST _inference/chat_completion/openai-completion/_stream
{
  "model": "gpt-4o",
  "messages": [
      {
          "role": "user",
          "content": "What is Elastic?"
      }
  ]
}
```

#### JavaScript

```javascript
const response = await client.inference.chatCompletionUnified({
  inference_id: "openai-completion",
  chat_completion_request: {
    model: "gpt-4o",
    messages: [
      {
        role: "user",
        content: "What is Elastic?",
      },
    ],
  },
});
```

#### Python

```python
resp = client.inference.chat_completion_unified(
    inference_id="openai-completion",
    chat_completion_request={
        "model": "gpt-4o",
        "messages": [
            {
                "role": "user",
                "content": "What is Elastic?"
            }
        ]
    },
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion/_stream" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is Elastic?"
      }
    ]
  }'
```

#### Java

```java
client.inference().chatCompletionUnified(c -> c
    .inferenceId("openai-completion")
    .chatCompletionRequest(ch -> ch
        .messages(m -> m
            .content(co -> co
                .string("What is Elastic?")
            )
            .role("user")
        )
        .model("gpt-4o")
    )
);
```

#### C#

```c#
var response = await client.Inference
    .ChatCompletionUnifiedAsync(inferenceId: "openai-completion", d1 => d1
        .ChatCompletionRequest(d2 => d2
            .Messages(d3 => d3
                .Content("What is Elastic?")
                .Role("user")
            )
            .Model("gpt-4o")
        )
    );
```

#### PHP

```php
$resp = $client->inference()->chatCompletionUnified([
    "inference_id" => "openai-completion",
    "body" => [
        "model" => "gpt-4o",
        "messages" => array(
            [
                "role" => "user",
                "content" => "What is Elastic?",
            ],
        ),
    ],
]);
```

#### Ruby

```ruby
response = client.inference.chat_completion_unified(
  inference_id: "openai-completion",
  body: {
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is Elastic?"
      }
    ]
  }
)
```

### A chat completion task with tool_calls

#### Console

```console
POST _inference/chat_completion/openai-completion/_stream
{
  "messages": [
      {
          "role": "assistant",
          "content": "Let's find out what the weather is",
          "tool_calls": [ 
              {
                  "id": "call_KcAjWtAww20AihPHphUh46Gd",
                  "type": "function",
                  "function": {
                      "name": "get_current_weather",
                      "arguments": "{\"location\":\"Boston, MA\"}"
                  }
              }
          ]
      },
      { 
          "role": "tool",
          "content": "The weather is cold",
          "tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
      }
  ]
}
```

#### JavaScript

```javascript
const response = await client.inference.chatCompletionUnified({
  inference_id: "openai-completion",
  chat_completion_request: {
    messages: [
      {
        role: "assistant",
        content: "Let's find out what the weather is",
        tool_calls: [
          {
            id: "call_KcAjWtAww20AihPHphUh46Gd",
            type: "function",
            function: {
              name: "get_current_weather",
              arguments: '{"location":"Boston, MA"}',
            },
          },
        ],
      },
      {
        role: "tool",
        content: "The weather is cold",
        tool_call_id: "call_KcAjWtAww20AihPHphUh46Gd",
      },
    ],
  },
});
```

#### Python

```python
resp = client.inference.chat_completion_unified(
    inference_id="openai-completion",
    chat_completion_request={
        "messages": [
            {
                "role": "assistant",
                "content": "Let's find out what the weather is",
                "tool_calls": [
                    {
                        "id": "call_KcAjWtAww20AihPHphUh46Gd",
                        "type": "function",
                        "function": {
                            "name": "get_current_weather",
                            "arguments": "{\"location\":\"Boston, MA\"}"
                        }
                    }
                ]
            },
            {
                "role": "tool",
                "content": "The weather is cold",
                "tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
            }
        ]
    },
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion/_stream" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"assistant","content":"Let' \
  "'" \
  's find out what the weather is","tool_calls":[{"id":"call_KcAjWtAww20AihPHphUh46Gd","type":"function","function":{"name":"get_current_weather","arguments":"{\"location\":\"Boston, MA\"}"}}]},{"role":"tool","content":"The weather is cold","tool_call_id":"call_KcAjWtAww20AihPHphUh46Gd"}]}'
```

#### Java

```java
client.inference().chatCompletionUnified(c -> c
  .inferenceId("openai-completion")
  .chatCompletionRequest(ch -> ch
      .messages(List.of(Message.of(m -> m
              .content(co -> co
                  .string("Let's find out what the weather is")
              )
              .role("assistant")
              .toolCalls(t -> t
                  .id("call_KcAjWtAww20AihPHphUh46Gd")
                  .function(f -> f
                      .arguments("{\"location\":\"Boston, MA\"}")
                      .name("get_current_weather")
                  )
                  .type("function")
              )),Message.of(me -> me
              .content(co -> co
                  .string("The weather is cold")
              )
              .role("tool")
              .toolCallId("call_KcAjWtAww20AihPHphUh46Gd"))))
  )
);
```

#### C#

```c#
var response = await client.Inference
    .ChatCompletionUnifiedAsync(inferenceId: "openai-completion", d1 => d1
        .ChatCompletionRequest(d2 => d2
            .Messages(d3 => d3
                .Content("Let's find out what the weather is")
                .Role("assistant")
                .ToolCalls(d4 => d4
                    .Function(d5 => d5
                        .Arguments("{\"location\":\"Boston, MA\"}")
                        .Name("get_current_weather")
                    )
                    .Id("call_KcAjWtAww20AihPHphUh46Gd")
                    .Type("function")
                ), d3 => d3
                .Content("The weather is cold")
                .Role("tool")
                .ToolCallId("call_KcAjWtAww20AihPHphUh46Gd")
            )
        )
    );
```

#### PHP

```php
$resp = $client->inference()->chatCompletionUnified([
    "inference_id" => "openai-completion",
    "body" => [
        "messages" => array(
            [
                "role" => "assistant",
                "content" => "Let's find out what the weather is",
                "tool_calls" => array(
                    [
                        "id" => "call_KcAjWtAww20AihPHphUh46Gd",
                        "type" => "function",
                        "function" => [
                            "name" => "get_current_weather",
                            "arguments" => "{\"location\":\"Boston, MA\"}",
                        ],
                    ],
                ),
            ],
            [
                "role" => "tool",
                "content" => "The weather is cold",
                "tool_call_id" => "call_KcAjWtAww20AihPHphUh46Gd",
            ],
        ),
    ],
]);
```

#### Ruby

```ruby
response = client.inference.chat_completion_unified(
  inference_id: "openai-completion",
  body: {
    "messages": [
      {
        "role": "assistant",
        "content": "Let's find out what the weather is",
        "tool_calls": [
          {
            "id": "call_KcAjWtAww20AihPHphUh46Gd",
            "type": "function",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Boston, MA\"}"
            }
          }
        ]
      },
      {
        "role": "tool",
        "content": "The weather is cold",
        "tool_call_id": "call_KcAjWtAww20AihPHphUh46Gd"
      }
    ]
  }
)
```

### A chat completion task with tools and tool_calls

#### Console

```console
POST _inference/chat_completion/openai-completion/_stream
{
  "messages": [
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "What's the price of a scarf?"
              }
          ]
      }
  ],
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_current_price",
              "description": "Get the current price of a item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "123"
                      }
                  }
              }
          }
      }
  ],
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_price"
      }
  }
}
```

#### JavaScript

```javascript
const response = await client.inference.chatCompletionUnified({
  inference_id: "openai-completion",
  chat_completion_request: {
    messages: [
      {
        role: "user",
        content: [
          {
            type: "text",
            text: "What's the price of a scarf?",
          },
        ],
      },
    ],
    tools: [
      {
        type: "function",
        function: {
          name: "get_current_price",
          description: "Get the current price of a item",
          parameters: {
            type: "object",
            properties: {
              item: {
                id: "123",
              },
            },
          },
        },
      },
    ],
    tool_choice: {
      type: "function",
      function: {
        name: "get_current_price",
      },
    },
  },
});
```

#### Python

```python
resp = client.inference.chat_completion_unified(
    inference_id="openai-completion",
    chat_completion_request={
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "What's the price of a scarf?"
                    }
                ]
            }
        ],
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_current_price",
                    "description": "Get the current price of a item",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "item": {
                                "id": "123"
                            }
                        }
                    }
                }
            }
        ],
        "tool_choice": {
            "type": "function",
            "function": {
                "name": "get_current_price"
            }
        }
    },
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion/_stream" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":[{"type":"text","text":"What' \
  "'" \
  's the price of a scarf?"}]}],"tools":[{"type":"function","function":{"name":"get_current_price","description":"Get the current price of a item","parameters":{"type":"object","properties":{"item":{"id":"123"}}}}}],"tool_choice":{"type":"function","function":{"name":"get_current_price"}}}'
```

#### Java

```java
client.inference().chatCompletionUnified(c -> c
  .inferenceId("openai-completion")
  .chatCompletionRequest(ch -> ch
      .messages(m -> m
          .content(co -> co
              .string("What's the price of a scarf?")
          )
          .role("user")
      )
      .toolChoice(t -> t
          .object(o -> o
              .type("function")
              .function(f -> f
                  .name("get_current_price")
              )
          )
      )
      .tools(to -> to
          .type("function")
          .function(f -> f
              .description("Get the current price of a item")
              .name("get_current_price")
              .parameters(JsonData.fromJson("{\"type\":\"object\",\"properties\":{\"item\":{\"id\":\"123\"}}}"))
          )
      )
  )
);
```

#### C#

```c#
var response = await client.Inference
    .ChatCompletionUnifiedAsync(inferenceId: "openai-completion", d1 => d1
        .ChatCompletionRequest(d2 => d2
            .Messages(d3 => d3
                .Content(new ContentObject[] { new()
                {
                    Text = "What's the price of a scarf?",
                    Type = "text"
                } })
                .Role("user")
            )
            .ToolChoice(new CompletionToolChoice()
            {
                Function = new()
                {
                    Name = "get_current_price"
                },
                Type = "function"
            })
            .Tools(d3 => d3
                .Function(d4 => d4
                    .Description("Get the current price of a item")
                    .Name("get_current_price")
                    .Parameters(JsonSerializer.Deserialize<JsonElement>("""
                    {
                      "type": "object",
                      "properties": {
                        "item": {
                          "id": "123"
                        }
                      }
                    }
                    """))
                )
                .Type("function")
            )
        )
    );
```

#### PHP

```php
$resp = $client->inference()->chatCompletionUnified([
    "inference_id" => "openai-completion",
    "body" => [
        "messages" => array(
            [
                "role" => "user",
                "content" => array(
                    [
                        "type" => "text",
                        "text" => "What's the price of a scarf?",
                    ],
                ),
            ],
        ),
        "tools" => array(
            [
                "type" => "function",
                "function" => [
                    "name" => "get_current_price",
                    "description" => "Get the current price of a item",
                    "parameters" => [
                        "type" => "object",
                        "properties" => [
                            "item" => [
                                "id" => "123",
                            ],
                        ],
                    ],
                ],
            ],
        ),
        "tool_choice" => [
            "type" => "function",
            "function" => [
                "name" => "get_current_price",
            ],
        ],
    ],
]);
```

#### Ruby

```ruby
response = client.inference.chat_completion_unified(
  inference_id: "openai-completion",
  body: {
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What's the price of a scarf?"
          }
        ]
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_current_price",
          "description": "Get the current price of a item",
          "parameters": {
            "type": "object",
            "properties": {
              "item": {
                "id": "123"
              }
            }
          }
        }
      }
    ],
    "tool_choice": {
      "type": "function",
      "function": {
        "name": "get_current_price"
      }
    }
  }
)
```

## Response examples

#### Post chat completion response example1

`application/json`

A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`.

```json
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

(...)

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}} 

event: message
data: [DONE]
```