---
type: api
title: Perform chat completion inference
description: The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation.
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-inference-chat-completion-unified
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-inference-chat-completion-unified
products:
  - Elasticsearch
---

# Perform chat completion inference


Availability: 8.18+

## Paths

- `POST` `/_inference/chat_completion/{inference_id}/_stream`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#apikeyauth)
- [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#basicauth)
- [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#bearerauth)

#### Path Parameters

- `inference_id` (string Id)
  The inference Id

## Description

The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation.
It only works with the `chat_completion` task type.

NOTE: The `chat_completion` task type is only available within the _stream API and only supports streaming.
The Chat completion inference API and the Stream inference API differ in their response structure and capabilities.
The Chat completion inference API provides more comprehensive customization options through more fields and function calling support.
To determine whether a given inference service supports this task type, please see the page for that service.

## Query String Parameters

- `timeout` (string Duration · default: 30s)
  Specifies the amount of time to wait for the inference request to complete.

## Request Body

`application/json`

- `messages` ([] Message) — required
  A list of objects representing the conversation.
Requests should generally only add new messages from the user (role `user`).
The other message roles (`assistant`, `system`, or `tool`) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.
  - `content` (MessageContent) — optional
    The content of the message.

String example:
```
{
   "content": "Some string"
}
```

Object example:
```
{
  "content": [
      {
       "text": "Some text",
       "type": "text"
      }
   ]
}
```
  - `role` (string) — required
    The role of the message author. Valid values are `user`, `assistant`, `system`, and `tool`.
  - `tool_call_id` (string Id) — optional
    Only for `tool` role messages. The tool call that this message is responding to.
  - `tool_calls` ([] ToolCall) — optional
    Only for `assistant` role messages. The tool calls generated by the model. If it's specified, the `content` field is optional.
Example:
```
{
  "tool_calls": [
      {
          "id": "call_KcAjWtAww20AihPHphUh46Gd",
          "type": "function",
          "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Boston, MA\"}"
          }
      }
  ]
}
```
    - `id` (string Id) — required
      The identifier of the tool call.
    - `function` ({} ToolCallFunction) — required
      The function that the model called.
      - `arguments` (string) — required
        The arguments to call the function with in JSON format.
      - `name` (string) — required
        The name of the function to call.
    - `type` (string) — required
      The type of the tool call.
- `model` (string) — optional
  The ID of the model to use.
- `max_completion_tokens` (number) — optional
  The upper bound limit for the number of tokens that can be generated for a completion request.
- `stop` ([] string) — optional
  A sequence of strings to control when the model should stop generating additional tokens.
- `temperature` (number) — optional
  The sampling temperature to use.
- `tool_choice` (CompletionToolType) — optional
  Controls which tool is called by the model.
String representation: One of `auto`, `none`, or `requrired`. `auto` allows the model to choose between calling tools and generating a message. `none` causes the model to not call any tools. `required` forces the model to call one or more tools.
Example (object representation):
```
{
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_weather"
      }
  }
}
```
- `tools` ([] CompletionTool) — optional
  A list of tools that the model can call.
Example:
```
{
  "tools": [
      {
          "type": "function",
          "function": {
              "name": "get_price_of_item",
              "description": "Get the current price of an item",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "item": {
                          "id": "12345"
                      },
                      "unit": {
                          "type": "currency"
                      }
                  }
              }
          }
      }
  ]
}
```
  - `type` (string) — required
    The type of tool.
  - `function` ({} CompletionToolFunction) — required
    The function definition.
    - `description` (string) — optional
      A description of what the function does.
This is used by the model to choose when and how to call the function.
    - `name` (string) — required
      The name of the function.
    - `parameters` (object) — optional
      The parameters the functional accepts. This should be formatted as a JSON object.
    - `strict` (boolean) — optional
      Whether to enable schema adherence when generating the function call.
- `top_p` (number) — optional
  Nucleus sampling, an alternative to sampling with temperature.

## Response

`application/json`

Type: `StreamResult`

## Code Examples

#### Console

```console
POST _inference/chat_completion/openai-completion/_stream
{
  "model": "gpt-4o",
  "messages": [
      {
          "role": "user",
          "content": "What is Elastic?"
      }
  ]
}
```

#### JavaScript

```javascript
const response = await client.inference.chatCompletionUnified({
  inference_id: "openai-completion",
  chat_completion_request: {
    model: "gpt-4o",
    messages: [
      {
        role: "user",
        content: "What is Elastic?",
      },
    ],
  },
});
```

#### Python

```python
resp = client.inference.chat_completion_unified(
    inference_id="openai-completion",
    chat_completion_request={
        "model": "gpt-4o",
        "messages": [
            {
                "role": "user",
                "content": "What is Elastic?"
            }
        ]
    },
)
```

#### curl

```curl
curl -X POST "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion/_stream" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is Elastic?"
      }
    ]
  }'
```

#### Java

```java
client.inference().chatCompletionUnified(c -> c
    .inferenceId("openai-completion")
    .chatCompletionRequest(ch -> ch
        .messages(m -> m
            .content(co -> co
                .string("What is Elastic?")
            )
            .role("user")
        )
        .model("gpt-4o")
    )
);
```

#### C#

```c#
var response = await client.Inference
    .ChatCompletionUnifiedAsync(inferenceId: "openai-completion", d1 => d1
        .ChatCompletionRequest(d2 => d2
            .Messages(d3 => d3
                .Content("What is Elastic?")
                .Role("user")
            )
            .Model("gpt-4o")
        )
    );
```

#### PHP

```php
$resp = $client->inference()->chatCompletionUnified([
    "inference_id" => "openai-completion",
    "body" => [
        "model" => "gpt-4o",
        "messages" => array(
            [
                "role" => "user",
                "content" => "What is Elastic?",
            ],
        ),
    ],
]);
```

#### Ruby

```ruby
response = client.inference.chat_completion_unified(
  inference_id: "openai-completion",
  body: {
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "What is Elastic?"
      }
    ]
  }
)
```

## Response Examples

#### Post chat completion response example1

A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`.

```json
event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}}

(...)

event: message
data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}} 

event: message
data: [DONE]
```