--- type: api title: Perform chat completion inference on the service description: The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/operation/operation-inference-chat-completion-unified resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/operation/operation-inference-chat-completion-unified products: - Elasticsearch --- # Perform chat completion inference on the service Availability: 8.18+ ## Paths - `POST` `/_inference/chat_completion/{inference_id}/_stream` - [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/authentication#apikeyauth) - [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/authentication#basicauth) - [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v9/authentication#bearerauth) #### Path Parameters - `inference_id` (string Id) The inference Id ## Description The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. It only works with the `chat_completion` task type. NOTE: The `chat_completion` task type is only available within the _stream API and only supports streaming. The Chat completion inference API and the Stream inference API differ in their response structure and capabilities. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. To determine whether a given inference service supports this task type, please see the page for that service. ## Query String Parameters - `timeout` (string Duration · default: 120s) Specifies the amount of time to wait for the inference request to complete. ## Request Body `application/json` - `messages` ([] Message) — required A list of objects representing the conversation. Requests should generally only add new messages from the user (role `user`). The other message roles (`assistant`, `system`, or `tool`) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation. - `content` (MessageContent) — optional The content of the message. String example: ``` { "content": "Some string" } ``` Text example: ``` { "content": [ { "text": "Some text", "type": "text" } ] } ``` Image example: ``` { "content": [ { "image_url": { "url": "data:image/jpeg;base64,..." }, "type": "image_url" } ] } ``` File example: ``` { "content": [ { "file": { "file_data": "data:application/pdf;base64,...", "filename": "somePDF" }, "type": "file" } ] } ``` - `role` (string) — required The role of the message author. Valid values are `user`, `assistant`, `system`, and `tool`. - `tool_call_id` (string Id) — optional Only for `tool` role messages. The tool call that this message is responding to. - `tool_calls` ([] ToolCall) — optional Only for `assistant` role messages. The tool calls generated by the model. If it's specified, the `content` field is optional. Example: ``` { "tool_calls": [ { "id": "call_KcAjWtAww20AihPHphUh46Gd", "type": "function", "function": { "name": "get_current_weather", "arguments": "{\"location\":\"Boston, MA\"}" } } ] } ``` - `id` (string Id) — required The identifier of the tool call. - `function` ({} ToolCallFunction) — required The function that the model called. - `arguments` (string) — required The arguments to call the function with in JSON format. - `name` (string) — required The name of the function to call. - `type` (string) — required The type of the tool call. - `reasoning` (string) — optional Only for `assistant` role messages. The reasoning details generated by the model as plaintext. Currently supported only for `elastic` provider. - `reasoning_details` ([] union ReasoningDetail) — optional Only for `assistant` role messages. The reasoning details generated by the model as structured data. Currently supported only for `elastic` provider. - **EncryptedReasoningDetail** The reasoning detail with encrypted reasoning data that may be redacted or protected. - `format` (string) — optional The format of the reasoning detail. - `id` (string) — optional The identifier of the reasoning detail. - `index` (number) — optional The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model. - `type` (enum) — required Values: `reasoning.encrypted` - `data` (string) — required The encrypted reasoning data generated by the model, which may be redacted or protected based on the model's configuration and the nature of the reasoning information. - **SummaryReasoningDetail** The reasoning summary detail includes a high-level summary of the model's reasoning process. - `format` (string) — optional The format of the reasoning detail. - `id` (string) — optional The identifier of the reasoning detail. - `index` (number) — optional The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model. - `type` (enum) — required Values: `reasoning.summary` - `summary` (string) — required The summary of the reasoning process generated by the model, which provides an overview of the key points and conclusions reached during the reasoning process. - **TextReasoningDetail** The reasoning text detail includes plaintext reasoning with optional signature verification. - `format` (string) — optional The format of the reasoning detail. - `id` (string) — optional The identifier of the reasoning detail. - `index` (number) — optional The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model. - `type` (enum) — required Values: `reasoning.text` - `signature` (string) — optional The signature of the reasoning text, which can be used to verify the authenticity and integrity of the reasoning information provided by the model. - `text` (string) — optional The plaintext reasoning generated by the model, which provides a detailed explanation of the model's reasoning process in human-readable form. - `model` (string) — optional The ID of the model to use. By default, the model ID is set to the value included when creating the inference endpoint. - `max_completion_tokens` (number) — optional The upper bound limit for the number of tokens that can be generated for a completion request. - `reasoning` ({} Reasoning) — optional The reasoning configuration for the completion request. This controls the model's reasoning process in one of two ways: * By specifying the model’s reasoning effort level with the `effort` field. * By enabling reasoning with default settings by setting `enabled` field to `true`. It also includes optional settings to control: * The level of detail in the summary returned in the response with the `summary` field. * Whether reasoning details are included in the response at all with the `exclude` field. Example (effort): ``` { "reasoning": { "effort": "high", "summary": "concise", "exclude": false } } ``` Example (enabled): ``` { "reasoning": { "enabled": true, "summary": "concise", "exclude": false } } ``` Currently supported only for `elastic` provider. - `effort` (enum ReasoningEffort) — optional The level of effort the model should put into reasoning. This is a hint that guides the model in how much effort to put into reasoning, with `xhigh` being the most effort and `none` being no effort. Values: `xhigh`, `high`, `medium`, `low`, `minimal`, `none` - `enabled` (boolean) — optional Whether to enable reasoning with default settings. This is a shortcut for enabling reasoning without having to specify the other parameters. If `enabled` is set to `true`, then reasoning at the `medium` effort level is enabled. Ignored if `effort` is specified, in which case that parameter will control the reasoning process instead. - `exclude` (boolean) — optional Whether to exclude reasoning information from the response. If `true`, the response will not include any reasoning details. - `summary` (enum ReasoningSummary) — optional The level of detail included in the reasoning summary returned in the response. This is a hint on how much detail to include in the summary of the reasoning that is returned in the response, with `auto` being the default level of detail, `concise` being less detail, and `detailed` being more detail. Values: `auto`, `concise`, `detailed` - `stop` ([] string) — optional A sequence of strings to control when the model should stop generating additional tokens. - `temperature` (number) — optional The sampling temperature to use. - `tool_choice` (CompletionToolType) — optional Controls which tool is called by the model. String representation: One of `auto`, `none`, or `requrired`. `auto` allows the model to choose between calling tools and generating a message. `none` causes the model to not call any tools. `required` forces the model to call one or more tools. Example (object representation): ``` { "tool_choice": { "type": "function", "function": { "name": "get_current_weather" } } } ``` - `tools` ([] CompletionTool) — optional A list of tools that the model can call. Example: ``` { "tools": [ { "type": "function", "function": { "name": "get_price_of_item", "description": "Get the current price of an item", "parameters": { "type": "object", "properties": { "item": { "id": "12345" }, "unit": { "type": "currency" } } } } } ] } ``` - `type` (string) — required The type of tool. - `function` ({} CompletionToolFunction) — required The function definition. - `description` (string) — optional A description of what the function does. This is used by the model to choose when and how to call the function. - `name` (string) — required The name of the function. - `parameters` (object) — optional The parameters the functional accepts. This should be formatted as a JSON object. - `strict` (boolean) — optional Whether to enable schema adherence when generating the function call. - `top_p` (number) — optional Nucleus sampling, an alternative to sampling with temperature. ## Response `application/json` Type: `StreamResult` ## Code Examples #### Console ```console POST _inference/chat_completion/openai-completion/_stream { "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] } ``` #### JavaScript ```javascript const response = await client.inference.chatCompletionUnified({ inference_id: "openai-completion", chat_completion_request: { model: "gpt-4o", messages: [ { role: "user", content: "What is Elastic?", }, ], }, }); ``` #### Python ```python resp = client.inference.chat_completion_unified( inference_id="openai-completion", chat_completion_request={ "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion/_stream" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] }' ``` #### Java ```java client.inference().chatCompletionUnified(c -> c .inferenceId("openai-completion") .chatCompletionRequest(ch -> ch .messages(m -> m .content(co -> co .string("What is Elastic?") ) .role("user") ) .model("gpt-4o") ) ); ``` #### C# ```c# var response = await client.Inference .ChatCompletionUnifiedAsync(inferenceId: "openai-completion", d1 => d1 .ChatCompletionRequest(d2 => d2 .Messages(d3 => d3 .Content("What is Elastic?") .Role("user") ) .Model("gpt-4o") ) ); ``` #### PHP ```php $resp = $client->inference()->chatCompletionUnified([ "inference_id" => "openai-completion", "body" => [ "model" => "gpt-4o", "messages" => array( [ "role" => "user", "content" => "What is Elastic?", ], ), ], ]); ``` #### Ruby ```ruby response = client.inference.chat_completion_unified( inference_id: "openai-completion", body: { "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] } ) ``` ## Response Examples #### Post chat completion response example1 A successful response when performing a chat completion task using a User message with `tools` and `tool_choice`. ```json event: message data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}} event: message data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":Elastic"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}} event: message data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[{"delta":{"content":" is"},"index":0}],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk"}} (...) event: message data: {"chat_completion":{"id":"chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3","choices":[],"model":"gpt-4o-2024-08-06","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44}}} event: message data: [DONE] ``` #### Post chat completion response example2 A successful response when performing a chat completion task with response-level reasoning data. ```json event: message data: {"chat_completion":{"id":"chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10","choices":[{"delta":{"content":"With two barbers, the paradox disappears.","role":"assistant"},"index":0,"reasoning":"The contradiction only occurs when a barber must determine whether to shave himself.","reasoning_details":[{"type":"reasoning.encrypted","data":"[REDACTED]"},{"type":"reasoning.summary","summary":"Two barbers can shave each other."},{"type":"reasoning.text","text":"The contradiction only occurs when a barber must determine whether to shave himself.","signature":"sig_example"}]}],"model":"openai-gpt-oss-120b","object":"chat.completion.chunk"}} event: message data: {"chat_completion":{"id":"chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10","choices":[{"delta":{"summary":"Each barber can shave the other, so neither needs to shave himself.","role":"assistant"},"index":0,"reasoning":"With two barbers, they can shave each other.","reasoning_details":[{"type":"reasoning.encrypted","data":"[REDACTED]"},{"type":"reasoning.summary","summary":"avoiding the self-reference paradox"},{"type":"reasoning.text","format":"some_text_reasoning_detail_format","text":"With two barbers, they can shave each other.","signature":"sig_example"}]}],"model":"openai-gpt-oss-120b","object":"chat.completion.chunk"}} (...) event: message data: {"chat_completion":{"id":"chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10","choices":[],"model":"openai-gpt-oss-120b","object":"chat.completion.chunk","usage":{"completion_tokens":28,"prompt_tokens":16,"total_tokens":44,"completion_tokens_details":{"reasoning_tokens":10}}}} event: message data: [DONE] ```