--- type: api title: Perform non-streaming chat completion inference on the service description: The chat completion inference API enables rich responses for chat completion tasks. url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch-serverless/operation/operation-inference-non-streaming-chat-completion resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch-serverless/operation/operation-inference-non-streaming-chat-completion products: - Elasticsearch Serverless --- # Perform non-streaming chat completion inference on the service Availability: ## Paths - `POST` `/_inference/chat_completion/{inference_id}` - [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch-serverless/authentication#apikeyauth) #### Path Parameters - `inference_id` (string Id) The inference Id ## Description The chat completion inference API enables rich responses for chat completion tasks. It only works with the `chat_completion` task type. NOTE: The `chat_completion` task type supports both streaming and non-streaming. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. To determine whether a given inference service supports this task type, please see the page for that service. These services support non-streaming chat completion inference: - AI21 - Azure OpenAI - Deepseek - Elastic - FireworksAI - Groq - Huggingface - IBMWatsonX - Llama - Mistral - NVIDIA - OpenAI - OpenShiftAI ## Query String Parameters - `timeout` (string Duration · default: 120s) Specifies the amount of time to wait for the inference request to complete. ## Request Body `application/json` - `messages` ([] Message) — required A list of objects representing the conversation. Requests should generally only add new messages from the user (role `user`). The other message roles (`assistant`, `system`, or `tool`) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation. - `content` (MessageContent) — optional The content of the message. String example: ``` { "content": "Some string" } ``` Text example: ``` { "content": [ { "text": "Some text", "type": "text" } ] } ``` Image example: ``` { "content": [ { "image_url": { "url": "data:image/jpeg;base64,..." }, "type": "image_url" } ] } ``` File example: ``` { "content": [ { "file": { "file_data": "data:application/pdf;base64,...", "filename": "somePDF" }, "type": "file" } ] } ``` - `role` (string) — required The role of the message author. Valid values are `user`, `assistant`, `system`, and `tool`. - `tool_call_id` (string Id) — optional Only for `tool` role messages. The tool call that this message is responding to. - `tool_calls` ([] ToolCall) — optional Only for `assistant` role messages. The tool calls generated by the model. If it's specified, the `content` field is optional. Example: ``` { "tool_calls": [ { "id": "call_KcAjWtAww20AihPHphUh46Gd", "type": "function", "function": { "name": "get_current_weather", "arguments": "{\"location\":\"Boston, MA\"}" } } ] } ``` - `id` (string Id) — required The identifier of the tool call. - `function` ({} ToolCallFunction) — required The function that the model called. - `arguments` (string) — required The arguments to call the function with in JSON format. - `name` (string) — required The name of the function to call. - `type` (string) — required The type of the tool call. - `reasoning` (string) — optional Only for `assistant` role messages. The reasoning details generated by the model as plaintext. Currently supported only for `elastic` provider. - `reasoning_details` ([] union ReasoningDetail) — optional Only for `assistant` role messages. The reasoning details generated by the model as structured data. Currently supported only for `elastic` provider. - **EncryptedReasoningDetail** The reasoning detail with encrypted reasoning data that may be redacted or protected. - `format` (string) — optional The format of the reasoning detail. - `id` (string) — optional The identifier of the reasoning detail. - `index` (number) — optional The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model. - `type` (enum) — required Values: `reasoning.encrypted` - `data` (string) — required The encrypted reasoning data generated by the model, which may be redacted or protected based on the model's configuration and the nature of the reasoning information. - **SummaryReasoningDetail** The reasoning summary detail includes a high-level summary of the model's reasoning process. - `format` (string) — optional The format of the reasoning detail. - `id` (string) — optional The identifier of the reasoning detail. - `index` (number) — optional The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model. - `type` (enum) — required Values: `reasoning.summary` - `summary` (string) — required The summary of the reasoning process generated by the model, which provides an overview of the key points and conclusions reached during the reasoning process. - **TextReasoningDetail** The reasoning text detail includes plaintext reasoning with optional signature verification. - `format` (string) — optional The format of the reasoning detail. - `id` (string) — optional The identifier of the reasoning detail. - `index` (number) — optional The index of the reasoning detail, which indicates its position in the sequence of reasoning details generated by the model. - `type` (enum) — required Values: `reasoning.text` - `signature` (string) — optional The signature of the reasoning text, which can be used to verify the authenticity and integrity of the reasoning information provided by the model. - `text` (string) — optional The plaintext reasoning generated by the model, which provides a detailed explanation of the model's reasoning process in human-readable form. - `model` (string) — optional The ID of the model to use. By default, the model ID is set to the value included when creating the inference endpoint. - `max_completion_tokens` (number) — optional The upper bound limit for the number of tokens that can be generated for a completion request. - `reasoning` ({} Reasoning) — optional The reasoning configuration for the completion request. This controls the model's reasoning process in one of two ways: * By specifying the model’s reasoning effort level with the `effort` field. * By enabling reasoning with default settings by setting `enabled` field to `true`. It also includes optional settings to control: * The level of detail in the summary returned in the response with the `summary` field. * Whether reasoning details are included in the response at all with the `exclude` field. Example (effort): ``` { "reasoning": { "effort": "high", "summary": "concise", "exclude": false } } ``` Example (enabled): ``` { "reasoning": { "enabled": true, "summary": "concise", "exclude": false } } ``` Currently supported only for `elastic` provider. - `effort` (enum ReasoningEffort) — optional The level of effort the model should put into reasoning. This is a hint that guides the model in how much effort to put into reasoning, with `xhigh` being the most effort and `none` being no effort. Values: `xhigh`, `high`, `medium`, `low`, `minimal`, `none` - `enabled` (boolean) — optional Whether to enable reasoning with default settings. This is a shortcut for enabling reasoning without having to specify the other parameters. If `enabled` is set to `true`, then reasoning at the `medium` effort level is enabled. Ignored if `effort` is specified, in which case that parameter will control the reasoning process instead. - `exclude` (boolean) — optional Whether to exclude reasoning information from the response. If `true`, the response will not include any reasoning details. - `summary` (enum ReasoningSummary) — optional The level of detail included in the reasoning summary returned in the response. This is a hint on how much detail to include in the summary of the reasoning that is returned in the response, with `auto` being the default level of detail, `concise` being less detail, and `detailed` being more detail. Values: `auto`, `concise`, `detailed` - `stop` ([] string) — optional A sequence of strings to control when the model should stop generating additional tokens. - `temperature` (number) — optional The sampling temperature to use. - `tool_choice` (CompletionToolType) — optional Controls which tool is called by the model. String representation: One of `auto`, `none`, or `requrired`. `auto` allows the model to choose between calling tools and generating a message. `none` causes the model to not call any tools. `required` forces the model to call one or more tools. Example (object representation): ``` { "tool_choice": { "type": "function", "function": { "name": "get_current_weather" } } } ``` - `tools` ([] CompletionTool) — optional A list of tools that the model can call. Example: ``` { "tools": [ { "type": "function", "function": { "name": "get_price_of_item", "description": "Get the current price of an item", "parameters": { "type": "object", "properties": { "item": { "id": "12345" }, "unit": { "type": "currency" } } } } } ] } ``` - `type` (string) — required The type of tool. - `function` ({} CompletionToolFunction) — required The function definition. - `description` (string) — optional A description of what the function does. This is used by the model to choose when and how to call the function. - `name` (string) — required The name of the function. - `parameters` (object) — optional The parameters the functional accepts. This should be formatted as a JSON object. - `strict` (boolean) — optional Whether to enable schema adherence when generating the function call. - `top_p` (number) — optional Nucleus sampling, an alternative to sampling with temperature. ## Response `application/json` - `id` (string) The unique identifier for the completion. - `choices` ([] ChatCompletionChoice) The list of completion choices the model generated for the input message. - `message` ({} ChatCompletionMessage) The message generated by the model for this choice. - `content` (string) The content of the message. - `refusal` (string) The refusal message generated by the model. - `role` (string) The role of the message author. - `reasoning` (string) The reasoning generated by the model as plaintext. Currently supported only for the `elastic` provider. - `tool_calls` ([] ChatCompletionToolCall) The tool calls generated by the model. - `index` (number) The index of the tool call in the list of tool calls generated by the model. - `id` (string) The identifier of the tool call. - `function` ({} ChatCompletionToolCallFunction) The function that the model called. - `arguments` (string) The arguments to call the function with, as a JSON string. - `name` (string) The name of the function. - `type` (string) The type of the tool call. - `reasoning_details` ([] union ReasoningDetail) The reasoning details generated by the model as structured data. Currently supported only for the `elastic` provider. Same options as `reasoning_details` in `messages` - `finish_reason` (string) The reason the model stopped generating tokens. Common values are `stop` (natural stopping point) and `tool_calls` (the model called a tool). Omitted when the reason is not available. - `index` (number) The index of this choice in the list of choices. - `model` (string) The model used to generate the completion. - `object` (string) The object type. - `usage` ({} ChatCompletionUsage) The token usage statistics for the completion request. Omitted when usage information is not available. - `completion_tokens` (number) The number of tokens in the generated completion. - `prompt_tokens` (number) The number of tokens in the prompt. - `total_tokens` (number) The total number of tokens used (prompt + completion). - `prompt_tokens_details` ({} ChatCompletionPromptTokensDetails) Breakdown of the tokens used in the prompt. Omitted when no details are available. - `cached_tokens` (number) The number of tokens that were cached from a previous request. - `cache_write_tokens` (number) The number of tokens written to the cache. - `completion_tokens_details` ({} ChatCompletionCompletionTokensDetails) Breakdown of the tokens used in the completion. Omitted when no details are available. - `reasoning_tokens` (number) The number of tokens used for reasoning by the model. ## Code Examples #### Console ```console POST _inference/chat_completion/openai-completion { "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] } ``` #### JavaScript ```javascript const response = await client.inference.nonStreamingChatCompletion({ inference_id: "openai-completion", chat_completion_request: { model: "gpt-4o", messages: [ { role: "user", content: "What is Elastic?", }, ], }, }); ``` #### Python ```python resp = client.inference.non_streaming_chat_completion( inference_id="openai-completion", chat_completion_request={ "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_inference/chat_completion/openai-completion" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] }' ``` #### PHP ```php $resp = $client->inference()->nonStreamingChatCompletion([ "inference_id" => "openai-completion", "body" => [ "model" => "gpt-4o", "messages" => array( [ "role" => "user", "content" => "What is Elastic?", ], ), ], ]); ``` #### Ruby ```ruby response = client.inference.non_streaming_chat_completion( inference_id: "openai-completion", body: { "model": "gpt-4o", "messages": [ { "role": "user", "content": "What is Elastic?" } ] } ) ``` ## Response Examples #### A non-streaming chat completion response A successful response from `POST _inference/chat_completion/openai-completion` for a plain user message. The `object` field is `chat.completion` (as opposed to `chat.completion.chunk` in streaming responses). The `message` wrapper contains the model-generated text. Usage statistics include per-category token breakdowns when available. ```json { "id": "chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3", "choices": [ { "message": { "content": "Elastic is a company that provides a range of software solutions for search, logging, security, and analytics, built around the open-source Elasticsearch engine.", "role": "assistant" }, "finish_reason": "stop", "index": 0 } ], "model": "gpt-4o-2024-08-06", "object": "chat.completion", "usage": { "completion_tokens": 34, "prompt_tokens": 14, "total_tokens": 48, "prompt_tokens_details": { "cached_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0 } } } ``` #### A non-streaming chat completion response with tool calls A successful response from `POST _inference/chat_completion/openai-completion` when the model decides to call a tool. The `message` contains `tool_calls` instead of `content`, and `finish_reason` is `tool_calls`. Each tool call includes an `index`, an `id`, the function `name` and `arguments` (as a JSON string), and a `type`. ```json { "id": "chatcmpl-Ae0TWsy2VPnSfBbv5UztnSdYUMFP3", "choices": [ { "message": { "role": "assistant", "tool_calls": [ { "index": 0, "id": "call_KcAjWtAww20AihPHphUh46Gd", "function": { "arguments": "{\"item\":\"scarf\"}", "name": "get_current_price" }, "type": "function" } ] }, "finish_reason": "tool_calls", "index": 0 } ], "model": "gpt-4o-2024-08-06", "object": "chat.completion", "usage": { "completion_tokens": 19, "prompt_tokens": 135, "total_tokens": 154, "prompt_tokens_details": { "cached_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0 } } } ``` #### A non-streaming chat completion response with reasoning details A successful response from `POST _inference/chat_completion/reasoning-chat-completion` when the model returns structured reasoning data alongside its answer. The `message` includes a `reasoning` plaintext summary and a `reasoning_details` array covering all three detail types: `reasoning.encrypted` (opaque data), `reasoning.summary` (human-readable summary), and `reasoning.text` (full reasoning text with a verification signature). Currently supported only for the `elastic` provider. ```json { "id": "chatcmpl-910TWsy2VPnSfBbv5UztnSdYUJA10", "choices": [ { "message": { "content": "With two barbers, the paradox disappears — each barber can shave the other.", "role": "assistant", "reasoning": "The contradiction only arises when a single barber must decide whether to shave himself. Two barbers eliminate the self-reference.", "reasoning_details": [ { "type": "reasoning.encrypted", "data": "[REDACTED]" }, { "type": "reasoning.summary", "summary": "Two barbers can shave each other, avoiding the self-reference paradox." }, { "type": "reasoning.text", "text": "The contradiction only arises when a single barber must decide whether to shave himself. Two barbers eliminate the self-reference.", "signature": "sig_example" } ] }, "finish_reason": "stop", "index": 0 } ], "model": "openai-gpt-oss-120b", "object": "chat.completion", "usage": { "completion_tokens": 38, "prompt_tokens": 74, "total_tokens": 112, "completion_tokens_details": { "reasoning_tokens": 10 } } } ```