Create an inference endpoint

PUT /_inference/{task_type}/{inference_id}
PUT /_inference/{inference_id}

IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Mistral, Azure OpenAI, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.

The following integrations are available through the inference API. You can find the available task types next to the integration name:

AlibabaCloud AI Search (completion, rerank, sparse_embedding, text_embedding)
Amazon Bedrock (completion, text_embedding)
Amazon SageMaker (chat_completion, completion, rerank, sparse_embedding, text_embedding)
Anthropic (completion)
Azure AI Studio (completion, text_embedding)
Azure OpenAI (completion, text_embedding)
Cohere (completion, rerank, text_embedding)
DeepSeek (chat_completion, completion)
Elasticsearch (rerank, sparse_embedding, text_embedding - this service is for built-in models and models uploaded through Eland)
ELSER (sparse_embedding)
Google AI Studio (completion, text_embedding)
Google Vertex AI (chat_completion, completion, rerank, text_embedding)
Hugging Face (chat_completion, completion, rerank, text_embedding)
JinaAI (rerank, text_embedding)
Llama (chat_completion, completion, text_embedding)
Mistral (chat_completion, completion, text_embedding)
OpenAI (chat_completion, completion, text_embedding)
VoyageAI (rerank, text_embedding)
Watsonx (rerank, text_embedding)

Required authorization

Cluster privileges: manage_inference

Path Parameters

Query Parameters

timeout string Duration · default: 30s: Specifies the amount of time to wait for the inference endpoint to be created.

Request

Response

application/json

Examples

An example body for a PUT _inference/rerank/my-rerank-model request.

Request Console 1 / 8 languages

Console Kibana Dev Tools

PUT _inference/rerank/my-rerank-model
{
 "service": "cohere",
 "service_settings": {
   "model_id": "rerank-english-v3.0",
   "api_key": "{{COHERE_API_KEY}}"
 }
}

JavaScript Elasticsearch JavaScript Client

const response = await client.inference.put({
  task_type: "rerank",
  inference_id: "my-rerank-model",
  inference_config: {
    service: "cohere",
    service_settings: {
      model_id: "rerank-english-v3.0",
      api_key: "{{COHERE_API_KEY}}",
    },
  },
});

Python Elasticsearch Python Client

resp = client.inference.put(
    task_type="rerank",
    inference_id="my-rerank-model",
    inference_config={
        "service": "cohere",
        "service_settings": {
            "model_id": "rerank-english-v3.0",
            "api_key": "{{COHERE_API_KEY}}"
        }
    },
)

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_inference/rerank/my-rerank-model" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "cohere",
    "service_settings": {
      "model_id": "rerank-english-v3.0",
      "api_key": "{{COHERE_API_KEY}}"
    }
  }'

Java Elasticsearch Java Client

client.inference().put(p -> p
    .inferenceId("my-rerank-model")
    .taskType(TaskType.Rerank)
    .inferenceConfig(i -> i
        .service("cohere")
        .serviceSettings(JsonData.fromJson("{\"model_id\":\"rerank-english-v3.0\",\"api_key\":\"{{COHERE_API_KEY}}\"}"))
    )
);

C# Elasticsearch .NET Client

var response = await client.Inference
    .PutAsync(new PutInferenceRequest()
    {
        InferenceId = "my-rerank-model",
        TaskType = TaskType.Rerank,
        InferenceConfig = new()
        {
            Service = "cohere",
            ServiceSettings = JsonSerializer.Deserialize<JsonElement>("""
            {
              "model_id": "rerank-english-v3.0",
              "api_key": "{{COHERE_API_KEY}}"
            }
            """)
        }
    });

PHP Elasticsearch PHP Client

$resp = $client->inference()->put([
    "task_type" => "rerank",
    "inference_id" => "my-rerank-model",
    "body" => [
        "service" => "cohere",
        "service_settings" => [
            "model_id" => "rerank-english-v3.0",
            "api_key" => "{{COHERE_API_KEY}}",
        ],
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.inference.put(
  task_type: "rerank",
  inference_id: "my-rerank-model",
  body: {
    "service": "cohere",
    "service_settings": {
      "model_id": "rerank-english-v3.0",
      "api_key": "{{COHERE_API_KEY}}"
    }
  }
)

Response

No example

Required authorization

Path Parameters { task_type , inference_id , }

Query Parameters

Request { chunking_settings , service , service_settings , task_settings , }

Response

Path Parameters

Request