Create a Hugging Face inference endpoint

PUT /_inference/{task_type}/{huggingface_inference_id}

Create an inference endpoint to perform an inference task with the hugging_face service. Supported tasks include: text_embedding, completion, and chat_completion.

To configure the endpoint, first visit the Hugging Face Inference Endpoints page and create a new endpoint. Select a model that supports the task you intend to use.

For Elastic's text_embedding task: The selected model must support the Sentence Embeddings task. On the new endpoint creation page, select the Sentence Embeddings task under the Advanced Configuration section. After the endpoint has initialized, copy the generated endpoint URL. Recommended models for text_embedding task:

all-MiniLM-L6-v2
all-MiniLM-L12-v2
all-mpnet-base-v2
e5-base-v2
e5-small-v2
multilingual-e5-base
multilingual-e5-small

For Elastic's chat_completion and completion tasks: The selected model must support the Text Generation task and expose OpenAI API. HuggingFace supports both serverless and dedicated endpoints for Text Generation. When creating dedicated endpoint select the Text Generation task. After the endpoint is initialized (for dedicated) or ready (for serverless), ensure it supports the OpenAI API and includes /v1/chat/completions part in URL. Then, copy the full endpoint URL for use. Recommended models for chat_completion and completion tasks:

Mistral-7B-Instruct-v0.2
QwQ-32B
Phi-3-mini-128k-instruct

For Elastic's rerank task: The selected model must support the sentence-ranking task and expose OpenAI API. HuggingFace supports only dedicated (not serverless) endpoints for Rerank so far. After the endpoint is initialized, copy the full endpoint URL for use. Tested models for rerank task:

bge-reranker-base
jina-reranker-v1-turbo-en-GGUF

Required authorization

Cluster privileges: manage_inference

Path Parameters

Query Parameters

timeout string Duration · default: 30s: Specifies the amount of time to wait for the inference endpoint to be created.

Request

Response

application/json

Examples

Run PUT _inference/text_embedding/hugging-face-embeddings to create an inference endpoint that performs a text_embedding task type.

Request Console 1 / 8 languages

Console Kibana Dev Tools

PUT _inference/text_embedding/hugging-face-embeddings
{
    "service": "hugging_face",
    "service_settings": {
        "api_key": "hugging-face-access-token", 
        "url": "url-endpoint" 
    }
}

JavaScript Elasticsearch JavaScript Client

const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "hugging-face-embeddings",
  inference_config: {
    service: "hugging_face",
    service_settings: {
      api_key: "hugging-face-access-token",
      url: "url-endpoint",
    },
  },
});

Python Elasticsearch Python Client

resp = client.inference.put(
    task_type="text_embedding",
    inference_id="hugging-face-embeddings",
    inference_config={
        "service": "hugging_face",
        "service_settings": {
            "api_key": "hugging-face-access-token",
            "url": "url-endpoint"
        }
    },
)

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_inference/text_embedding/hugging-face-embeddings" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "hugging_face",
    "service_settings": {
      "api_key": "hugging-face-access-token",
      "url": "url-endpoint"
    }
  }'

Java Elasticsearch Java Client

client.inference().put(p -> p
    .inferenceId("hugging-face-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("hugging_face")
        .serviceSettings(JsonData.fromJson("""
{"api_key":"hugging-face-access-token","url":"url-endpoint"}
"""))
    )
);

C# Elasticsearch .NET Client

var response = await client.Inference
    .PutAsync(new PutInferenceRequest()
    {
        InferenceId = "hugging-face-embeddings",
        TaskType = TaskType.TextEmbedding,
        InferenceConfig = new()
        {
            Service = "hugging_face",
            ServiceSettings = JsonSerializer.Deserialize<JsonElement>("""
            {
              "api_key": "hugging-face-access-token",
              "url": "url-endpoint"
            }
            """)
        }
    });

PHP Elasticsearch PHP Client

$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "hugging-face-embeddings",
    "body" => [
        "service" => "hugging_face",
        "service_settings" => [
            "api_key" => "hugging-face-access-token",
            "url" => "url-endpoint",
        ],
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "hugging-face-embeddings",
  body: {
    "service": "hugging_face",
    "service_settings": {
      "api_key": "hugging-face-access-token",
      "url": "url-endpoint"
    }
  }
)

Response

No example

Run PUT _inference/rerank/hugging-face-rerank to create an inference endpoint that performs a rerank task type.

Request Console 1 / 2 languages

Console Kibana Dev Tools

PUT _inference/rerank/hugging-face-rerank
{
    "service": "hugging_face",
    "service_settings": {
        "api_key": "hugging-face-access-token", 
        "url": "url-endpoint" 
    },
    "task_settings": {
        "return_documents": true,
        "top_n": 3
    }
}

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_inference/rerank/hugging-face-rerank" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "hugging_face",
    "service_settings": {
        "api_key": "hugging-face-access-token", 
        "url": "url-endpoint" 
    },
    "task_settings": {
        "return_documents": true,
        "top_n": 3
    }
}'

Response

No example

Required authorization

Path Parameters { task_type , huggingface_inference_id , }

Query Parameters

Request { chunking_settings , service , service_settings , task_settings , }

Response

Path Parameters

Request