---
type: api
title: Create a Hugging Face inference endpoint
description: Create an inference endpoint to perform an inference task with the `hugging_face` service.
url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-inference-put-hugging-face
resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-inference-put-hugging-face
products:
  - Elasticsearch
---

# Create a Hugging Face inference endpoint

Availability: 8.12+

## Paths

- `PUT` `/_inference/{task_type}/{huggingface_inference_id}`

- [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#apikeyauth)
- [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#basicauth)
- [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#bearerauth)

#### Path Parameters

- `task_type` (enum HuggingFaceTaskType)
  The type of the inference task that the model will perform.
  Values: `chat_completion`, `completion`, `rerank`, `text_embedding`
- `huggingface_inference_id` (string Id)
  The unique identifier of the inference endpoint.

## Description

Create an inference endpoint to perform an inference task with the `hugging_face` service.
Supported tasks include: `text_embedding`, `completion`, and `chat_completion`.

To configure the endpoint, first visit the Hugging Face Inference Endpoints page and create a new endpoint.
Select a model that supports the task you intend to use.

For Elastic's `text_embedding` task:
The selected model must support the `Sentence Embeddings` task. On the new endpoint creation page, select the `Sentence Embeddings` task under the `Advanced Configuration` section.
After the endpoint has initialized, copy the generated endpoint URL.
Recommended models for `text_embedding` task:

* `all-MiniLM-L6-v2`
* `all-MiniLM-L12-v2`
* `all-mpnet-base-v2`
* `e5-base-v2`
* `e5-small-v2`
* `multilingual-e5-base`
* `multilingual-e5-small`

For Elastic's `chat_completion` and `completion` tasks:
The selected model must support the `Text Generation` task and expose OpenAI API. HuggingFace supports both serverless and dedicated endpoints for `Text Generation`. When creating dedicated endpoint select the `Text Generation` task.
After the endpoint is initialized (for dedicated) or ready (for serverless), ensure it supports the OpenAI API and includes `/v1/chat/completions` part in URL. Then, copy the full endpoint URL for use.
Recommended models for `chat_completion` and `completion` tasks:

* `Mistral-7B-Instruct-v0.2`
* `QwQ-32B`
* `Phi-3-mini-128k-instruct`

For Elastic's `rerank` task:
The selected model must support the `sentence-ranking` task and expose OpenAI API.
HuggingFace supports only dedicated (not serverless) endpoints for `Rerank` so far.
After the endpoint is initialized, copy the full endpoint URL for use.
Tested models for `rerank` task:

* `bge-reranker-base`
* `jina-reranker-v1-turbo-en-GGUF`

## Required authorization

* Cluster privileges: `manage_inference`

## Query String Parameters

- `timeout` (string Duration · default: 30s)
  Specifies the amount of time to wait for the inference endpoint to be created.

## Request Body

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings) — optional
  The chunking configuration object.
  - `max_chunk_size` (number · default: 250) — optional
    The maximum size of a chunk in words.
This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
  - `overlap` (number · default: 100) — optional
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1) — optional
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `strategy` (string · default: sentence) — optional
    The chunking strategy: `sentence` or `word`.
- `service` (enum HuggingFaceServiceType) — required
  The type of service supported for the specified task type. In this case, `hugging_face`.
  Values: `hugging_face`
- `service_settings` ({} HuggingFaceServiceSettings) — required
  Settings used to install the inference model. These settings are specific to the `hugging_face` service.
  - `api_key` (string) — required
    A valid access token for your HuggingFace account.
You can create or find your access tokens on the HuggingFace settings page.

IMPORTANT: You need to provide the API key only once, during the inference model creation.
The get inference endpoint API does not retrieve your API key.
  - `rate_limit` ({} RateLimitSetting) — optional
    This setting helps to minimize the number of rate limit errors returned from Hugging Face.
By default, the `hugging_face` service sets the number of requests allowed per minute to 3000 for all supported tasks.
Hugging Face does not publish a universal rate limit — actual limits may vary.
It is recommended to adjust this value based on the capacity and limits of your specific deployment environment.
    - `requests_per_minute` (number) — optional
      The number of requests allowed per minute.
  - `url` (string) — required
    The URL endpoint to use for the requests.
For `completion` and `chat_completion` tasks, the deployed model must be compatible with the Hugging Face Chat Completion interface (see the linked external documentation for details). The endpoint URL for the request must include `/v1/chat/completions`.
If the model supports the OpenAI Chat Completion schema, a toggle should appear in the interface. Enabling this toggle doesn't change any model behavior, it reveals the full endpoint URL needed (which should include `/v1/chat/completions`) when configuring the inference endpoint in Elasticsearch. If the model doesn't support this schema, the toggle may not be shown.
  - `model_id` (string) — optional
    The name of the HuggingFace model to use for the inference task.
For `completion` and `chat_completion` tasks, this field is optional but may be required for certain models — particularly when using serverless inference endpoints.
For the `text_embedding` task, this field should not be included. Otherwise, the request will fail.
- `task_settings` ({} HuggingFaceTaskSettings) — optional
  Settings to configure the inference task.
These settings are specific to the task type you specified.
  - `return_documents` (boolean) — optional
    For a `rerank` task, return doc text within the results.
  - `top_n` (number) — optional
    For a `rerank` task, the number of most relevant documents to return.
It defaults to the number of the documents.

## Response

`application/json`

- `chunking_settings` ({} InferenceChunkingSettings)
  Chunking configuration object
  - `max_chunk_size` (number · default: 250)
    The maximum size of a chunk in words.
This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy).
  - `overlap` (number · default: 100)
    The number of overlapping words for chunks.
It is applicable only to a `word` chunking strategy.
This value cannot be higher than half the `max_chunk_size` value.
  - `sentence_overlap` (number · default: 1)
    The number of overlapping sentences for chunks.
It is applicable only for a `sentence` chunking strategy.
It can be either `1` or `0`.
  - `strategy` (string · default: sentence)
    The chunking strategy: `sentence` or `word`.
- `service` (string)
  The service type
- `service_settings` (ServiceSettings)
  Settings specific to the service
- `task_settings` (TaskSettings)
  Task settings specific to the service and task type
- `inference_id` (string)
  The inference Id
- `task_type` (enum TaskTypeHuggingFace)
  The task type
  Values: `text_embedding`
## Code Examples

#### Console

```console
PUT _inference/text_embedding/hugging-face-embeddings
{
    "service": "hugging_face",
    "service_settings": {
        "api_key": "hugging-face-access-token", 
        "url": "url-endpoint" 
    }
}
```

#### JavaScript

```javascript
const response = await client.inference.put({
  task_type: "text_embedding",
  inference_id: "hugging-face-embeddings",
  inference_config: {
    service: "hugging_face",
    service_settings: {
      api_key: "hugging-face-access-token",
      url: "url-endpoint",
    },
  },
});
```

#### Python

```python
resp = client.inference.put(
    task_type="text_embedding",
    inference_id="hugging-face-embeddings",
    inference_config={
        "service": "hugging_face",
        "service_settings": {
            "api_key": "hugging-face-access-token",
            "url": "url-endpoint"
        }
    },
)
```

#### curl

```curl
curl -X PUT "$ELASTICSEARCH_URL/_inference/text_embedding/hugging-face-embeddings" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "service": "hugging_face",
    "service_settings": {
      "api_key": "hugging-face-access-token",
      "url": "url-endpoint"
    }
  }'
```

#### Java

```java
client.inference().put(p -> p
    .inferenceId("hugging-face-embeddings")
    .taskType(TaskType.TextEmbedding)
    .inferenceConfig(i -> i
        .service("hugging_face")
        .serviceSettings(JsonData.fromJson("{\"api_key\":\"hugging-face-access-token\",\"url\":\"url-endpoint\"}"))
    )
);
```

#### C#

```c#
var response = await client.Inference
    .PutAsync(new PutInferenceRequest()
    {
        InferenceId = "hugging-face-embeddings",
        TaskType = TaskType.TextEmbedding,
        InferenceConfig = new()
        {
            Service = "hugging_face",
            ServiceSettings = JsonSerializer.Deserialize<JsonElement>("""
            {
              "api_key": "hugging-face-access-token",
              "url": "url-endpoint"
            }
            """)
        }
    });
```

#### PHP

```php
$resp = $client->inference()->put([
    "task_type" => "text_embedding",
    "inference_id" => "hugging-face-embeddings",
    "body" => [
        "service" => "hugging_face",
        "service_settings" => [
            "api_key" => "hugging-face-access-token",
            "url" => "url-endpoint",
        ],
    ],
]);
```

#### Ruby

```ruby
response = client.inference.put(
  task_type: "text_embedding",
  inference_id: "hugging-face-embeddings",
  body: {
    "service": "hugging_face",
    "service_settings": {
      "api_key": "hugging-face-access-token",
      "url": "url-endpoint"
    }
  }
)
```