Create an JinaAI inference endpoint
- PUT /_inference/{task_type}/{jinaai_inference_id}
Create an inference endpoint to perform an inference task with the jinaai service.
To review the available rerank models, refer to https://jina.ai/reranker.
To review the available text_embedding models, refer to the https://jina.ai/embeddings/.
- Cluster privileges:
manage_inference
Path Parameters
-
task_typeJinaAITaskTyperequired -
The type of the inference task that the model will perform.
-
Values:
reranktext_embedding -
jinaai_inference_idstring Idrequired -
The unique identifier of the inference endpoint.
Query Parameters
-
timeoutstring Duration · default: 30s Specifies the amount of time to wait for the inference endpoint to be created.
Request
-
chunking_settingsInferenceChunkingSettings -
The chunking configuration object.
- Reference
-
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be higher than
300or lower than20(forsentencestrategy) or10(forwordstrategy). -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
strategystring · default: sentence -
The chunking strategy:
sentenceorword. -
serviceJinaAIServiceTyperequired -
The type of service supported for the specified task type. In this case,
jinaai. -
Values:
jinaai -
service_settingsJinaAIServiceSettingsrequired -
Settings used to install the inference model. These settings are specific to the
jinaaiservice. -
api_keystringrequired -
A valid API key of your JinaAI account.
IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key.
- External docs
-
model_idstringrequired -
The name of the model to use for the inference task.
-
rate_limitRateLimitSetting -
This setting helps to minimize the number of rate limit errors returned from JinaAI. By default, the
jinaaiservice sets the number of requests allowed per minute to 2000 for all task types. - External docs
-
requests_per_minutenumber -
The number of requests allowed per minute.
-
similarityJinaAISimilarityType -
For a
text_embeddingtask, the similarity measure. One of cosine, dot_product, l2_norm. The default values varies with the embedding type. For example, a float embedding type uses adot_productsimilarity measure by default. -
Values:
cosinedot_productl2_norm -
dimensionsnumber -
For a
text_embeddingtask, the number of dimensions the resulting output embeddings should have. By default, the model's standard output dimension is used. Refer to the Jina documentation for more information. - External docs
-
embedding_typeJinaAIElementType · default: float -
For a
text_embeddingtask, the data type returned by the model. Usebitfor binary embeddings, which are encoded as bytes with signed int8 precision. Usebinaryfor binary embeddings, which are encoded as bytes with signed int8 precision (this is a synonym ofbit). Usefloatfor the default float embeddings. -
Values:
binarybitfloat -
task_settingsJinaAITaskSettings -
Settings to configure the inference task. These settings are specific to the task type you specified.
-
return_documentsboolean -
For a
reranktask, return the doc text within the results. -
taskJinaAITextEmbeddingTask -
For a
text_embeddingtask, the task passed to the model. Valid values are:classification: Use it for embeddings passed through a text classifier.clustering: Use it for the embeddings run through a clustering algorithm.ingest: Use it for storing document embeddings in a vector database.search: Use it for storing embeddings of search queries run against a vector database to find relevant documents.
-
Values:
classificationclusteringingestsearch -
top_nnumber -
For a
reranktask, the number of most relevant documents to return. It defaults to the number of the documents. If this inference endpoint is used in atext_similarity_rerankerretriever query andtop_nis set, it must be greater than or equal torank_window_sizein the query.
Response
-
chunking_settingsInferenceChunkingSettings -
Chunking configuration object
-
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be higher than
300or lower than20(forsentencestrategy) or10(forwordstrategy). -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
strategystring · default: sentence -
The chunking strategy:
sentenceorword. -
servicestringrequired -
The service type
-
service_settingsServiceSettingsrequired -
Settings specific to the service
-
task_settingsTaskSettings -
Task settings specific to the service and task type
-
inference_idstringrequired -
The inference Id