Create an AlibabaCloud AI Search inference endpoint
- PUT /_inference/{task_type}/{alibabacloud_inference_id}
Create an inference endpoint to perform an inference task with the alibabacloud-ai-search service.
- Cluster privileges:
manage_inference
Path Parameters
-
task_typeAlibabaCloudTaskTyperequired -
The type of the inference task that the model will perform.
-
Values:
completionrerankspace_embeddingtext_embedding -
alibabacloud_inference_idstring Idrequired -
The unique identifier of the inference endpoint.
Query Parameters
-
timeoutstring Duration · default: 30s Specifies the amount of time to wait for the inference endpoint to be created.
Request
-
chunking_settingsInferenceChunkingSettings -
The chunking configuration object.
- Reference
-
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be higher than
300or lower than20(forsentencestrategy) or10(forwordstrategy). -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
strategystring · default: sentence -
The chunking strategy:
sentenceorword. -
serviceAlibabaCloudServiceTyperequired -
The type of service supported for the specified task type. In this case,
alibabacloud-ai-search. -
Values:
alibabacloud-ai-search -
service_settingsAlibabaCloudServiceSettingsrequired -
Settings used to install the inference model. These settings are specific to the
alibabacloud-ai-searchservice. -
api_keystringrequired -
A valid API key for the AlibabaCloud AI Search API.
-
hoststringrequired -
The name of the host address used for the inference task. You can find the host address in the API keys section of the documentation.
- External docs
-
rate_limitRateLimitSetting -
This setting helps to minimize the number of rate limit errors returned from AlibabaCloud AI Search. By default, the
alibabacloud-ai-searchservice sets the number of requests allowed per minute to1000. -
requests_per_minutenumber -
The number of requests allowed per minute.
-
service_idstringrequired -
The name of the model service to use for the inference task. The following service IDs are available for the
completiontask:ops-qwen-turboqwen-turboqwen-plusqwen-max ÷ qwen-max-longcontext
The following service ID is available for the
reranktask:ops-bge-reranker-larger
The following service ID is available for the
sparse_embeddingtask:ops-text-sparse-embedding-001
The following service IDs are available for the
text_embeddingtask:ops-text-embedding-001ops-text-embedding-zh-001ops-text-embedding-en-001ops-text-embedding-002 -
workspacestringrequired -
The name of the workspace used for the inference task.
-
task_settingsAlibabaCloudTaskSettings -
Settings to configure the inference task. These settings are specific to the task type you specified.
-
input_typestring -
For a
sparse_embeddingortext_embeddingtask, specify the type of input passed to the model. Valid values are:ingestfor storing document embeddings in a vector database.searchfor storing embeddings of search queries run against a vector database to find relevant documents.
-
return_tokenboolean -
For a
sparse_embeddingtask, it affects whether the token name will be returned in the response. It defaults tofalse, which means only the token ID will be returned in the response.
Response
-
chunking_settingsInferenceChunkingSettings -
Chunking configuration object
-
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be higher than
300or lower than20(forsentencestrategy) or10(forwordstrategy). -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
strategystring · default: sentence -
The chunking strategy:
sentenceorword. -
servicestringrequired -
The service type
-
service_settingsServiceSettingsrequired -
Settings specific to the service
-
task_settingsTaskSettings -
Task settings specific to the service and task type
-
inference_idstringrequired -
The inference Id
-
task_typeTaskTypeAlibabaCloudAIrequired -
The task type
-
Values:
text_embeddingrerankcompletionsparse_embedding