Create an AlibabaCloud AI Search inference endpoint

Auth

Create an inference endpoint to perform an inference task with the alibabacloud-ai-search service.

Required authorization

Cluster privileges manage_inference

Path parameters

task_type enum required

The type of the inference task that the model will perform.

Values: completion rerank space_embedding text_embedding
alibabacloud_inference_id string required

The unique identifier of the inference endpoint.

Query parameters

timeout string · default: 30s

Specifies the amount of time to wait for the inference endpoint to be created.

Request body

chunking_settings object

The chunking configuration object.

Reference
max_chunk_size number · default: 250

The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

overlap number · default: 100

The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

sentence_overlap number · default: 1

The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

strategy string · default: sentence

The chunking strategy: sentence or word.

service enum required

The type of service supported for the specified task type. In this case, alibabacloud-ai-search.

Values: alibabacloud-ai-search
service_settings object required

Settings used to install the inference model. These settings are specific to the alibabacloud-ai-search service.

api_key string required

A valid API key for the AlibabaCloud AI Search API.

host string required

The name of the host address used for the inference task. You can find the host address in the API keys section of the documentation.

External docs
rate_limit object

This setting helps to minimize the number of rate limit errors returned from AlibabaCloud AI Search. By default, the alibabacloud-ai-search service sets the number of requests allowed per minute to 1000.

requests_per_minute number

The number of requests allowed per minute.

service_id string required

The name of the model service to use for the inference task. The following service IDs are available for the completion task:

  • ops-qwen-turbo
  • qwen-turbo
  • qwen-plus
  • qwen-max ÷ qwen-max-longcontext

The following service ID is available for the rerank task:

  • ops-bge-reranker-larger

The following service ID is available for the sparse_embedding task:

  • ops-text-sparse-embedding-001

The following service IDs are available for the text_embedding task:

ops-text-embedding-001 ops-text-embedding-zh-001 ops-text-embedding-en-001 ops-text-embedding-002

workspace string required

The name of the workspace used for the inference task.

task_settings object

Settings to configure the inference task. These settings are specific to the task type you specified.

input_type string

For a sparse_embedding or text_embedding task, specify the type of input passed to the model. Valid values are:

  • ingest for storing document embeddings in a vector database.
  • search for storing embeddings of search queries run against a vector database to find relevant documents.
return_token boolean

For a sparse_embedding task, it affects whether the token name will be returned in the response. It defaults to false, which means only the token ID will be returned in the response.

Response

application/json
chunking_settings object

Chunking configuration object

max_chunk_size number · default: 250

The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

overlap number · default: 100

The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

sentence_overlap number · default: 1

The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

strategy string · default: sentence

The chunking strategy: sentence or word.

service string required

The service type

service_settings object required

Settings specific to the service

task_settings object

Task settings specific to the service and task type

inference_id string required

The inference Id

task_type enum required

The task type

Values: text_embedding rerank completion sparse_embedding