Create an Amazon SageMaker inference endpoint
- PUT /_inference/{task_type}/{amazonsagemaker_inference_id}
Create an inference endpoint to perform an inference task with the amazon_sagemaker service.
- Cluster privileges:
manage_inference
Path Parameters
-
task_typeTaskTypeAmazonSageMakerrequired -
The type of the inference task that the model will perform.
-
Values:
text_embeddingcompletionchat_completionsparse_embeddingrerank -
amazonsagemaker_inference_idstring Idrequired -
The unique identifier of the inference endpoint.
Query Parameters
-
timeoutstring Duration · default: 30s Specifies the amount of time to wait for the inference endpoint to be created.
Request
-
chunking_settingsInferenceChunkingSettings -
The chunking configuration object. Applies only to the
sparse_embeddingortext_embeddingtask types. Not applicable to thererank,completion, orchat_completiontask types. - Reference
-
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be lower than
20(forsentencestrategy) or10(forwordstrategy). This value should not exceed the window size for the associated model. -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
separator_groupstring -
Only applicable to the
recursivestrategy and required when using it.Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be
markdownorplaintext.Using this parameter is an alternative to manually specifying a custom
separatorslist. -
separatorsstring -
Only applicable to the
recursivestrategy and required when using it.A list of strings used as possible split points when chunking text.
Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the
max_chunk_sizelimit, to reduce the total number of chunks generated. -
serviceAmazonSageMakerServiceTyperequired -
The type of service supported for the specified task type. In this case,
amazon_sagemaker. -
Values:
amazon_sagemaker -
service_settingsAmazonSageMakerServiceSettingsrequired -
Settings used to install the inference model. These settings are specific to the
amazon_sagemakerservice andservice_settings.apiyou specified. -
access_keystringrequired -
A valid AWS access key that has permissions to use Amazon SageMaker and access to models for invoking requests.
-
apiAmazonSageMakerApirequired -
The API format to use when calling SageMaker. Elasticsearch will convert the POST _inference request to this data format when invoking the SageMaker endpoint.
-
Values:
openaielastic -
regionstringrequired -
The region that your endpoint or Amazon Resource Name (ARN) is deployed in. The list of available regions per model can be found in the Amazon SageMaker documentation.
- External docs
-
secret_keystringrequired -
A valid AWS secret key that is paired with the
access_key. For information about creating and managing access and secret keys, refer to the AWS documentation. - External docs
-
similarityAmazonSageMakerSimilarity -
Required when
apiiselasticand task type istext_embedding. The similarity measure used when invoking thetext_embeddingtask type. -
Values:
cosinedot_productl2_norm -
element_typeAmazonSageMakerElementType -
Required when
apiiselasticand task type istext_embedding. The data type returned by the text embedding model. This value is used when parsing the response back to Elasticsearch data structures. -
Values:
bytefloatbit -
target_container_hostnamestring -
The container to directly invoke when calling a multi-container endpoint.
- External docs
-
inference_component_namestring -
The inference component to directly invoke when calling a multi-component endpoint.
- External docs
-
batch_sizenumber · default: 256 -
The maximum number of inputs in each batch. This value is used by inference ingestion pipelines when processing semantic values. It correlates to the number of times the SageMaker endpoint is invoked (one per batch of input).
-
dimensionsnumber -
The number of dimensions returned by the text embedding models. If this value is not provided, then it is guessed by making invoking the endpoint for the
text_embeddingtask. -
task_settingsAmazonSageMakerTaskSettings -
Settings to configure the inference task. These settings are specific to the task type and
service_settings.apiyou specified. -
custom_attributesstring -
The AWS custom attributes passed verbatim through to the model running in the SageMaker Endpoint. Values will be returned in the
X-elastic-sagemaker-custom-attributesheader. - External docs
-
enable_explanationsstring -
The optional JMESPath expression used to override the EnableExplanations provided during endpoint creation.
- External docs
-
session_idstring -
The stateful session identifier for a new or existing session. New sessions will be returned in the
X-elastic-sagemaker-new-session-idheader. Closed sessions will be returned in theX-elastic-sagemaker-closed-session-idheader. - External docs
-
target_variantstring -
Specifies the variant when running with multi-variant Endpoints.
- External docs
Response
-
chunking_settingsInferenceChunkingSettings -
The chunking configuration object. Applies only to the
embedding,sparse_embeddingandtext_embeddingtask types. Not applicable to thererank,completion, orchat_completiontask types. -
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be lower than
20(forsentencestrategy) or10(forwordstrategy). This value should not exceed the window size for the associated model. -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
separator_groupstring -
Only applicable to the
recursivestrategy and required when using it.Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be
markdownorplaintext.Using this parameter is an alternative to manually specifying a custom
separatorslist. -
separatorsstring -
Only applicable to the
recursivestrategy and required when using it.A list of strings used as possible split points when chunking text.
Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the
max_chunk_sizelimit, to reduce the total number of chunks generated. -
servicestringrequired -
The service type
-
service_settingsServiceSettingsrequired -
Settings specific to the service
-
task_settingsTaskSettings -
Task settings specific to the service and task type
-
inference_idstringrequired -
The inference Id
-
task_typeTaskTypeAmazonSageMakerrequired -
The task type
-
Values:
text_embeddingcompletionchat_completionsparse_embeddingrerank