Update a trained model deployment
- POST /_ml/trained_models/{model_id}/deployment/_update
- Cluster privileges:
manage_ml
Path Parameters
-
model_idstring Idrequired -
The unique identifier of the trained model. Currently, only PyTorch models are supported.
Query Parameters
-
number_of_allocationsnumber · default: 1 The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads.
Request
-
number_of_allocationsnumber · default: 1 -
The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads. If adaptive_allocations is enabled, do not set this value, because it’s automatically set.
-
adaptive_allocationsAdaptiveAllocationsSettings -
Adaptive allocations configuration. When enabled, the number of allocations is set based on the current load. If adaptive_allocations is enabled, do not set the number of allocations manually.
-
enabledbooleanrequired -
If true, adaptive_allocations is enabled
-
min_number_of_allocationsnumber -
Specifies the minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.
-
max_number_of_allocationsnumber -
Specifies the maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.
Response
-
assignmentTrainedModelAssignmentrequired -
adaptive_allocationsoneOfnullable -
enabledbooleanrequired -
If true, adaptive_allocations is enabled
-
min_number_of_allocationsnumber -
Specifies the minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.
-
max_number_of_allocationsnumber -
Specifies the maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.
-
assignment_stateDeploymentAssignmentStaterequired -
The overall assignment state.
Supported values include:
started: The deployment is usable; at least one node has the model allocated.starting: The deployment has recently started but is not yet usable; the model is not allocated on any nodes.stopping: The deployment is preparing to stop and deallocate the model from the relevant nodes.failed: The deployment is on a failed state and must be re-deployed.
-
Values:
startedstartingstoppingfailed -
max_assigned_allocationsnumber -
reasonstring -
routing_tablemap string to TrainedModelAssignmentRoutingTablerequired -
The allocation state for each node.
-
<string>TrainedModelAssignmentRoutingTable -
reasonstring -
The reason for the current state. It is usually populated only when the
routing_stateisfailed. -
routing_stateRoutingStaterequired -
The current routing state.
Supported values include:
failed: The allocation attempt failed.started: The trained model is allocated and ready to accept inference requests.starting: The trained model is attempting to allocate on this node; inference requests are not yet accepted.stopped: The trained model is fully deallocated from this node.stopping: The trained model is being deallocated from this node.
-
Values:
failedstartedstartingstoppedstopping -
current_allocationsnumberrequired -
Current number of allocations.
-
target_allocationsnumberrequired -
Target number of allocations.
-
start_timeDateTimerequired -
The timestamp when the deployment started.
-
task_parametersTrainedModelAssignmentTaskParametersrequired -
model_bytesnumber | string ByteSizerequired -
The size of the trained model in bytes.
-
model_idstring Idrequired -
The unique identifier for the trained model.
-
deployment_idstring Idrequired -
The unique identifier for the trained model deployment.
-
cache_sizenumber | string ByteSize -
The size of the trained model cache.
-
number_of_allocationsnumberrequired -
The total number of allocations this model is assigned across ML nodes.
-
priorityTrainingPriorityrequired -
Values:
normallow -
per_deployment_memory_bytesnumber | string ByteSizerequired -
per_allocation_memory_bytesnumber | string ByteSizerequired -
queue_capacitynumberrequired -
Number of inference requests are allowed in the queue at a time.
-
threads_per_allocationnumberrequired -
Number of threads per allocation.
string