Update a trained model deployment

  • POST /_ml/trained_models/{model_id}/deployment/_update
Auth

Required authorization

Cluster privileges manage_ml

Path parameters

model_id string required

The unique identifier of the trained model. Currently, only PyTorch models are supported.

Query parameters

number_of_allocations number · default: 1

The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads.

Request body

number_of_allocations number · default: 1

The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads. If adaptive_allocations is enabled, do not set this value, because it’s automatically set.

adaptive_allocations object

Adaptive allocations configuration. When enabled, the number of allocations is set based on the current load. If adaptive_allocations is enabled, do not set the number of allocations manually.

enabled boolean required

If true, adaptive_allocations is enabled

min_number_of_allocations number

Specifies the minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.

max_number_of_allocations number

Specifies the maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.

Response

application/json
assignment object required
adaptive_allocations union oneOf nullable
One of:
enabled boolean required

If true, adaptive_allocations is enabled

min_number_of_allocations number

Specifies the minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.

max_number_of_allocations number

Specifies the maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.

assignment_state enum required

The overall assignment state.

Supported values include:

  • started: The deployment is usable; at least one node has the model allocated.
  • starting: The deployment has recently started but is not yet usable; the model is not allocated on any nodes.
  • stopping: The deployment is preparing to stop and deallocate the model from the relevant nodes.
  • failed: The deployment is on a failed state and must be re-deployed.
Values: started starting stopping failed
max_assigned_allocations number
reason string
routing_table map string to object required

The allocation state for each node.

<string> object
reason string

The reason for the current state. It is usually populated only when the routing_state is failed.

routing_state enum required

The current routing state.

Supported values include:

  • failed: The allocation attempt failed.
  • started: The trained model is allocated and ready to accept inference requests.
  • starting: The trained model is attempting to allocate on this node; inference requests are not yet accepted.
  • stopped: The trained model is fully deallocated from this node.
  • stopping: The trained model is being deallocated from this node.
Values: failed started starting stopped stopping
current_allocations number required

Current number of allocations.

target_allocations number required

Target number of allocations.

start_time union oneOf required

The timestamp when the deployment started.

One of: string or number
task_parameters object required
model_bytes number | string required

The size of the trained model in bytes.

model_id string required

The unique identifier for the trained model.

deployment_id string required

The unique identifier for the trained model deployment.

cache_size number | string

The size of the trained model cache.

number_of_allocations number required

The total number of allocations this model is assigned across ML nodes.

priority enum required
Values: normal low
per_deployment_memory_bytes number | string required
per_allocation_memory_bytes number | string required
queue_capacity number required

Number of inference requests are allowed in the queue at a time.

threads_per_allocation number required

Number of threads per allocation.