Start a trained model deployment
- POST /_ml/trained_models/{model_id}/deployment/_start
It allocates the model to every machine learning node.
Required authorization
- Cluster privileges manage_ml
Query parameters
-
cache_sizenumber | string -
The inference cache size (in memory outside the JVM heap) per node for the model. The default value is the same size as the
model_size_bytes. To disable the cache,0bcan be provided. -
number_of_allocationsnumber · default: 1 -
The number of model allocations on each node where the model is deployed. All allocations on a node share the same copy of the model in memory but use a separate set of threads to evaluate the model. Increasing this value generally increases the throughput. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads. If adaptive_allocations is enabled, do not set this value, because it’s automatically set.
-
queue_capacitynumber · default: 1024 -
Specifies the number of inference requests that are allowed in the queue. After the number of requests exceeds this value, new requests are rejected with a 429 error.
-
threads_per_allocationnumber · default: 1 -
Sets the number of threads used by each model allocation during inference. This generally increases the inference speed. The inference process is a compute-bound process; any number greater than the number of available hardware threads on the machine does not increase the inference speed. If this setting is greater than the number of hardware threads it will automatically be changed to a value less than the number of hardware threads.
-
wait_for· default: started -
Specifies the allocation status to wait for before returning.
Supported values include:
started: The trained model is started on at least one node.starting: Trained model deployment is starting but it is not yet deployed on any nodes.fully_allocated: Trained model deployment has started on all valid nodes.
-
Values:
startedstartingfully_allocated
Request body
Response
-
assignmentobjectrequired -
assignment_staterequired -
The overall assignment state.
Supported values include:
started: The deployment is usable; at least one node has the model allocated.starting: The deployment has recently started but is not yet usable; the model is not allocated on any nodes.stopping: The deployment is preparing to stop and deallocate the model from the relevant nodes.failed: The deployment is on a failed state and must be re-deployed.
-
Values:
startedstartingstoppingfailed -
routing_tablemap string to objectrequired -
The allocation state for each node.
-
<string>object -
reasonstring -
The reason for the current state. It is usually populated only when the
routing_stateisfailed. -
routing_staterequired -
The current routing state.
Supported values include:
failed: The allocation attempt failed.started: The trained model is allocated and ready to accept inference requests.starting: The trained model is attempting to allocate on this node; inference requests are not yet accepted.stopped: The trained model is fully deallocated from this node.stopping: The trained model is being deallocated from this node.
-
Values:
failedstartedstartingstoppedstopping