--- type: api title: Create a Google Vertex AI inference endpoint description: Create an inference endpoint to perform an inference task with the `googlevertexai` service. url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-inference-put-googlevertexai resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/operation/operation-inference-put-googlevertexai products: - Elasticsearch --- # Create a Google Vertex AI inference endpoint Availability: 8.15+ ## Paths - `PUT` `/_inference/{task_type}/{googlevertexai_inference_id}` - [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#apikeyauth) - [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#basicauth) - [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/v8/authentication#bearerauth) #### Path Parameters - `task_type` (enum GoogleVertexAITaskType) The type of the inference task that the model will perform. Values: `rerank`, `text_embedding`, `completion`, `chat_completion` - `googlevertexai_inference_id` (string Id) The unique identifier of the inference endpoint. ## Description Create an inference endpoint to perform an inference task with the `googlevertexai` service. ## Required authorization * Cluster privileges: `manage_inference` ## Query String Parameters - `timeout` (string Duration · default: 30s) Specifies the amount of time to wait for the inference endpoint to be created. ## Request Body `application/json` - `chunking_settings` ({} InferenceChunkingSettings) — optional The chunking configuration object. - `max_chunk_size` (number · default: 250) — optional The maximum size of a chunk in words. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). - `overlap` (number · default: 100) — optional The number of overlapping words for chunks. It is applicable only to a `word` chunking strategy. This value cannot be higher than half the `max_chunk_size` value. - `sentence_overlap` (number · default: 1) — optional The number of overlapping sentences for chunks. It is applicable only for a `sentence` chunking strategy. It can be either `1` or `0`. - `strategy` (string · default: sentence) — optional The chunking strategy: `sentence` or `word`. - `service` (enum GoogleVertexAIServiceType) — required The type of service supported for the specified task type. In this case, `googlevertexai`. Values: `googlevertexai` - `service_settings` ({} GoogleVertexAIServiceSettings) — required Settings used to install the inference model. These settings are specific to the `googlevertexai` service. - `location` (string) — required The name of the location to use for the inference task. Refer to the Google documentation for the list of supported locations. - `model_id` (string) — required The name of the model to use for the inference task. Refer to the Google documentation for the list of supported models. - `project_id` (string) — required The name of the project to use for the inference task. - `rate_limit` ({} RateLimitSetting) — optional This setting helps to minimize the number of rate limit errors returned from Google Vertex AI. By default, the `googlevertexai` service sets the number of requests allowed per minute to 30.000. - `requests_per_minute` (number) — optional The number of requests allowed per minute. - `service_account_json` (string) — required A valid service account in JSON format for the Google Vertex AI API. - `task_settings` ({} GoogleVertexAITaskSettings) — optional Settings to configure the inference task. These settings are specific to the task type you specified. - `auto_truncate` (boolean) — optional For a `text_embedding` task, truncate inputs longer than the maximum token length automatically. - `top_n` (number) — optional For a `rerank` task, the number of the top N documents that should be returned. ## Response `application/json` - `chunking_settings` ({} InferenceChunkingSettings) Chunking configuration object - `max_chunk_size` (number · default: 250) The maximum size of a chunk in words. This value cannot be higher than `300` or lower than `20` (for `sentence` strategy) or `10` (for `word` strategy). - `overlap` (number · default: 100) The number of overlapping words for chunks. It is applicable only to a `word` chunking strategy. This value cannot be higher than half the `max_chunk_size` value. - `sentence_overlap` (number · default: 1) The number of overlapping sentences for chunks. It is applicable only for a `sentence` chunking strategy. It can be either `1` or `0`. - `strategy` (string · default: sentence) The chunking strategy: `sentence` or `word`. - `service` (string) The service type - `service_settings` (ServiceSettings) Settings specific to the service - `task_settings` (TaskSettings) Task settings specific to the service and task type - `inference_id` (string) The inference Id - `task_type` (enum TaskTypeGoogleVertexAI) The task type Values: `text_embedding`, `rerank` ## Code Examples #### Console ```console PUT _inference/text_embedding/google_vertex_ai_embeddingss { "service": "googlevertexai", "service_settings": { "service_account_json": "service-account-json", "model_id": "model-id", "location": "location", "project_id": "project-id" } } ``` #### JavaScript ```javascript const response = await client.inference.put({ task_type: "text_embedding", inference_id: "google_vertex_ai_embeddingss", inference_config: { service: "googlevertexai", service_settings: { service_account_json: "service-account-json", model_id: "model-id", location: "location", project_id: "project-id", }, }, }); ``` #### Python ```python resp = client.inference.put( task_type="text_embedding", inference_id="google_vertex_ai_embeddingss", inference_config={ "service": "googlevertexai", "service_settings": { "service_account_json": "service-account-json", "model_id": "model-id", "location": "location", "project_id": "project-id" } }, ) ``` #### curl ```curl curl -X PUT "$ELASTICSEARCH_URL/_inference/text_embedding/google_vertex_ai_embeddingss" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "service": "googlevertexai", "service_settings": { "service_account_json": "service-account-json", "model_id": "model-id", "location": "location", "project_id": "project-id" } }' ``` #### Java ```java client.inference().put(p -> p .inferenceId("google_vertex_ai_embeddingss") .taskType(TaskType.TextEmbedding) .inferenceConfig(i -> i .service("googlevertexai") .serviceSettings(JsonData.fromJson("{\"service_account_json\":\"service-account-json\",\"model_id\":\"model-id\",\"location\":\"location\",\"project_id\":\"project-id\"}")) ) ); ``` #### C# ```c# var response = await client.Inference .PutAsync(new PutInferenceRequest() { InferenceId = "google_vertex_ai_embeddingss", TaskType = TaskType.TextEmbedding, InferenceConfig = new() { Service = "googlevertexai", ServiceSettings = JsonSerializer.Deserialize(""" { "service_account_json": "service-account-json", "model_id": "model-id", "location": "location", "project_id": "project-id" } """) } }); ``` #### PHP ```php $resp = $client->inference()->put([ "task_type" => "text_embedding", "inference_id" => "google_vertex_ai_embeddingss", "body" => [ "service" => "googlevertexai", "service_settings" => [ "service_account_json" => "service-account-json", "model_id" => "model-id", "location" => "location", "project_id" => "project-id", ], ], ]); ``` #### Ruby ```ruby response = client.inference.put( task_type: "text_embedding", inference_id: "google_vertex_ai_embeddingss", body: { "service": "googlevertexai", "service_settings": { "service_account_json": "service-account-json", "model_id": "model-id", "location": "location", "project_id": "project-id" } } ) ```