--- type: api title: Perform dense embedding inference on the service url: https://www.elastic.co/elastic/docs-builder/docs/4641/api/doc/elasticsearch-serverless/operation/operation-inference-embedding resource: https://www.elastic.co/elastic/docs-builder/docs/4641/api/doc/elasticsearch-serverless/operation/operation-inference-embedding products: - Elasticsearch Serverless --- # Perform dense embedding inference on the service Authentication: required Availability: Experimental ## Paths - `POST` `/_inference/embedding/{inference_id}` #### Path parameters - `inference_id` (string) — required The inference Id ## Query string parameters - `timeout` (string · default: 30s) — optional Specifies the amount of time to wait for the inference request to complete. ## Request body `application/json` - `input` (union oneOf) — required Inference input. Either a string, an array of strings, a `content` object, or an array of `content` objects. `content` objects may contain a single item or an array of items. Models that support multiple items per `content` object will return a single embedding for each `content` object, regardless of how many items it contains. Support for multiple items in a single `content` object is available in Elasticsearch 9.5.0 and later. string example: ``` "input": "Some text" ``` string array example: ``` "input": ["Some text", "Some more text"] ``` `content` object example: ``` "input": { "content": { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." } } ``` `content` object array example: ``` "input": [ { "content": { "type": "text", "format": "text", "value": "Some text to generate an embedding" } }, { "content": { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." } } ] ``` `content` object using the `url` format example (available in Elasticsearch 9.6.0 and later): ``` "input": { "content": { "type": "image", "format": "url", "value": "https://example.com/image.jpg" } } ``` Multiple items in one `content` object example (available in Elasticsearch 9.5.0 and later): ``` "input": [ { "content": [ { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." }, { "type": "text", "value": "Some text to create an embedding" } ] } ] ``` - **EmbeddingStringInput** Allows specifying text-only inputs for the `embedding` task. - **EmbeddingContentInput** Allows specifying multimodal inputs for the `embedding` task. - `input_type` (string) — optional The input data type for the embedding model. Possible values include: * `SEARCH` * `INGEST` * `CLASSIFICATION` * `CLUSTERING` Not all models support all values. Unsupported values will trigger a validation exception. Accepted values depend on the configured inference service, refer to the relevant service-specific documentation for more info. > info > The `input_type` parameter specified on the root level of the request body will take precedence over the `input_type` parameter specified in `task_settings`. - `task_settings` (object) — optional Task settings for the individual inference request. These settings are specific to the you specified and override the task settings specified when initializing the service. ## Response `application/json` - `embeddings_bytes` (object[]) - `embedding` (number[]) - `embeddings_bits` (object[]) - `embedding` (number[]) - `embeddings` (object[]) - `embedding` (number[]) ## Code examples ### Multimodal embedding task #### Console ```console POST _inference/embedding/my-multimodal-endpoint { "input": [ { "content": { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." } }, { "content": { "type": "text", "value": "Some text to create an embedding" } } ] } ``` #### JavaScript ```javascript const response = await client.inference.embedding({ inference_id: "my-multimodal-endpoint", embedding: { input: [ { content: { type: "image", format: "base64", value: "data:image/jpeg;base64,...", }, }, { content: { type: "text", value: "Some text to create an embedding", }, }, ], }, }); ``` #### Python ```python resp = client.inference.embedding( inference_id="my-multimodal-endpoint", embedding={ "input": [ { "content": { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." } }, { "content": { "type": "text", "value": "Some text to create an embedding" } } ] }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_inference/embedding/my-multimodal-endpoint" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": [ { "content": { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." } }, { "content": { "type": "text", "value": "Some text to create an embedding" } } ] }' ``` #### Java ```java client.inference().embedding(e -> e .inferenceId("my-multimodal-endpoint") .embedding(em -> em .input(i -> i .content(List.of(EmbeddingContentObject.of(emb -> emb .content(c -> c .type(EmbeddingContentType.Image) .format(EmbeddingContentFormat.Base64) .value("data:image/jpeg;base64,...") ) ),EmbeddingContentObject.of(emb -> emb .content(c -> c .type(EmbeddingContentType.Text) .value("Some text to create an embedding") ) ))) ) ) ); ``` #### PHP ```php $resp = $client->inference()->embedding([ "inference_id" => "my-multimodal-endpoint", "body" => [ "input" => array( [ "content" => [ "type" => "image", "format" => "base64", "value" => "data:image/jpeg;base64,...", ], ], [ "content" => [ "type" => "text", "value" => "Some text to create an embedding", ], ], ), ], ]); ``` #### Ruby ```ruby response = client.inference.embedding( inference_id: "my-multimodal-endpoint", body: { "input": [ { "content": { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." } }, { "content": { "type": "text", "value": "Some text to create an embedding" } } ] } ) ``` ### Text-only embedding task #### Console ```console POST _inference/embedding/my-text-only-endpoint { "input": ["The first text", "The second text"] } ``` #### JavaScript ```javascript const response = await client.inference.embedding({ inference_id: "my-text-only-endpoint", embedding: { input: ["The first text", "The second text"], }, }); ``` #### Python ```python resp = client.inference.embedding( inference_id="my-text-only-endpoint", embedding={ "input": [ "The first text", "The second text" ] }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_inference/embedding/my-text-only-endpoint" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": [ "The first text", "The second text" ] }' ``` #### C# ```c# var response = await client.Inference .EmbeddingAsync(inferenceId: "my-text-only-endpoint", d1 => d1 .Embedding(d2 => d2 .Input(new string[] { "The first text", "The second text" }) ) ); ``` #### PHP ```php $resp = $client->inference()->embedding([ "inference_id" => "my-text-only-endpoint", "body" => [ "input" => array( "The first text", "The second text", ), ], ]); ``` #### Ruby ```ruby response = client.inference.embedding( inference_id: "my-text-only-endpoint", body: { "input": [ "The first text", "The second text" ] } ) ``` ### Embedding task using multiple items grouped under a single content object #### Console ```console POST _inference/embedding/my-multimodal-endpoint { "input": [ { "content": [ { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." }, { "type": "text", "value": "Some text to create an embedding" } ] } ] } ``` #### JavaScript ```javascript const response = await client.inference.embedding({ inference_id: "my-multimodal-endpoint", embedding: { input: [ { content: [ { type: "image", format: "base64", value: "data:image/jpeg;base64,...", }, { type: "text", value: "Some text to create an embedding", }, ], }, ], }, }); ``` #### Python ```python resp = client.inference.embedding( inference_id="my-multimodal-endpoint", embedding={ "input": [ { "content": [ { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." }, { "type": "text", "value": "Some text to create an embedding" } ] } ] }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_inference/embedding/my-multimodal-endpoint" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": [ { "content": [ { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." }, { "type": "text", "value": "Some text to create an embedding" } ] } ] }' ``` #### Java ```java client.inference().embedding(e -> e .inferenceId("my-multimodal-endpoint") .embedding(em -> em .input(i -> i .content(c -> c .content(List.of(EmbeddingContentObjectItem.of(emb -> emb .type(EmbeddingContentType.Image) .format(EmbeddingContentFormat.Base64) .value("data:image/jpeg;base64,...") ),EmbeddingContentObjectItem.of(emb -> emb .type(EmbeddingContentType.Text) .value("Some text to create an embedding") ))) ) ) ) ); ``` #### PHP ```php $resp = $client->inference()->embedding([ "inference_id" => "my-multimodal-endpoint", "body" => [ "input" => array( [ "content" => array( [ "type" => "image", "format" => "base64", "value" => "data:image/jpeg;base64,...", ], [ "type" => "text", "value" => "Some text to create an embedding", ], ), ], ), ], ]); ``` #### Ruby ```ruby response = client.inference.embedding( inference_id: "my-multimodal-endpoint", body: { "input": [ { "content": [ { "type": "image", "format": "base64", "value": "data:image/jpeg;base64,..." }, { "type": "text", "value": "Some text to create an embedding" } ] } ] } ) ``` ### Embedding task using the url format #### Console ```console POST _inference/embedding/my-multimodal-endpoint { "input": { "content": { "type": "image", "format": "url", "value": "https://example.com/image.jpg" } } } ``` #### JavaScript ```javascript const response = await client.inference.embedding({ inference_id: "my-multimodal-endpoint", embedding: { input: { content: { type: "image", format: "url", value: "https://example.com/image.jpg", }, }, }, }); ``` #### Python ```python resp = client.inference.embedding( inference_id="my-multimodal-endpoint", embedding={ "input": { "content": { "type": "image", "format": "url", "value": "https://example.com/image.jpg" } } }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_inference/embedding/my-multimodal-endpoint" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": { "content": { "type": "image", "format": "url", "value": "https://example.com/image.jpg" } } }' ``` #### PHP ```php $resp = $client->inference()->embedding([ "inference_id" => "my-multimodal-endpoint", "body" => [ "input" => [ "content" => [ "type" => "image", "format" => "url", "value" => "https://example.com/image.jpg", ], ], ], ]); ``` #### Ruby ```ruby response = client.inference.embedding( inference_id: "my-multimodal-endpoint", body: { "input": { "content": { "type": "image", "format": "url", "value": "https://example.com/image.jpg" } } } ) ``` ## Response examples #### Multimodal embedding task `application/json` An abbreviated response from `POST _inference/embedding/my-multimodal-endpoint`. ```json { "embeddings": [ { "embedding": [ -0.0189209, -0.04174805, 0.00854492, 0.01556396, 0.01928711, -0.00616455, -0.00460815, 0.01477051, -0.00656128, 0.05419922 ] }, { "embedding": [ -0.01379395, -0.02368164, 0.01068115, 0.0279541, 0.01043701, -7.7057E-4, 0.04150391, 0.00836182, -0.01135254, 0.0246582 ] } ] } ``` #### Text-only embedding task `application/json` An abbreviated response from `POST _inference/embedding/my-text-only-endpoint`. ```json { "embeddings": [ { "embedding": [ 0.00854492, -0.00616455, -0.0189209, 0.01556396, -0.00460815, 0.01477051, -0.04174805, 0.01928711, -0.00656128, 0.05419922 ] }, { "embedding": [ -0.01135254, 0.0279541, -0.02368164, 0.01068115, 0.01043701, 0.04150391, 0.00836182, -7.7057E-4, -0.01379395, 0.0246582 ] } ] } ```