Optimize performance and accuracy

Approximate kNN search balances query speed, result quality, and resource use, and small configuration changes can have a large impact on latency, recall, and storage costs in production.

This page covers trade-offs among search speed, recall, indexing cost, vector storage, quantization, and rescoring for approximate kNN search.

To gather results, the kNN API first finds a num_candidates number of approximate neighbors per shard, computes similarity to the query vector, selects the top k per shard, and merges them into the global top k nearest neighbors.

For HNSW indices, num_candidates is the main search-time speed/accuracy control:

• Increase num_candidates to improve recall and accuracy (at the cost of higher latency).
• Decrease num_candidates for faster queries (with a potential accuracy trade-off).

For DiskBBQ (bbq_disk) indices, you can also use visit_percentage to control the total percentage of vectors visited during search. visit_percentage accepts values from 0 to 100, including decimal values such as 0.5 (half a percent):

• A good starting value is 3 (3%).
• Increase visit_percentage to improve recall and accuracy (at the cost of higher latency).
• Decrease visit_percentage for faster queries (with a potential accuracy trade-off).

When a search targets both HNSW and DiskBBQ indices, use visit_percentage with num_candidates to tune performance and recall across both index types.

When quantization is involved, rescore_vector is an additional speed/accuracy tuning tool. It reranks a larger candidate set using original vectors after approximate retrieval.

• Increase rescore_vector.oversample to improve accuracy (at the cost of higher latency).
• Decrease rescore_vector.oversample for faster queries (with a potential accuracy trade-off).
• For detailed behavior and usage guidance, see Oversampling and rescoring for quantized vectors.

The approximate kNN search API also supports byte (int8) value vectors alongside float vectors. Use the knn option to search a dense_vector field with element_type set to byte and indexing enabled. Byte vectors reduce memory footprint and can improve cache efficiency for large-scale vector similarity search.

1. Explicitly map one or more dense_vector fields with element_type set to byte and indexing enabled.

   				PUT byte-image-index
   					{
     "mappings": {
       "properties": {
         "byte-image-vector": {
           "type": "dense_vector",
           "element_type": "byte",
           "dims": 2
         },
         "title": {
           "type": "text"
         }
       }
     }
   }
   		

2. Index your data ensuring all vector values are integers within the range [-128, 127].

   				POST byte-image-index/_bulk?refresh=true
   					{ "index": { "_id": "1" } }
   { "byte-image-vector": [5, -20], "title": "moose family" }
   { "index": { "_id": "2" } }
   { "byte-image-vector": [8, -15], "title": "alpine lake" }
   { "index": { "_id": "3" } }
   { "byte-image-vector": [11, 23], "title": "full moon" }
   		

3. Run the search using the knn option ensuring the query_vector values are integers within the range [-128, 127].

   				POST byte-image-index/_search
   					{
     "knn": {
       "field": "byte-image-vector",
       "query_vector": [-5, 9],
       "k": 10,
       "num_candidates": 100
     },
     "fields": [ "title" ]
   }
   		

Note: In addition to the standard byte array, one can also provide a hex-encoded string value for the query_vector param. As an example, the search request above can also be expressed as follows, which would yield the same results

				POST byte-image-index/_search
					{
  "knn": {
    "field": "byte-image-vector",
    "query_vector": "fb09",
    "k": 10,
    "num_candidates": 100
  },
  "fields": [ "title" ]
}
		

If you want to provide float vectors but still get the memory savings of byte vectors, use the quantization feature. Quantization allows you to provide float vectors, but internally they are indexed as byte vectors. Additionally, the original float vectors are still retained in the index.

You can use the default quantization strategy or specify an index option. For example, use int8_hnsw:

				PUT quantized-image-index
					{
  "mappings": {
    "properties": {
      "image-vector": {
        "type": "dense_vector",
        "element_type": "float",
        "dims": 2,
        "index": true,
        "index_options": {
          "type": "int8_hnsw"
        }
      },
      "title": {
        "type": "text"
      }
    }
  }
}
		

1. Index your float vectors.

   				POST quantized-image-index/_bulk?refresh=true
   					{ "index": { "_id": "1" } }
   { "image-vector": [0.1, -2], "title": "moose family" }
   { "index": { "_id": "2" } }
   { "image-vector": [0.75, -1], "title": "alpine lake" }
   { "index": { "_id": "3" } }
   { "image-vector": [1.2, 0.1], "title": "full moon" }
   		

2. Run the search using the knn option. When searching, the float vector is automatically quantized to a byte vector.

   				POST quantized-image-index/_search
   					{
     "knn": {
       "field": "image-vector",
       "query_vector": [0.1, -2],
       "k": 10,
       "num_candidates": 100
     },
     "fields": [ "title" ]
   }
   		

Because the original float vectors are retained alongside the quantized index, you can use them for re-scoring: retrieve candidates quickly via the int8_hnsw index, then rescore the top k hits using the original float vectors. This provides the best of both worlds, fast search and accurate scoring.

				POST quantized-image-index/_search
					{
  "knn": {
    "field": "image-vector",
    "query_vector": [0.1, -2],
    "k": 15,
    "num_candidates": 100
  },
  "fields": [ "title" ],
  "rescore": {
    "window_size": 10,
    "query": {
      "rescore_query": {
        "script_score": {
          "query": {
            "match_all": {}
          },
          "script": {
            "source": "cosineSimilarity(params.query_vector, 'image-vector') + 1.0",
            "params": {
              "query_vector": [0.1, -2]
            }
          }
        }
      }
    }
  }
}
		

Instead of storing raw vectors as 4-byte values, you can use element_type: bfloat16 to store each dimension as a 2-byte value. This can be useful if your indexed vectors are at bfloat16 precision already, or if you want to reduce the disk space required to store vector data. When this element type is used, Elasticsearch automatically rounds 4-byte float values to 2-byte bfloat16 values when indexing vectors.

Due to the reduced precision of bfloat16, any vectors retrieved from the index might have slightly different values to those originally indexed.

When using quantized vectors for kNN search, you can optionally rescore results to balance performance and accuracy, by doing:

• Oversampling: retrieving more candidates per shard.
• Rescoring: recalculating scores on those oversampled candidates using the original (non-quantized) vectors.

Because final scores are computed with the original float vectors, rescoring combines:

• The performance and memory benefits of approximate retrieval with quantized vectors.
• The accuracy of using the original vectors for rescoring the top candidates.

All quantization introduces some accuracy loss, and higher compression generally increases that loss. In practice:

• int8 typically needs little to no rescoring.
• int4 often benefits from rescoring for higher accuracy or recall; 1.5×–2× oversampling usually recovers most loss.
• bbq commonly requires rescoring except on very large indices or models specifically designed for quantization; 3×–5× oversampling is generally sufficient, but higher might be needed for low-dimension vectors or embeddings that quantize poorly.

Use rescore_vector to automatically perform reranking. When you specify an oversample value, approximate kNN will:

• Retrieve num_candidates candidates per shard.
• Rescore the top k * oversample candidates per shard using the original vectors.
• Return the top k rescored candidates.

Here is an example of using the rescore_vector option with the oversample parameter:

				POST image-index/_search
					{
  "knn": {
    "field": "image-vector",
    "query_vector": [-5, 9, -12],
    "k": 10,
    "num_candidates": 100,
    "rescore_vector": {
      "oversample": 2.0
    }
  },
  "fields": [ "title", "file-type" ]
}
		

This example will:

• Search using approximate kNN for the top 100 candidates.
• Rescore the top 20 candidates (oversample * k) per shard using the original, non quantized vectors.
• Return the top 10 (k) rescored candidates.
• Merge the rescored candidates from all shards, and return the top 10 (k) results.

By default, Elasticsearch reads raw vector data into memory to perform rescoring. This can have an effect on performance if the vector data is too large to all fit in off-heap memory at once. When the on_disk_rescore: true index setting is set, Elasticsearch reads vector data directly from disk during rescoring.

This setting only applies to newly indexed vectors. To apply the option to all vectors in the index, the vectors must be re-indexed or force-merged after changing the setting.

The following sections provide additional ways of rescoring:

You can use this option when you don’t want to rescore on each shard, but on the top results from all shards.

Use the rescore section in the _search request to rescore the top results from a kNN search.

Here is an example using the top level knn search with oversampling and using rescore to rerank the results:

				POST /my-index/_search
					{
  "size": 10,
  "knn": {
    "query_vector": [0.04283529, 0.85670587, -0.51402352, 0],
    "field": "my_int4_vector",
    "k": 20,
    "num_candidates": 50
  },
  "rescore": {
    "window_size": 20,
    "query": {
      "rescore_query": {
        "script_score": {
          "query": {
            "match_all": {}
          },
          "script": {
            "source": "(dotProduct(params.queryVector, 'my_int4_vector') + 1.0)",
            "params": {
              "queryVector": [0.04283529, 0.85670587, -0.51402352, 0]
            }
          }
        }
      },
      "query_weight": 0,
      "rescore_query_weight": 1
    }
  }
}
		

1. The number of results to return, note its only 10 and we will oversample by 2x, gathering 20 nearest neighbors.
2. The number of results to return from the KNN search. This will do an approximate KNN search with 50 candidates per HNSW graph and use the quantized vectors, returning the 20 most similar vectors according to the quantized score. Additionally, because this is the top-level knn object, the global top 20 results from all shards will be gathered before rescoring. Combining with rescore, this is oversampling by 2x, meaning gathering 20 nearest neighbors according to quantized scoring and rescoring with higher fidelity float vectors.
3. The number of results to rescore, if you want to rescore all results, set this to the same value as k
4. The script to rescore the results. Script score will interact directly with the originally provided float32 vector.
5. The weight of the original query, here we throw away the original score
6. The weight of the rescore query, here we only use the rescore query

You can use this option when you want to rescore on each shard and want more fine-grained control on the rescoring than the rescore_vector option provides.

Use rescore per shard with the knn query and script_score query . Generally, this means that there will be more rescoring per shard, but this can increase overall recall at the cost of compute.

				POST /my-index/_search
					{
  "size": 10,
  "query": {
    "script_score": {
      "query": {
        "knn": {
          "query_vector": [0.04283529, 0.85670587, -0.51402352, 0],
          "field": "my_int4_vector",
          "num_candidates": 20
        }
      },
      "script": {
        "source": "(dotProduct(params.queryVector, 'my_int4_vector') + 1.0)",
        "params": {
          "queryVector": [0.04283529, 0.85670587, -0.51402352, 0]
        }
      }
    }
  }
}
		

1. The number of results to return
2. The knn query to perform the initial search, this is executed per-shard
3. The number of candidates to use for the initial approximate knn search. This will search using the quantized vectors and return the top 20 candidates per shard to then be scored
4. The script to score the results. Script score will interact directly with the originally provided float32 vector.

• Approximate kNN search: Learn how to map, index, and run a basic approximate kNN search, including indexing considerations and limitations.
• Build search queries: Learn how to construct approximate kNN queries for filtering, hybrid retrieval, semantic search, multiple vector fields, and similarity thresholds.
• Nested kNN search: Learn how to run approximate kNN search on nested vectors for passage retrieval, filtering, inner hits, and chunked content.
• kNN search on Elasticsearch: Explore common use cases, prerequisites for kNN search, and a comparison of approximate and exact kNN methods.
• Exact kNN search: Learn how to run exact brute-force kNN search with script_score queries for small datasets or precise scoring.
• Tune approximate kNN search: Learn production sizing, memory, indexing, and cluster configuration for approximate kNN search.
• Vector search in Elasticsearch: Learn the core concepts and terminology for vector search in Elasticsearch, including embeddings, field types, and how vector retrieval fits with other search strategies.
• dense_vector field type: API reference for vector mapping options, including index_options and quantization settings.
• Knn query: API reference for the knn query, including parameters, query_vector_builder options, and usage with dense_vector and semantic_text fields.