﻿---
title: GenAI Search - High Availability
description: This reference architecture illustrates a production-grade, highly available GenAI search solution built on Elasticsearch. It shows the physical deployment...
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/reference-architectures/genai-search-high-availability
products:
  - Elastic Cloud Enterprise
  - Elastic Cloud Hosted
  - Elastic Cloud on Kubernetes
applies_to:
  - Elastic Cloud Hosted: Generally available
  - Elastic Cloud on Kubernetes: Generally available
  - Elastic Cloud Enterprise: Generally available
  - Self-managed Elastic deployments: Generally available
---

# GenAI Search - High Availability
This reference architecture illustrates a production-grade, highly available GenAI search solution built on Elasticsearch. It shows the physical deployment model, logical integration points, and key best practices for implementing a retrieval layer that grounds generative AI responses.
Elasticsearch can combine [lexical search](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/solutions/search/full-text), [dense vector search](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/solutions/search/vector/dense-vector), [sparse vector search](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/solutions/search/vector/sparse-vector), temporal and geospatial filtering, and hybrid ranking techniques. These capabilities form the foundation for [Retrieval Augmented Generation (RAG)](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/solutions/search/rag), [agentic workflows](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/ai-features/elastic-agent-builder), and AI-assisted applications.

## GenAI search use cases

The GenAI search – high availability architecture is intended for organizations that:
- Require high-performance, low-latency retrieval across large, diverse datasets with highly relevant results at scale.
- Need lexical, vector, semantic, temporal, hybrid, or multimodal search across text, code, images, video, and geospatial content.
- Power assistants, agents, and agentic workflows using RAG and MCP, where grounding models in the most relevant information is essential. RAG in Elastic is explicitly built around retrieving relevant context, and MCP is an open standard for connecting AI applications to external data and tools.
- Integrate with foundation models and LLM frameworks, while improving relevance with re-ranking, filtering, faceting, highlighting, personalization, and metadata-aware retrieval.
- Support secure multi-tenant deployments, agent memory, and domain copilots such as observability and SOC assistants.


## Vector search optimization

Elasticsearch supports multiple vector search execution models and optimizations, allowing this architecture to balance recall, latency, and cost based on workload requirements.
- [Better Binary Quantization (BBQ)](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq) is Elastic’s quantization technology that dramatically reduces vector memory footprint while preserving high recall, enabling large-scale in-memory vector search at lower cost.
- [DiskBBQ](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-disk) extends BBQ by efficiently paging vector data from disk, providing predictable performance even when the full vector working set does not fit in memory and enabling cost-efficient scaling for very large datasets.
- [HNSW](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-hnsw) is used where maximum recall and lowest tail latency are required and sufficient memory is available to keep vector indexes resident in RAM.
- [ACORN](https://www.elastic.co/search-labs/blog/elasticsearch-9-1-bbq-acorn-vector-search) further optimizes filtered vector search by reducing unnecessary graph traversals, making it especially effective for high-selectivity queries common in security, observability, and multi-tenant environments.

Together, these approaches allow the architecture to be tuned precisely for different latency, recall, update, and cost constraints without changing the overall system design.

## Architecture

The following architecture describes a multi-availability-zone deployment designed for continuous availability, fault tolerance, and predictable performance under sustained query and ingestion load. It applies to both time-series and non-time-series use cases and supports a broad set of GenAI patterns including assistants, autonomous agents, observability copilots for SREs, and SOC analysis platforms.
![GenAI Search high availability logical diagram](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/images/genai-search-ha-logical-diagram.png)

![GenAI Search high availability physical diagram](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/images/genai-search-ha-physical-diagram.png)

<important>
  This architecture employs a single uniform hot/content data tier, as most search and generative AI workloads require very low latency across the full corpus, regardless of data age. However, disk-based vector storage methods such as [DiskBBQ](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-disk) (Elastic’s [advanced evolution of IVF](https://www.elastic.co/search-labs/blog/diskbbq-elasticsearch-introduction)) offer a memory-efficient alternative to HNSW that can support larger datasets, such as IoT telemetry and financial transaction logs, on lower-cost tiers.
</important>

The physical deployment architecture for GenAI applications is built around a resilient Elasticsearch cluster deployed across three availability zones (AZ). For production-grade deployments, two AZs are the minimum, with three AZs strongly recommended to maximize high availability and fault tolerance. In Elastic Cloud, shards are automatically distributed across zones, ensuring that primaries and replicas never reside in the same AZ.
In self-managed deployments, [shard allocation awareness](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/distributed-architecture/shard-allocation-relocation-recovery/shard-allocation-awareness) and forced awareness should be configured to achieve the same resiliency. Kibana instances are deployed in each AZ to not become a single point of failure. And, in some use cases, it should be fronted by a load balancer. For example, Elastic MCP server is a foundational component of Agent Builder. It enables agentic workflows through API using natural language to query Elasticsearch. As a result, high volume programmatic access is likely required and should be balanced across multiple nodes.
All data nodes host both primary and replica shards, with replicas contributing both to redundancy and to query throughput. High-performance SSD-backed storage is recommended, along with memory-rich nodes to support vector indexes, hybrid lexical and vector search, and high concurrent load. Machine learning nodes are also distributed across zones to support optional embedding generation using self-hosted models such as Elastic’s Jina text embedding models.
In addition, the [Elastic Inference Service (EIS)](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/elastic-inference/eis) is provided as part of Elastic Cloud, which uses GPUs for high speed vector embedding generation. Finally, snapshots are stored externally for disaster recovery, with Snapshot Lifecycle Management (SLM) handling automation.

## Recommended hardware specifications

The following specifications are recommended for clusters that are self-deployed on-prem or self-deployed in a cloud provider. With Elastic Cloud Hosted, you can deploy clusters in AWS, Azure, or Google Cloud. Available hardware types and configurations vary across providers, but each offers instance families that meet the performance needs of search and generative AI applications. For details, refer to our documentation on [Elastic Cloud Hosted hardware](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/hardware) for AWS, Azure, and GCP.
In the following table, the "Physical" column provides approximate per-node guidance when self-deploying Elasticsearch in your own data center, based on equivalent CPU, RAM, and storage profiles.
Elastic has performance-tested hardware profiles across the major cloud providers to identify the optimal balance for each node type. Significantly deviating from these tested ratios may appear to reduce costs, but typically leads to degraded performance, query latency spikes, or search scalability.
Dedicated ML nodes are needed when inference is performed within the cluster, such as running [ELSER](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/machine-learning/nlp/ml-nlp-elser), [Elastic Jina AI Models] (explore-analyze/machine-learning/nlp/ml-nlp-jina.md), or [custom transformer models](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/machine-learning/nlp/ml-nlp-model-ref) locally. These nodes should be provisioned with sufficient memory to load models into RAM. When inference is offloaded to an LLM or embedding model external to the cluster (for example, [Elastic Inference Service](https://docs-v3-preview.elastic.dev/elastic/kibana/tree/main/reference/connectors-kibana/elastic-managed-llm), Azure OpenAI, Anthropic, or Bedrock), dedicated ML nodes are not required.
For GenAI search workloads in Elastic Cloud Hosted, use **Vector Search Optimized** profiles as the primary reference, and consider **CPU Optimized** profiles for workloads with higher CPU and disk requirements.

| Type        | AWS            | Azure          | GCP         | Physical                                     |
|-------------|----------------|----------------|-------------|----------------------------------------------|
| hot/content | `r6gd`, `c8gd` | `lsv3`, `fsv2` | `N2d`, `N2` | 16-32 vCPU, 64-256 GB RAM, 0.5-6 TB NVMe SSD |
| ml          | `m5dn`         | `fsv2`         | `N2`        | 16-32 vCPU, 64+ GB RAM, 256 GB+ SSD          |
| master      | `c8gd`         | `fsv2`         | `N2`        | 4 vCPU, 16 GB RAM, 256 GB SSD                |
| kibana      | `c8gd`         | `fsv2`         | `N2`        | 8-16 vCPU, 8-16 GB RAM, 256 GB SSD           |

These recommendations provide a practical baseline, but available instance families evolve over time as newer provider hardware becomes available. For additional guidance on selecting Elastic Cloud hardware profiles for specific workloads, refer to:
- [Selecting the right configuration for you on AWS](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/ec_selecting_the_right_configuration_for_you)
- [Selecting the right configuration for you on Azure](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/ec-azure-configuration-choose)
- [Selecting the right configuration for you on GCP](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/ec-gcp-configuration-choose)

For the full list of currently available instance configurations, refer to:
- [AWS default instances](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/aws-default)
- [Azure default instances](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/azure-default)
- [GCP default instances](https://docs-v3-preview.elastic.dev/elastic/cloud/tree/master/reference/cloud-hosted/gcp-default-provider)


## Vector search considerations


### Storage and memory

Approximate nearest-neighbor (ANN) algorithms are dominated by latency-bound memory access patterns, meaning performance is typically limited by data in memory rather than raw vector computation. As a result, all vector distance computations occur in off-heap RAM and CPU. To make large-scale in-memory vector search feasible, Elasticsearch supports [many quantization techniques](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/dense-vector#dense-vector-quantization) for up to a 32x reduction in vector footprint in RAM, and defaults to HNSW with [Better Binary Quantization (BBQ)](https://www.elastic.co/search-labs/blog/better-binary-quantization-lucene-elasticsearch). BBQ is Elastic’s memory-efficient approach for maximizing recall with a low vector memory footprint.
To increase off-heap memory available for vector search, you have the following options:
- Increase RAM per node to expand the off-heap working set.
- Slightly reduce the JVM heap size per node to free additional off-heap memory. If you choose this approach, benchmark (for example with [Elastic Rally](https://esrally.readthedocs.io/en/stable/)) to confirm the smaller heap remains sufficient under peak workload.

When vector quantization is enabled, Elasticsearch stores both the original `float32` vectors and their quantized representations on disk. Retaining the `float32` vectors preserves flexibility for re-ranking, re-indexing, and future model changes. Only the quantized vectors are loaded into memory for search.
- Use [HNSW](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-hnsw) when you are optimizing for 99%+ recall and absolute lowest tail latency, and you can afford the RAM/off-heap footprint (and you’re not constantly updating the index).
- Use [DiskBBQ](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-disk) when you’re cost/memory sensitive, your recall target is more like around 95% and you want performance that doesn’t drop when the working set no longer fits RAM.
- Use [BBQ flat](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-flat) (BBQ without HNSW) when filters reduce comparisons to < ~100k vectors; typically lowest operational complexity.


### Vector sizing

- To learn about sizing formulas refer to [Ensure data nodes have enough memory](/elastic/docs-content/pull/5498/deploy-manage/production-guidance/optimize-performance/approximate-knn-search#_ensure_data_nodes_have_enough_memory). Use the formulas to calculate the disk needs.
- For HNSW, the amount of RAM configured must match the amount required by the vector index storage.
- For DiskBBQ, you can provision as little as 5% of disk storage as RAM, as DiskBBQ swaps to disk effectively. However, query performance improves significantly when more RAM is provisioned for vector search, especially for large volumes of data.


### Generating vector embeddings

- If you'll be generating embeddings on Elastic ML nodes, capacity can be [autoscaled](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/autoscaling) using Elastic Cloud Hosted (ECH) or Elastic Cloud on Kubernetes on-prem.
- Elastic provides [world class vector models](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/machine-learning/nlp/ml-nlp-jina) from Jina AI that can be used for NLP embeddings, and a re-ranker that can be either self-hosted on ML nodes or used through [Elastic Inference Service](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/elastic-inference/eis).
- Embeddings can also be generated with [Elastic’s inference API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-inference-put) from many external model providers, or from internally or externally hosted foundation models.


### kNN search performance tuning

- **Recall:** General guidance is to start with defaults → measure → [increase num_candidates](https://www.elastic.co/search-labs/blog/elasticsearch-knn-and-num-candidates-strategies) (oversample) → optionally add a [re-ranker](https://www.elastic.co/search-labs/blog/elastic-semantic-reranker-part-2).
- **Runtime:** Refer to [Tune approximate kNN search](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/production-guidance/optimize-performance/approximate-knn-search) to understand all the vector search performance optimizations to ensure high speed search.


### Updating vector data

Updating dense or sparse vector data can be more resource-intensive than updating keyword-based fields, since embeddings often need to be regenerated. For applications with frequent document updates, plan for additional indexing throughput and consider whether embeddings should be pre-computed, updated asynchronously, or generated on demand.

## General considerations


### Hybrid Retrieval

- Over-fetch on both retrieval paths. Increase size, use RRF with a larger `rank_window_size`, and raise `num_candidates` on kNN so ANN does not miss critical neighbors. Then trim results to find the best recall vs. latency balance.
- Keep `minimum_should_match` loose, and rely on proper analyzers and multi-fields (text, .keyword, shingles/ngrams). Hybrid search cannot recover documents BM25 never retrieves.
- The easiest framework for executing hybrid search on Elastic is [retrievers](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/solutions/search/retrievers-overview) or [ES|QL FORK and FUSE](/elastic/docs-content/pull/5498/solutions/search/esql-for-search#fork-and-fuse).


### Multi-AZ deployment

Three availability zones is ideal, but at least two availability zones are recommended to ensure resiliency. This helps guarantee that there will be data nodes available in the event of an AZ failure. Elastic Cloud automatically distributes replicas across zones, while self-managed clusters should use shard allocation awareness to achieve the same.

### Shard management

Proper shard management is foundational to maintaining performance at scale. This includes ensuring an even distribution of shards across nodes, choosing an appropriate shard size, and managing shard counts as the cluster grows. For detailed guidance, refer to [Size your shards](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/production-guidance/optimize-performance/size-shards).
- Recommendations for time series data can be found in the [Hot/Frozen architecture](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/reference-architectures/hotfrozen-high-availability).
- **Size shards intentionally:** estimate total index size and choose primaries that land in the **~10–50 GB, <200M docs** range. This is the natural shard count for a catalog; don’t add shards unless search metrics demand it.
- **Plan scale at creation:** catalogs don’t roll over, so set shard math up front. If growth is possible, configure routing shards so the index can be **split later by clean multiples** (read-only + extra IO/disk required).
- **Scale with replicas, not primaries:** increase **replicas** first to improve search throughput and availability. Replicas increase concurrency by spreading queries across shard copies; **fan-out is still driven by primary shard count**.


### Snapshots

Regular backups are essential when indexing business-critical or auditable data. [Snapshots](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/tools/snapshot-and-restore) provide recoverability in case of data corruption or accidental deletion. For production workloads, configure automated snapshots through Snapshot Lifecycle Management (SLM) and integrate them with [Index Lifecycle Management (ILM)](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/manage-data/lifecycle/index-lifecycle-management) policies.

### Kibana, MCP, and Agent Builder

- If deploying outside of Elastic Cloud, ensure [Kibana is configured for high availability](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/production-guidance/kibana-load-balance-traffic) to avoid a single point of failure. Deploying Kibana across multiple availability zones also improves resiliency for management and user access.
- Ensure [monitoring data](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/monitor) is collected from Kibana nodes if the [MCP server](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/explore-analyze/ai-features/agent-builder/mcp-server) is used. This allows for proper capacity planning if usage is extensive.
- For self-deployed clusters, use a proxy in front of multiple Kibana nodes to avoid a single point of failure. Elastic Cloud Hosted provides a single connection endpoint that is backed by a highly available proxy layer.


### Data tiering

This architecture uses a single, uniform hot/content data tier, which is recommended for most search and generative AI workloads. Keeping all data on a single tier simplifies operations and helps ensure consistent query latency, especially for vector and hybrid search.
Elasticsearch also supports multi-tier architectures that place data on different tiers based on access patterns and retention requirements. In some scenarios, disk-based vector storage methods such as [DiskBBQ](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/mapping-reference/bbq#bbq-disk) (Elastic’s [advanced evolution of IVF](https://www.elastic.co/search-labs/blog/diskbbq-elasticsearch-introduction)) provide a memory-efficient alternative to HNSW. This approach can support larger datasets, such as IoT telemetry or financial transaction logs, while allowing vector data to be stored efficiently on lower-cost storage tiers.
For guidance on designing multi-tier architectures and selecting appropriate data tiers for your workload, refer to the [Elasticsearch documentation on data tiers](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/manage-data/lifecycle/data-tiers) and the [Hot/Frozen reference architecture](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/reference-architectures/hotfrozen-high-availability). If you need help estimating capacity or choosing a configuration for your scenario, you can also [contact us](https://www.elastic.co/contact).

## Resources and references

- [Vector search in Elasticsearch](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/solutions/search/vector)
- [Run Elasticsearch in production](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/production-guidance/elasticsearch-in-production-environments)
- [Run Elasticsearch in production](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/production-guidance/elasticsearch-in-production-environments)
- [Size your shards](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/5498/deploy-manage/production-guidance/optimize-performance/size-shards)