Semantic search

Elastic Stack Serverless

Elasticsearch provides various semantic search capabilities using natural language processing (NLP) and vector search.

Learn more about use cases for AI-powered search in the overview page.

You have several options for using NLP models for semantic search in the Elastic Stack:

• Option 1: Use the semantic_text workflow (recommended)
• Option 2: Use the inference API workflow
• Option 3: Deploy models directly in Elasticsearch

This diagram summarizes the relative complexity of each workflow:

The simplest way to use NLP models in the Elastic Stack is through the semantic_text workflow. We recommend using this approach because it abstracts away a lot of manual work. All you need to do is create an inference endpoint and an index mapping to start ingesting, embedding, and querying data. There is no need to define model-related settings and parameters, or to create inference ingest pipelines. Refer to the Create an inference endpoint API documentation for a list of supported services.

For an end-to-end tutorial, refer to Semantic search with semantic_text.

The inference API workflow is more complex but offers greater control over the inference endpoint configuration. You need to create an inference endpoint, provide various model-related settings and parameters, define an index mapping, and set up an inference ingest pipeline with the appropriate settings.

For an end-to-end tutorial, refer to Semantic search with the inference API.

You can also deploy NLP in Elasticsearch manually, without using an inference endpoint. This is the most complex and labor intensive workflow for performing semantic search in the Elastic Stack. You need to select an NLP model from the list of supported dense and sparse vector models, deploy it using the Eland client, create an index mapping, and set up a suitable ingest pipeline to start ingesting and querying data.

For an end-to-end tutorial, refer to Semantic search with a model deployed in Elasticsearch.

• The elasticsearch-labs repo contains a number of interactive semantic search examples in the form of executable Python notebooks, using the Elasticsearch Python client
• Semantic search with ELSER using the model deployment workflow
• Semantic search with semantic_text

• Elasticsearch new semantic_text mapping: Simplifying semantic search
• Introducing Elastic Learned Sparse Encoder: Elastic's AI model for semantic search
• How to get the best of lexical and AI-powered search with Elastic's vector database
• Information retrieval blog series:
  • Part 1: Steps to improve search relevance
  • Part 2: Benchmarking passage retrieval
  • Part 3: Introducing Elastic Learned Sparse Encoder, our new retrieval model
  • Part 4: Hybrid retrieval