ES|QL FUSE command

The FUSE processing command merges rows from multiple result sets and assigns new relevance scores.

FUSE enables hybrid search to combine and score results from multiple queries, together with the FORK command.

FUSE works by:

1. Merging rows with matching <key_columns> values
2. Assigning new relevance scores using the specified <fuse_method> algorithm and the values from the <group_column> and <score_column>

A LIMIT is required before FUSE, because FUSE can only work with a finite set of rows.

Use default parameters:

FUSE
		

Specify custom parameters:

FUSE <fuse_method> SCORE BY <score_column> GROUP BY <group_column> KEY BY <key_columns> WITH <options>
		

fuse_method

Defaults to RRF. Can be one of RRF (for Reciprocal Rank Fusion) or LINEAR (for linear combination of scores). Designates which method to use to assign new relevance scores.

options

Options for the fuse_method.

score_column

Defaults to _score. Designates which column to use to retrieve the relevance scores of the input row and where to output the new relevance scores of the merged rows.

group_column

Defaults to _fork. Designates which column represents the result set.

key_columns

Defaults to _id, _index. Rows with matching key_columns values are merged.

The following examples use FORK to run parallel queries and FUSE to merge the results.

Run a lexical and a semantic query in parallel with FORK, then merge with FUSE (applies RRF by default):

FROM books METADATA _id, _index, _score
| FORK (WHERE title:"Shakespeare" | SORT _score DESC | LIMIT 100)
       (WHERE semantic_title:"Shakespeare" | SORT _score DESC | LIMIT 100)
| FUSE
| SORT _score DESC
		

1. Include document ID, index name, and relevance score
2. Fork 1: Lexical search on title field, sorted by relevance score
3. Fork 2: Semantic search on semantic_title field, sorted by relevance score
4. Merge results using RRF algorithm by default
5. sort results by the new scores, since `FUSE` does not do any sorting.

FUSE can also use linear score combination:

FROM books METADATA _id, _index, _score
| FORK (WHERE title:"Shakespeare" | SORT _score DESC | LIMIT 100)
       (WHERE semantic_title:"Shakespeare" | SORT _score DESC | LIMIT 100)
| FUSE LINEAR
| SORT _score DESC
		

1. Fork 1: Lexical search on title
2. Fork 2: Semantic search on semantic_title
3. Merge results using linear combination of scores (equal weights by default)
4. sort results by the new scores, since `FUSE` does not do any sorting.

When combining results from semantic and lexical queries through linear combination, we recommend first normalizing the scores from each result set.

The following example uses minmax score normalization. This means the scores normalize and assign values between 0 and 1, before combining the rows:

FROM books METADATA _id, _index, _score
| FORK (WHERE title:"Shakespeare" | SORT _score DESC | LIMIT 100)
       (WHERE semantic_title:"Shakespeare" | SORT _score DESC | LIMIT 100)
| FUSE LINEAR WITH { "normalizer": "minmax" }
| SORT _score DESC
		

1. Fork 1: Lexical search
2. Fork 2: Semantic search
3. Linear combination with min-max normalization (scales scores to 0-1 range)
4. sort results by the new scores, since `FUSE` does not do any sorting.

FUSE allows you to specify different weights to scores, based on the _fork column values, enabling you to control the relative importance of each query branch in the final results.

FROM books METADATA _id, _index, _score
| FORK (WHERE title:"Shakespeare" | SORT _score DESC | LIMIT 100)
       (WHERE semantic_title:"Shakespeare" | SORT _score DESC | LIMIT 100)
| FUSE LINEAR WITH { "weights": { "fork1": 0.7, "fork2": 0.3 }, "normalizer": "minmax" }
| SORT _score DESC
		

1. Fork 1: Lexical search
2. Fork 2: Semantic search
3. Weighted linear combination: 70% lexical, 30% semantic, with min-max normalization
4. sort results by the new scores, since `FUSE` does not do any sorting.

These limitations can be present either when:

• FUSE is not combined with FORK

• FUSE doesn't use the default metadata columns _id, _index, _score and _fork

  1. FUSE assumes that key_columns are single valued. When key_columns are multivalued, FUSE can produce unreliable relevance scores.
  2. FUSE automatically assigns a score value of NULL if the <score_column> or <group_column> are multivalued.
  3. FUSE assumes that the combination of key_columns and group_column is unique. If not, FUSE can produce unreliable relevance scores.