Cross-project search

Cross-project search (CPS) enables you to run a single search request across multiple Serverless projects. When your data is split across projects to organize ownership, use cases, or environments, cross-project search lets you query all that data from a single place, without having to search each project individually.

Cross-project search relies on linking projects within your Elastic Cloud organization. After you link projects together, searches from the origin project automatically run across all linked projects.

This overview explains how cross-project search works, including project linking, search expressions, tags, and project routing.

Projects are intended to act as logical namespaces for data, not hard boundaries for querying it. You can split data into projects to organize ownership, use cases, or environments, while still expecting to search and analyze that data from a single place.

Because of this, after you link additional projects to your current (origin) project, all searches from the origin project query every linked project by default. Searches are designed to run across projects automatically, providing the same experience for querying, analysis, and insights across projects as within a single project. Restricting search scope is always possible, but it requires explicitly scoping the search request using qualified expressions or routing parameters.

In Serverless, projects can be linked together. The project from which links are created is called the origin project, and the connected projects are referred to as linked projects.

The origin project is the project you are currently working in and from which you run cross-project searches. Linked projects are other projects that are connected to the origin project and whose data can be searched from it.

After you link projects, searches that you run from the origin project are no longer local to the origin project by default. Any search initiated on the origin project automatically runs across the origin project and all its linked projects (cross-project search).

When you search from an origin project, the query runs against its linked projects automatically unless you explicitly change the query scope by using project routing expressions or qualified index expressions.

Project linking is not bidirectional. Searches initiated from a linked project do not run against the origin project.

You can link projects by using the Elastic Cloud UI. For step-by-step instructions, refer to Link projects for cross-project search.

Each project has a unique project ID and a project alias. The project alias is derived from the project name and can be modified.

The project ID uniquely identifies a project and is system-generated.

The project alias is a human-readable identifier derived from the project’s connection alias. If you want to change the project alias, you must update the connection alias of the linked project.

While both the project ID and project alias uniquely identify a project, cross-project search uses project aliases in index expressions. Project aliases are intended to be user-friendly and descriptive, making search expressions easier to read and maintain.

In addition to using a project alias, CPS provides a reserved identifier, _origin, that always refers to the origin project of the search. You can use _origin in search expressions to explicitly target the origin project, without having to reference its specific project alias. Refer to Qualified and unqualified search expressions for detailed examples and to learn more.

This section explains how search works in CPS, including:

• the CPS search model
• unqualified search expressions (for example, logs and logs*), qualified search expressions (expressions with a project alias prefix, for example project1:logs) and how they control search scope
• how search options such as ignore_unavailable and allow_no_indices behave in CPS
• common edge cases and examples involving mixed qualified and unqualified expressions

With CPS, searches are resolved across all linked projects by default—not just the origin project. You explicitly need to limit the scope of your search to override this behavior. Refer to the Unqualified and qualified search expressions section to learn more. When you refer to a resource (such an index, a data stream, or an alias) by a name, CPS resolves that name across the origin project and all of its linked projects. This means that when you run a search from the origin project and refer to a searchable resource such as logs, the search is executed against all resources named logs across the origin project and its linked projects, for example:

				GET logs/_search
		

For each linked project, the search runs only if a resource named logs exists. If a linked project does not have a logs resource, that project is skipped and the search continues without returning an error. No error is returned as long as at least one project has the logs resource.

Cross-project search supports two types of search expressions: unqualified and qualified. The type of search expression determines where a search request runs and how errors are handled.

• Unqualified search expressions follow the cross-project search model and represent the default, native behavior in CPS. An unqualified search expression does not include a project alias prefix. In this case, the search runs against the origin project and all its linked projects.
• Qualified search expressions explicitly override the default behavior, enabling you to precisely control which projects a search runs on and how errors are handled. It includes additional qualifiers, such as project alias prefixes, that explicitly control the scope of the search.

For example, the following qualified search expression request searches only the origin project:

				GET _origin:logs/_search
		

For additional examples of qualified search expressions, refer to the examples section.

The distinction between qualified and unqualified search expressions affects how the ignore_unavailable and allow_no_indices search options are applied in cross-project search. When you use an unqualified expression, index resolution is performed against the merged project view. In this case, search options are evaluated based on whether the target resources exist in any of the searched projects, not only in the origin project.

Project routing expressions do not affect the behavior of the ignore_unavailable or allow_no_indices settings.

The following describes the standard Elasticsearch behavior:

ignore_unavailable defaults to false. When set to false, the request returns an error if it targets a missing resource (such as an index or data stream). When set to true, missing resources are ignored and the request returns an empty result instead of an error. For example, if the logs index does not exist, the following request returns an error because the default value is false:

				GET logs/_search
		

allow_no_indices defaults to true. When set to true, the request succeeds and returns an empty result if it targets a missing resource. When set to false, the request returns an error if any wildcard expression, index alias, or _all value does not resolve to an existing resource.

For example, if no indices match logs*, the following request returns an empty result because the default value is true:

				GET logs*/_search
		

When you use a qualified search expression, the default behavior of ignore_unavailable and allow_no_indices outlined above applies independently to each qualified project.

The next section explains how this behavior differs when using unqualified search expressions in CPS.

When you use an unqualified search expression, the behavior is different:

• As long as the targeted resources exist in at least one of the searched projects, the request succeeds, even if ignore_unavailable or allow_no_indices are set to false.
• The request returns an error only if:
  • the targeted resources are missing from all searched projects, or
  • a search expression explicitly targets a specific project and the resource is missing from that project.

You have two projects linked to your origin project: project1 and project2. Resources:

• origin has a logs index
• project1 has a metrics index
• project2 has a books index

The following request succeeds, even with ignore_unavailable=false:

				GET logs,metrics/_search?ignore_unavailable=false
		

Although logs is not present in project2 and metrics is not present in origin, each index exists in at least one searched project, so the request succeeds.

If the projects have the following resources, however:

• origin has a metrics index
• project1 has a metrics index
• project2 has a books index

The following request returns an error:

				GET logs,metrics/_search?ignore_unavailable=false
		

In this case, the logs index does not exist in any of the searched projects, so the request fails.

In the next example, the request combines qualified and unqualified index expressions. Resources:

• origin has a logs index
• project1 has a metrics index
• project2 has a books index

The following request returns an error:

				GET logs,project2:metrics/_search?ignore_unavailable=false
		

Because the request explicitly targets project2 for the metrics index using a qualified expression and ignore_unavailable is set to false, the entire request returns an error, even though the logs index exists in one of the projects.

Refer to the examples section for more.

You can assign tags to projects and use them to control cross-project search behavior.

CPS supports two kinds of project tags:

• Predefined tags, which are provided by Elastic and describe built-in project metadata.
• Custom tags, which you define and manage to organize projects according to your own needs. These tags are managed in the Elastic Cloud UI.

Only custom tags can be added, modified, or removed. Predefined tags are always available and cannot be changed.

With tags, you can:

• route API calls to specific projects based on tag values
• include tag values in search or ES|QL results to identify which project each document came from
• filter and aggregate results using tags

The following tags are predefined:

• _alias: the project alias
• _csp: the cloud service provider
• _id: the project identifier
• _organization: the organization identifier
• _region: the Cloud region where the project is located
• _type: the project type (Observability, Search, Security)

Predefined tags always start with an underscore _.

There are two ways to use tags in CPS:

• project routing
• queries

Project routing enables you to limit a search to a subset of projects, including the origin project and linked projects, based on tag values.

When you use project routing, the routing decision is made before the search request is performed. Based on the specified tags, CPS determines which projects the query is sent to, and the search is performed only on those projects.

The project_routing parameter is available on all CPS-enabled endpoints. Refer to the Cross-project search > Supported APIs for a full list of endpoints.

For example, the following API request searches the logs resource only on projects that have the _alias:my_search_project tag.

				GET logs/_search 
					{
  "project_routing": "_alias:my_search_project"
}
		

Refer to the examples section for more.

You can also use project tags within a search query. In this case, tags are treated as query-time metadata fields, not as routing criteria. You can explicitly request project tags to be included in search results. For both _search and ES|QL, you must request one or more tags to include them in the response.

For example, with the _search endpoint:

				GET logs/_search
					{
  "fields": ["*", "_project.mytag", "_project._region"]
}
		

For example, with ES|QL:

				GET /_query
					{
  "query": "FROM logs METADATA _project._csp, _project._region | ..."
}
		

In both cases, the returned documents include the requested project metadata, which lets you identify which project each document originated from.

You can also use project tags in queries to filter, sort, or aggregate search results. Unlike project routing, using tags inside a query does not affect which projects the query is sent to. It only affects which results are returned. The routing decision has already been made before the query is performed.

For example, the following request aggregates results by cloud service provider:

				GET foo/_search
					{
 "query": { ... }
 "aggs": {
    "myagg": {
      "terms": {
        "field": "_project._csp"
      }
    }
  }
}
		

When you use project tags in ES|QL, you must explicitly include them in the METADATA clause. This is required not only to return tag values in the results, but also to use them in the query for filtering, sorting, or aggregation.

For example, the following ES|QL query counts documents per project alias:

				
					FROM logs* METADATA _project._alias | STATS COUNT(*) by _project._alias
		

The following APIs support cross-project search:

• Async search
• Count and CAT count
• ES|QL query and ES|QL async query
• EQL search
• Field capabilities
• Multi search
• Multi search template
• PIT (point in time) close, open
• Reindex
• Resolve Index API
• SQL
• Search
• Search a vector tile
• Search scroll clear, run
• Search template

Currently, each origin project can have up to 20 linked projects. A linked project can be associated with any number of origin projects.