Loading

Cross-project search

Cross-project search (CPS) enables you to run a single search request across multiple Serverless projects. When your data is split across projects to organize ownership, use cases, or environments, cross-project search lets you query all that data from a single place, without having to search each project individually.

Cross-project search relies on linking projects within your Elastic Cloud organization. After you link projects together, searches from the origin project automatically run across all linked projects.

This overview explains how cross-project search works, including project linking and security. For prerequisites, compatibility requirements, architecture planning, and scope defaults, refer to Configure cross-project search in Deploy and manage. For details on how search, tags, and project routing work in CPS, refer to the following pages:

Projects are intended to act as logical namespaces for data, not hard boundaries for querying it. You can split data into projects to organize ownership, use cases, or environments, while still expecting to search and analyze that data from a single place.

After you link projects, searches from the origin project run across the origin and all linked projects by default. Searches are designed to run across projects automatically, providing the same experience for querying, analysis, and insights across projects as within a single project. Restricting search scope is always possible, but it requires explicitly scoping the search request using qualified expressions, the _origin identifier, or routing parameters.

In Serverless, projects can be linked together.

Cross-project search runs across origin and linked projects within your Elastic Cloud organization:

  • Origin project: The base project where you create links and run cross-project searches.
  • Linked projects: The projects you connect to the origin project. Data in the linked projects becomes searchable from the origin project.

After you link projects, searches that you run from the origin project are no longer local to the origin project by default. Any search initiated on the origin project automatically runs across the origin project and all its linked projects (cross-project search).

When you search from an origin project, the query runs against its linked projects automatically unless you explicitly change the query scope by using project routing expressions or qualified index expressions.

Project linking is not bidirectional. Searches initiated from a linked project do not run against the origin project. If you need bidirectional search, link the projects twice, in both directions.

You can link projects by using the Elastic Cloud UI. For step-by-step instructions, refer to Link projects for cross-project search.

Each project has a unique project ID and a project alias. The project alias is derived from the project name and can be modified.

The project ID uniquely identifies a project and is system-generated.

The project alias is a human-readable identifier derived from the project's connection alias. If you want to change the project alias, you must update the connection alias of the linked project.

While both the project ID and project alias uniquely identify a project, cross-project search uses project aliases in index expressions. Project aliases are intended to be user-friendly and descriptive, making search expressions easier to read and maintain.

In addition to using a project alias, CPS provides a reserved identifier, _origin, that always refers to the origin project of the search. You can use _origin in search expressions to explicitly target the origin project, without having to reference its specific project alias. Refer to Qualified and unqualified search expressions for detailed examples and to learn more.

This section gives you a high-level overview of how security works in cross-project search.

In CPS, access to a project's data is determined by the roles assigned to you in that project. Your access does not change based on how you perform a search: whether you query directly within a project or access it through cross-project search, the same permissions apply.

Note

Cross-project search is not available when performing programmatic searches using Elasticsearch API keys, since they're project-scoped and they return results from the local project only.

Access control operates in two stages:

  • Authentication verifies the identity associated with a request (for example, a Cloud user or API key) and retrieves that identity's role assignments in each project.
  • Authorization evaluates those roles to determine which actions and resources the request can access within each project.

For example, if you have a viewer role in project 1, an admin role in project 2, and a custom role in project 3, you can access all three projects through cross-project search. Each project enforces the permissions associated with the role you have in that project.

When a cross-project search query targets a linked project that you have access to, authorization checks are performed locally in that project to determine whether you have the required privileges to access the requested resources.

Example You have read access to the logs index in project 1, but no access to the logs index in project 2. If you run GET logs/_search:

  • documents from the logs index in project 1 are returned
  • the logs index in project 2 is not accessible and is excluded from the results

The following APIs support cross-project search:

  • Maximum of 20 linked projects: Each origin project can have up to 20 linked projects. A linked project can be associated with any number of origin projects.
  • Chaining/transitivity not supported: If Project A links to Project B, and Project B links to Project C, Project A cannot automatically search Project C. Each link is independent.
  • Links are unidirectional: Searches that run from a linked project do not run against the origin project. If you need bidirectional search, link the projects twice, in both directions.
  • System indices are excluded: System indices (such as .security and .fleet-*) are excluded from cross-project search.
  • Unavailable APIs: _transform and _fleet_search requests do not support CPS.
  • Workplace AI projects: Workplace AI projects are not compatible with cross-project search.
  • New projects only: During technical preview, only newly created projects can function as origin projects.
  • ML and transforms: ML anomaly detection jobs and transforms are not supported in the technical preview. They continue to run on origin project data only.
  • Failure store: 🚧 TODO

For administrator-focused details including compatibility, architecture patterns, and feature impacts, refer to Configure cross-project search in Deploy and manage.

The following examples demonstrate how search requests behave in different CPS scenarios.

In the following example, an origin project and a linked project both contain an index named my-index.

				GET /my-index/_search
					{
  "size": 2,
  "query": {
    "match_all": {}
  }
}

The request will return a response similar to this:

				
					{
  "took": 34,
  "timed_out": false,
  "num_reduce_phases": 3,
  "_shards": {
    "total": 12,
    "successful": 12,
    "skipped": 0,
    "failed": 0
  },
  "_clusters": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "running": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "_origin": {
        "status": "successful",
        "indices": "my-index",
        "took": 21,
        "timed_out": false,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      },
      "linked_project": {
        "status": "successful",
        "indices": "my-index",
        "took": 5,
        "timed_out": false,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "linked_project:my-index",
        "_id": "IH-mupwBMZyy2F9u2IQz",
        "_score": 1.0,
        "_source": {
          "project": "linked"
        }
      },
      {
        "_index": "my-index",
        "_id": "u0SnupwBaOrMOsBImb7G",
        "_score": 1.0,
        "_source": {
          "project": "origin"
        }
      }
    ]
  }
}

In this example, both the origin project and a linked project contain an index named my-index:

				POST /_query
					{
 "query": "FROM my-index",
  "include_execution_metadata": true
}

The query will return a response similar to this:

				
					{
  "took": 39,
  "is_partial": false,
  "completion_time_in_millis": 1772659251830,
  "documents_found": 2,
  "values_loaded": 4,
  "start_time_in_millis": 1772659251791,
  "expiration_time_in_millis": 1773091251753,
  "columns": [
    {
      "name": "project",
      "type": "text"
    },
    {
      "name": "project.keyword",
      "type": "keyword"
    }
  ],
  "values": [
    [
      "origin",
      "origin"
    ],
    [
      "linked",
      "linked"
    ]
  ],
  "_clusters": {
    "total": 2,
    "successful": 2,
    "running": 0,
    "skipped": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "_origin": {
        "status": "successful",
        "indices": "my-index",
        "took": 39,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      },
      "linked_project": {
        "status": "successful",
        "indices": "my-index",
        "took": 23,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  }
}

These requests don’t include a project prefix. The my-index index is searched in the origin project and in the linked project.

Search limited to the origin project:

				GET _origin:my-index/_search
		
				POST /_query
					{
  "query": "FROM _origin:my-index | LIMIT 10"
}

The requests include the _origin prefix. Only the origin project is searched.

Search across all projects using a wildcard expression:

				GET *:my-index/_search
		
				POST /_query
					{
  "query": "FROM *:my-index | LIMIT 10"
}

The requests explicitly target all projects using the *: prefix. The my-index index is evaluated separately in each project. The index my-index must exist in every project, otherwise the search returns an error.

In the following example, there is an origin project and a linked project. The origin project contains one index, my-index. The linked project contains two indices: my-index and logs.

The following request searches all indices on projects whose alias starts with "lin".

				GET /*/_search
					{
  "project_routing":"_alias:lin*",
  "query": {
    "match_all": {}
  }
}
		
				GET /_query
					{
  "query": "SET project_routing=\"_alias:lin*\"; FROM * METADATA _index",
  "include_execution_metadata":true
}

The request will return a response similar to this:

				
					{
  "took": 60,
  "timed_out": false,
  "_shards": {
    "total": 12,
    "successful": 12,
    "skipped": 0,
    "failed": 0
  },
  "_clusters": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "running": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "linked_project": {
        "status": "successful",
        "indices": "*",
        "took": 11,
        "timed_out": false,
        "_shards": {
          "total": 12,
          "successful": 12,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "linked_project:my-index",
        "_id": "ytm_v5wB1c8L_6vBSeM6",
        "_score": 1.0,
        "_source": {
          "project": "linked"
        }
      },
      {
        "_index": "linked_project:logs",
        "_id": "y9m_v5wB1c8L_6vBW-Mu",
        "_score": 1.0,
        "_source": {
          "project": "linked-logs-data"
        }
      }
    ]
  }
}
		
				
					{
  "took": 54,
  "is_partial": false,
  "completion_time_in_millis": 1772740419771,
  "documents_found": 2,
  "values_loaded": 6,
  "start_time_in_millis": 1772740419717,
  "expiration_time_in_millis": 1773172419734,
  "columns": [
    {
      "name": "project",
      "type": "text"
    },
    {
      "name": "project.keyword",
      "type": "keyword"
    },
    {
      "name": "_index",
      "type": "keyword"
    }
  ],
  "values": [
    [
      "linked-logs-data",
      "linked-logs-data",
      "linked_project:logs"
    ],
    [
      "linked",
      "linked",
      "linked_project:my-index"
    ]
  ],
  "_clusters": {
    "total": 1,
    "successful": 1,
    "running": 0,
    "skipped": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "linked_project": {
        "status": "successful",
        "indices": "*",
        "took": 35,
        "_shards": {
          "total": 12,
          "successful": 12,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  }
}

First, create the named expression:

				PUT /_project_routing/origin-only
					{
  "expression": "_alias:_origin"
}

Then, query it:

				GET /my*/_search
					{
  "project_routing": "@origin-only",
  "query": {
    "match_all": {}
  }
}
		
				GET /_query
					{
  "project_routing": "@origin-only",
  "query": "FROM *",
  "include_execution_metadata": true
}

In the first example, both the project routing rule and the qualified index expression limit the search to the linked project:

				GET /linked_project:my*/_search
					{
  "project_routing": "_alias:lin*",
  "query": {
    "match_all": {}
  }
}

In the next example, the project routing rule and the qualified index expression target different projects which causes a conflict:

				GET /_origin:*,linked_project:*/_search
					{
  "project_routing": "@origin-only",
  "query": {
    "match_all": {}
  }
}

This request returns an error:

				
					{
  "error": {
    "root_cause": [
      {
        "type": "no_matching_project_exception",
        "reason": "No such project: [linked_project] with project routing [@origin-only]"
      }
    ],
    "type": "no_matching_project_exception",
    "reason": "No such project: [linked_project] with project routing [@origin-only]"
  },
  "status": 404
}