Loading

Cross-project search

Cross-project search (CPS) enables you to run a single search request across multiple Serverless projects. When your data is split across projects to organize ownership, use cases, or environments, cross-project search lets you query all that data from a single place, without having to search each project individually.

Cross-project search relies on linking projects within your Elastic Cloud organization. After you link projects together, searches from the origin project automatically run across all linked projects.

This overview explains how cross-project search works, including project linking and security. For details on how search, tags, and project routing work in CPS, refer to the following pages:

Projects are intended to act as logical namespaces for data, not hard boundaries for querying it. You can split data into projects to organize ownership, use cases, or environments, while still expecting to search and analyze that data from a single place.

Because of this, after you link additional projects to your current (origin) project, all searches from the origin project query every linked project by default. Searches are designed to run across projects automatically, providing the same experience for querying, analysis, and insights across projects as within a single project. Restricting search scope is always possible, but it requires explicitly scoping the search request using qualified expressions or routing parameters.

In Serverless, projects can be linked together. The project from which links are created is called the origin project, and the connected projects are referred to as linked projects.

The origin project is the project you are currently working in and from which you run cross-project searches. Linked projects are other projects that are connected to the origin project and whose data can be searched from it.

After you link projects, searches that you run from the origin project are no longer local to the origin project by default. Any search initiated on the origin project automatically runs across the origin project and all its linked projects (cross-project search).

When you search from an origin project, the query runs against its linked projects automatically unless you explicitly change the query scope by using project routing expressions or qualified index expressions.

Project linking is not bidirectional. Searches initiated from a linked project do not run against the origin project.

You can link projects by using the Elastic Cloud UI. For step-by-step instructions, refer to Link projects for cross-project search.

Each project has a unique project ID and a project alias. The project alias is derived from the project name and can be modified.

The project ID uniquely identifies a project and is system-generated.

The project alias is a human-readable identifier derived from the project's connection alias. If you want to change the project alias, you must update the connection alias of the linked project.

While both the project ID and project alias uniquely identify a project, cross-project search uses project aliases in index expressions. Project aliases are intended to be user-friendly and descriptive, making search expressions easier to read and maintain.

In addition to using a project alias, CPS provides a reserved identifier, _origin, that always refers to the origin project of the search. You can use _origin in search expressions to explicitly target the origin project, without having to reference its specific project alias. Refer to Qualified and unqualified search expressions for detailed examples and to learn more.

This section gives you a high-level overview of how security works in cross-project search.

In CPS, access to a project's data is determined by the roles assigned to you in that project. Your access does not change based on how you perform a search: whether you query directly within a project or access it through cross-project search, the same permissions apply.

Note

Cross-project search is not available when performing programmatic searches using Elasticsearch API keys, since they're project-scoped and they return results from the local project only.

Access control operates in two stages:

  • Authentication verifies the identity associated with a request (for example, a Cloud user or API key) and retrieves that identity's role assignments in each project.
  • Authorization evaluates those roles to determine which actions and resources the request can access within each project.

For example, if you have a viewer role in project 1, an admin role in project 2, and a custom role in project 3, you can access all three projects through cross-project search. Each project enforces the permissions associated with the role you have in that project.

When a cross-project search query targets a linked project that you have access to, authorization checks are performed locally in that project to determine whether you have the required privileges to access the requested resources.

Example You have read access to the logs index in project 1, but no access to the logs index in project 2. If you run GET logs/_search:

  • documents from the logs index in project 1 are returned
  • the logs index in project 2 is not accessible and is excluded from the results

The following APIs support cross-project search:

Currently, each origin project can have up to 20 linked projects. A linked project can be associated with any number of origin projects.

The following examples demonstrate how search requests behave in different cross-project scenarios in CPS.

In the following example, an origin project and a linked project both contain an index named my-index.

				GET /my-index/_search
					{
  "size": 2,
  "query": {
    "match_all": {}
  }
}

The request will return a response similar to this:

				
					{
  "took": 34,
  "timed_out": false,
  "num_reduce_phases": 3,
  "_shards": {
    "total": 12,
    "successful": 12,
    "skipped": 0,
    "failed": 0
  },
  "_clusters": {
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "running": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "_origin": {
        "status": "successful",
        "indices": "my-index",
        "took": 21,
        "timed_out": false,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      },
      "linked_project": {
        "status": "successful",
        "indices": "my-index",
        "took": 5,
        "timed_out": false,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "linked_project:my-index",
        "_id": "IH-mupwBMZyy2F9u2IQz",
        "_score": 1.0,
        "_source": {
          "project": "linked"
        }
      },
      {
        "_index": "my-index",
        "_id": "u0SnupwBaOrMOsBImb7G",
        "_score": 1.0,
        "_source": {
          "project": "origin"
        }
      }
    ]
  }
}

In this example, both the origin project and a linked project contain an index named my-index:

				POST /_query
					{
 "query": "FROM my-index",
  "include_ccs_metadata": true
}

The query will return a response similar to this:

				
					{
  "took": 39,
  "is_partial": false,
  "completion_time_in_millis": 1772659251830,
  "documents_found": 2,
  "values_loaded": 4,
  "start_time_in_millis": 1772659251791,
  "expiration_time_in_millis": 1773091251753,
  "columns": [
    {
      "name": "project",
      "type": "text"
    },
    {
      "name": "project.keyword",
      "type": "keyword"
    }
  ],
  "values": [
    [
      "origin",
      "origin"
    ],
    [
      "linked",
      "linked"
    ]
  ],
  "_clusters": {
    "total": 2,
    "successful": 2,
    "running": 0,
    "skipped": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "_origin": {
        "status": "successful",
        "indices": "my-index",
        "took": 39,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      },
      "linked_project": {
        "status": "successful",
        "indices": "my-index",
        "took": 23,
        "_shards": {
          "total": 6,
          "successful": 6,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  }
}

These requests don’t include a project prefix. The my-index index is searched in the origin project and in the linked project.

Search limited to the origin project:

				GET _origin:my-index/_search
		
				POST /_query
					{
  "query": "FROM _origin:my-index | LIMIT 10"
}

The requests include the _origin prefix. Only the origin project is searched.

Search across all projects using a wildcard expression:

				GET *:my-index/_search
		
				POST /_query
					{
  "query": "FROM *:my-index | LIMIT 10"
}

The requests explicitly target all projects using the *: prefix. The my-index index is evaluated separately in each project. The search runs only in projects where the my-index index exists.

In the following example, there is an origin project and a linked project. The origin project contains one index, my-index. The linked project contains two indices: my-index and logs.

The following request searches all indices on projects whose alias starts with "lin".

				GET /*/_search
					{
  "project_routing":"_alias:lin*",
  "query": {
    "match_all": {}
  }
}
		
				GET /_query
					{
  "query": "SET project_routing=\"_alias:lin*\"; FROM * METADATA _index",
  "include_ccs_metadata":true
}

The request will return a response similar to this:

				
					{
  "took": 60,
  "timed_out": false,
  "_shards": {
    "total": 12,
    "successful": 12,
    "skipped": 0,
    "failed": 0
  },
  "_clusters": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "running": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "linked_project": {
        "status": "successful",
        "indices": "*",
        "took": 11,
        "timed_out": false,
        "_shards": {
          "total": 12,
          "successful": 12,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 1.0,
    "hits": [
      {
        "_index": "linked_project:my-index",
        "_id": "ytm_v5wB1c8L_6vBSeM6",
        "_score": 1.0,
        "_source": {
          "project": "linked"
        }
      },
      {
        "_index": "linked_project:logs",
        "_id": "y9m_v5wB1c8L_6vBW-Mu",
        "_score": 1.0,
        "_source": {
          "project": "linked-logs-data"
        }
      }
    ]
  }
}
		
				
					{
  "took": 54,
  "is_partial": false,
  "completion_time_in_millis": 1772740419771,
  "documents_found": 2,
  "values_loaded": 6,
  "start_time_in_millis": 1772740419717,
  "expiration_time_in_millis": 1773172419734,
  "columns": [
    {
      "name": "project",
      "type": "text"
    },
    {
      "name": "project.keyword",
      "type": "keyword"
    },
    {
      "name": "_index",
      "type": "keyword"
    }
  ],
  "values": [
    [
      "linked-logs-data",
      "linked-logs-data",
      "linked_project:logs"
    ],
    [
      "linked",
      "linked",
      "linked_project:my-index"
    ]
  ],
  "_clusters": {
    "total": 1,
    "successful": 1,
    "running": 0,
    "skipped": 0,
    "partial": 0,
    "failed": 0,
    "details": {
      "linked_project": {
        "status": "successful",
        "indices": "*",
        "took": 35,
        "_shards": {
          "total": 12,
          "successful": 12,
          "skipped": 0,
          "failed": 0
        }
      }
    }
  }
}

First, create the named expression:

				PUT /_project_routing/origin-only
					{
  "expression": "_alias:_origin"
}

Then, query it:

				GET /my*/_search
					{
  "project_routing": "@origin-only",
  "query": {
    "match_all": {}
  }
}
		
				GET /_query
					{
  "project_routing": "@origin-only",
  "query": "FROM *",
  "include_ccs_metadata": true,
}

In the first example, both the project routing rule and the qualified index expression limit the search to the linked project:

				GET /linked_project:my*/_search
					{
  "project_routing": "_alias:lin*",
  "query": {
    "match_all": {}
  }
}

In the next example, the project routing rule and the qualified index expression target different projects which causes a conflict:

				GET /_origin:*,linked_project:*/_search
					{
  "project_routing": "@origin-only",
  "query": {
    "match_all": {}
  }
}

This request returns an error:

				
					{
  "error": {
    "root_cause": [
      {
        "type": "no_matching_project_exception",
        "reason": "No such project: [linked_project] with project routing [@origin-only]"
      }
    ],
    "type": "no_matching_project_exception",
    "reason": "No such project: [linked_project] with project routing [@origin-only]"
  },
  "status": 404
}