---
title: Troubleshoot project scope changes
description: Changing project_routing (the Project scope field in Kibana) decides which linked projects a cross-project search datafeed searches. This page covers...
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8277/troubleshoot/elasticsearch/machine-learning/cps-datafeed-scope-change
products:
  - Elastic Cloud Serverless
  - Elasticsearch
  - Machine Learning
applies_to:
  - Elastic Cloud Serverless: Preview
  - Elastic Stack: Unavailable
---

# Troubleshoot project scope changes
Changing `project_routing` (the **Project scope** field in Kibana) decides which linked projects a cross-project search datafeed searches. This page covers three related problems: Elasticsearch or Kibana rejects the change, a bulk update succeeds for some jobs but not others, or the change took effect and the anomaly detection job model is reacting to a different data set.

## Where to look

Use these sources to gather diagnostic information:
- **Anomaly detection job job messages in Kibana**: Open **Machine Learning → Anomaly Detection**, select the job, and review the **Job messages** tab for audit entries and warnings about linked projects, credentials, or scope changes. On the **Datafeed** tab, **View datafeed counts** opens the datafeed chart flyout for extraction timing.
  The same entries are stored in `.ml-notifications-*`.
- **`GET _ml/datafeeds/{datafeed_id}`**: Shows the effective `project_routing` value and, when an internal cloud API key exists, `authorization.cloud_api_key.id`.
- **`GET _ml/datafeeds/{datafeed_id}/_stats`**: While the datafeed runs, shows `remote_cluster_stats` with `total_clusters`, `available_clusters`, `skipped_clusters`, `availability_ratio`, `stabilized_cluster_aliases`, and `per_cluster_consecutive_skips`. The object is absent until the first search cycle establishes a baseline.
- **`.ml-annotations-read`**: Scope-change annotations for the job. The annotation `event` field carries `search_scope_changed` (not the separate `type` field).
- **`GET /_project/tags`**: Lists linked projects and their tags so you can compare them with a routing expression.
- **Elastic Cloud console**: Review linked projects in [Link and manage projects](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8277/deploy-manage/cross-project-search-config/cps-config-link-and-manage).

If extraction failures are ongoing, check **Job messages** first. `remote_cluster_stats` from [get datafeed stats](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ml-get-datafeed-stats) only updates after a cycle completes.

## Scope change rejected


### What you see

If the datafeed is still running, the update fails with:
```txt
Cannot update datafeed [my-datafeed] while its status is started
```

If the anomaly detection job is still open, the update fails with:
```txt
Cannot update project_routing for datafeed [my-datafeed] while job [my-job] is opened. Close the job so a rollback model snapshot can be retained.
```

The bracketed datafeed id, job id, and job state vary with your configuration. From the API, a rejected update returns HTTP `409`.
A job with no model snapshot can still change scope. Elasticsearch retains a rollback snapshot only when one already exists.
Assigning `_alias:_origin` for the first time to a datafeed that had no `project_routing` preserves local-only scope. That update does not require a closed job.
A bulk update stops and restarts a running datafeed for you, but it does **not** close the anomaly detection job. Because a `project_routing` change requires a closed job, open jobs fail the bulk update even though their datafeed was stopped automatically. Close those jobs first, then run the update again.
When you try to update a single job while the datafeed is running, the datafeed tab shows:
```txt
Datafeed settings cannot be edited while the datafeed is running. Please stop the job if you wish to edit these settings.
```


### Fix


| Entry point                          | Fix                                                                                                                             |
|--------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|
| API update                           | Stop the datafeed, close the anomaly detection job, then retry `POST _ml/datafeeds/{datafeed_id}/_update`.                      |
| Kibana bulk **Change project scope** | Close every open job in the selection (the bulk action does not close jobs for you), then re-run the update.                    |
| Kibana single-job **Datafeed** tab   | Stop the datafeed so **Project scope** becomes editable, close the job if you are changing to a new effective scope, then save. |

```json
```

For routing expressions that match no project or reference stale aliases, see [Project scope problems](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8277/troubleshoot/elasticsearch/machine-learning/cps-datafeed-project-scope).

### Verify

The update succeeds without HTTP `409`. `GET _ml/datafeeds/{datafeed_id}` returns the new `project_routing` value.

## Bulk update partly succeeded


### What you see

When you [bulk update](/elastic/docs-content/pull/8277/explore-analyze/machine-learning/anomaly-detection/ml-ad-run-jobs#ml-ad-cps-update) multiple jobs, the bulk update might not be completely successful.
Failed jobs display the job id and a cross icon (`cross`). To learn why a job failed, open **Job messages** for that job or retry the update through the API and read the error response.
If the flyout cannot load the job list, Kibana shows **Could not load jobs for project routing update**. If the bulk API call fails before per-job results are returned, Kibana shows **Project routing update failed**.
A separate message appears if Kibana could not restart a datafeed after a successful routing change: **Failed to restart datafeed for job *job id***.
After submission, a bulk update status message appears:
- Partial: *Project routing was updated for `X` of `Y` jobs. Any jobs that were previously running will need to be restarted if their update failed.*

Jobs that fail show a cross icon in the flyout job list. Jobs that succeed show a check icon.

### Fix

1. Note every job ID that shows a cross icon in the flyout (or that appeared in the partial-failure message).
2. For each failed job, open **Job messages** and read the API error. Common causes are an open job, invalid `project_routing`, or a credential problem.
3. Fix the underlying cause per job. For credential failures, see [Cloud credential problems](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8277/troubleshoot/elasticsearch/machine-learning/cps-datafeed-credentials).
4. Re-run the update for the failed jobs only. Select just those jobs and use **Change project scope** again.
5. When one job keeps failing, use the single-job path: edit the job, open the **Datafeed** tab, set **Project scope**, and save after closing the job.

Example retry for one job after stopping the datafeed and closing the job:
```json



{
  "project_routing": "_alias:production-*"
}
```

Adjust `project_routing` to the scope you chose in the bulk flyout.

### Verify

Every selected job shows a check icon in the flyout, or the success toast reports all jobs updated.

## Scope changed and model is reacting


### What you see

When linked projects are added or removed (whether because you changed `project_routing`, or linked or unlinked a project in Elastic Cloud), the datafeed might search a different set of projects than when the model was trained. Elasticsearch records scope changes only after they stabilize. Temporary anomalies are expected while the model adapts.
Scope-change confirmations appear in job messages once stabilization completes. Examples with realistic alias names (linked/unlinked aliases vary):
When a project is newly linked:
```txt
Datafeed search scope changed: [staging] linked. Data distribution may have changed due to new data sources, which can cause temporary anomalies while the model adapts. If detection quality degrades, consider specifying the source clusters explicitly and reviewing recent model snapshots for potential rollback.
```

When a project is unlinked:
```txt
Datafeed search scope changed: [staging] unlinked. Data distribution may have changed due to removed data sources, which can cause temporary anomalies as patterns the model learned are no longer present. If detection quality degrades, consider specifying the source clusters explicitly and reviewing recent model snapshots for potential rollback.
```

When both happen in the same stabilization window:
```txt
Datafeed search scope changed: [production] linked, [staging] unlinked. Data distribution may have changed, which can cause temporary anomalies while the model adapts. If detection quality degrades, consider specifying the source clusters explicitly and reviewing recent model snapshots for potential rollback.
```

If anomaly scores spike after a confirmed change, job messages might also show:
```txt
Elevated anomaly scores detected after search scope change at [2026-07-28T10:15:00.000Z] (production linked). [12] buckets with anomaly score >= 75 observed since the scope change. This is likely caused by the data distribution shift. Consider reviewing model snapshots if the anomalies are not meaningful.
```

The timestamp, change summary in parentheses, and bucket count vary with your job.
When Elasticsearch retains a rollback snapshot, job messages include an entry such as:
```txt
Rollback model snapshot [1720000000] retained before project_routing scope change: Automatic rollback snapshot retained before project_routing scope change [_alias:_origin] -> [_alias:prod-*]
```

If the datafeed had no stored routing, the left side of the pair is empty. If the job had no snapshot, this message does not appear and the update still succeeds.
By default, Elasticsearch confirms a scope change only after the linked-project set stays stable for **12 consecutive extraction cycles** and at least **5 minutes** since the change was first observed. Until both conditions are met, you do not see scope-change messages or annotations even if projects were linked or unlinked in Elastic Cloud.
Elasticsearch writes scope-change annotations to `.ml-annotations-read`. The annotation `event` field carries `search_scope_changed`. Search that index for the job id and filter on `event: search_scope_changed` to see when scope stabilized.

### Fix

If the scope change was intentional, allow several extraction cycles for the model to adapt. Monitor job messages and annotations until elevated-score warnings stop.
If the change was unintentional (for example a project was unlinked in Elastic Cloud), restore the link or update `project_routing` to the intended expression. See [Project scope problems](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8277/troubleshoot/elasticsearch/machine-learning/cps-datafeed-project-scope) for routing syntax.
If detection quality degrades and anomalies are not meaningful, revert to the model snapshot Elasticsearch retained before the scope change, or to an earlier snapshot:
1. Close the anomaly detection job.
2. In Kibana, open **Machine Learning → Anomaly Detection → your job → Model snapshots**, select a snapshot from before the scope change (or the automatic rollback snapshot), and revert.
3. Confirm `project_routing` on the datafeed matches your intended scope with `GET _ml/datafeeds/{datafeed_id}`.
4. Start the datafeed.


### Verify

New `search_scope_changed` annotations stop appearing and job messages no longer report scope-change or elevated-score warnings on every cycle. `GET _ml/datafeeds/{datafeed_id}/_stats` shows successful extraction cycles with `remote_cluster_stats` listing only the intended projects.