Project scope changes

Changing project_routing (the Project scope field in Kibana) decides which linked projects a cross-project search datafeed searches. This page covers three related problems: Elasticsearch or Kibana rejects the change, a bulk update succeeds for some jobs but not others, or the change took effect and the anomaly detection job model is reacting to a different data set.

Diagnose project scope changes

Where to look

anomaly detection job job messages in Kibana: Open Machine Learning → Anomaly Detection, select the job, and review the Job messages tab for audit entries and warnings about linked projects, credentials, or scope changes. On the Datafeed tab, View datafeed counts opens the datafeed chart flyout for extraction timing.

The same entries are stored in .ml-notifications-*.
GET _ml/datafeeds/{datafeed_id}: Shows the effective project_routing value and, when an internal cloud credential exists, authorization.cloud_api_key.id.
GET _ml/datafeeds/{datafeed_id}/_stats: While the datafeed runs, shows remote_cluster_stats with total_clusters, available_clusters, skipped_clusters, availability_ratio, stabilized_cluster_aliases, and per_cluster_consecutive_skips. The object is absent until the first search cycle establishes a baseline.
.ml-annotations-*: Scope-change annotations for the job. The annotation event field carries search_scope_changed (not the separate type field).
GET /_project/tags: Lists linked projects and their tags so you can compare them with a routing expression.
Elastic Cloud console: Review linked projects in Link and manage projects.

During active extraction problems, Job messages are authoritative. remote_cluster_stats from get datafeed stats reflects the last completed cycle.

The scope change is rejected

The most common rejection is an open anomaly detection job. Changing project_routing to a new effective scope requires a closed job so Elasticsearch can retain the current model snapshot as a rollback point.

From the API, a rejected update returns HTTP 409 (datafeed still running or job still open) or 400 (no model snapshot). The response body repeats the messages shown in the preconditions block in Resolve project scope changes.

From Kibana, the bulk Change project scope action (multi-select jobs list) opens the flyout titled Update project routing for N anomaly detection jobs.

Important: the bulk update stops and restarts a running datafeed for you, but it does not close the anomaly detection job. Because a project_routing change requires a closed job, open jobs fail the bulk update even though their datafeed was stopped automatically. Close those jobs first, then run the update again.

The single-job path has the same datafeed gate: open Machine Learning → Anomaly Detection, edit the job, open the Datafeed tab, and adjust Project scope. While the datafeed is running, the tab shows:

		Datafeed settings cannot be edited while the datafeed is running. Please stop the job if you wish to edit these settings.
		
	

The bulk update partly succeeded

Entry point

Select the anomaly detection jobs in the jobs list, then choose Change project scope from the multi-select action menu. That action opens the bulk flyout titled Update project routing for N anomaly detection jobs, which shows a Project scope picker for the routing expression and an Update N jobs button.

When you pick any routing expression other than _alias:_origin, the flyout shows a warning callout titled Non-default project scope selected:

		Using a project routing scope other than _alias:_origin may negatively affect the job's anomaly detection results.
		
	

That warning reflects the risk of a data-distribution shift, not a hard block. Cross-project scope is appropriate when the anomaly detection job should analyze linked-project data; use _alias:_origin only when origin-project-only search is intended.

Clicking Update N jobs opens a confirmation dialog titled Update project scope? (or Change project scope for job id? / Change project scope for N jobs? for specific selections). The dialog body reads:

		The model for this job was trained on a specific set of data. Changing this data set may cause temporary model instability and an increase in false-positives. Are you sure you want to apply these changes?
		
	

When you select more than one job, the dialog also lists Affected jobs with per-job added and removed project counts so you can review the scope delta before confirming.

Confirm with Yes, save or cancel with Cancel.

Failed jobs display no inline error text in the flyout, only the job id and a cross icon. To learn why a job failed, open Job messages for that job or retry the update through the API and read the error response.

If the flyout cannot load the job list, Kibana shows Could not load jobs for project routing update. If the bulk API call fails before per-job results are returned, Kibana shows Project routing update failed. Both differ from the per-job cross-icon path above.

After submission, result toasts report:

All succeeded: Successfully updated project routing for N jobs.
All failed: Project routing was not updated for any job.
Partial: Project routing was updated for X of Y jobs. Any jobs that were previously running will need to be restarted if their update failed.

Jobs that fail show a cross icon in the flyout job list. Jobs that succeed show a check icon. A separate toast appears if Kibana could not restart a datafeed after a successful routing change: Failed to restart datafeed for job job id.

When a legacy job is migrated without an explicit scope choice, Elasticsearch defaults routing to preserve local-only search and writes this audit entry in Job messages:

		CPS migration: project_routing defaulted to [_alias:_origin] to preserve local search scope. Use the update API to change the scope.
		
	

The bracketed routing value is always _alias:_origin for first-time CPS migration defaults.

Scope changed, and the model is reacting

When linked projects are added or removed (whether because you changed project_routing, linked or unlinked a project in Elastic Cloud, or a migration defaulted routing), the datafeed might search a different set of projects than when the model was trained. Elasticsearch records scope changes only after they stabilize (see below). Temporary anomalies are expected while the model adapts.

Scope-change confirmations appear in Job messages once stabilization completes. Examples with realistic alias names (linked/unlinked aliases vary):

When a project is newly linked:

		Datafeed search scope changed: [staging] linked. Data distribution may have changed due to new data sources, which can cause temporary anomalies while the model adapts. If detection quality degrades, consider specifying the source clusters explicitly and reviewing recent model snapshots for potential rollback.
		
	

When a project is unlinked:

		Datafeed search scope changed: [staging] unlinked. Data distribution may have changed due to removed data sources, which can cause temporary anomalies as patterns the model learned are no longer present. If detection quality degrades, consider specifying the source clusters explicitly and reviewing recent model snapshots for potential rollback.
		
	

When both happen in the same stabilization window:

		Datafeed search scope changed: [production] linked, [staging] unlinked. Data distribution may have changed, which can cause temporary anomalies while the model adapts. If detection quality degrades, consider specifying the source clusters explicitly and reviewing recent model snapshots for potential rollback.
		
	

If anomaly scores spike after a confirmed change, Job messages might also show:

		Elevated anomaly scores detected after search scope change at [2026-07-28T10:15:00.000Z] (production linked). [12] buckets with anomaly score >= 75 observed since the scope change. This is likely caused by the data distribution shift. Consider reviewing model snapshots if the anomalies are not meaningful.
		
	

The timestamp, change summary in parentheses, and bucket count vary with your job.

Detection delay. By default, Elasticsearch confirms a scope change only after the linked-project set stays stable for 12 consecutive extraction cycles and at least 5 minutes of wall-clock time since the change was first observed. Until both conditions are met, you will not see scope-change messages or annotations even if projects were linked or unlinked in Elastic Cloud.

Check annotations. Elasticsearch writes scope-change annotations to .ml-annotations-*. The annotation event field carries search_scope_changed. Search that index for the job id and filter on event: search_scope_changed to see when scope stabilized.

Resolve project scope changes

Stop the datafeed before updating it. If the datafeed is still running, the update fails with:

Cannot update datafeed [my-datafeed] while its status is started

The bracketed datafeed id and status vary with your configuration.

When a project_routing update changes the effective search scope, Elasticsearch retains the job's current model snapshot as a rollback point before applying the change. By default, that rollback gate requires the anomaly detection job to be closed. If the job is still open, the update fails with:

		Cannot update project_routing for datafeed [my-datafeed] while job [my-job] is opened. Close the job so a rollback model snapshot can be retained.
		
	

The bracketed datafeed id, job id, and job state vary with your configuration.

The rollback gate also requires an existing model snapshot. If the job has never produced one, the update fails with:

		Cannot update project_routing for datafeed [my-datafeed] because job [my-job] has no model snapshot to use as a rollback point. Open the job, ingest data, then close it before changing scope.
		
	

A job that has never run has no snapshot. Open it, let it process data, then close it before changing project scope.

Elasticsearch records the retained rollback snapshot in the job's Job messages (or .ml-notifications-*), for example:

		Rollback model snapshot [1720000000] retained before project_routing scope change: Automatic rollback snapshot retained before project_routing scope change [] -> [_alias:prod-*]
		
	

When the datafeed already had a stored routing value, the bracketed pair shows the old and new expressions instead of an empty left side:

		Rollback model snapshot [1720000000] retained before project_routing scope change: Automatic rollback snapshot retained before project_routing scope change [_alias:_origin] -> [_alias:prod-*]
		
	

You can revert to that snapshot if detection quality degrades after a scope change.

Exception: assigning _alias:_origin for the first time to a datafeed that had no project_routing preserves the existing local-only scope. That update bypasses the rollback gate entirely. Neither the closed-job check nor the snapshot requirement applies.

The scope change is rejected

Entry point	Fix
API update	Stop the datafeed, close the anomaly detection job, confirm a model snapshot exists, then retry `POST _ml/datafeeds/{datafeed_id}/_update`.
Kibana bulk Change project scope	Close every open job in the selection (the bulk action does not close jobs for you), then re-run the update.
Kibana single-job Datafeed tab	Stop the datafeed so Project scope becomes editable, close the job if you are changing to a new effective scope, then save.

For routing expressions that match no project or reference stale aliases, see Project scope problems.

The bulk update partly succeeded

Note every job id that shows a cross icon in the flyout (or that appeared in the partial-failure toast).
For each failed job, open Job messages and read the API error. Common causes are an open job, a missing model snapshot, invalid project_routing, or a credential problem.
Fix the underlying cause per job. For credential failures, see Cloud credential problems.
Re-run the update for the failed jobs only. Select just those jobs and use Change project scope again.
When one job keeps failing, use the single-job path: edit the job, open the Datafeed tab, set Project scope, and save after closing the job.

Example retry for one job after closing it:

						POST _ml/datafeeds/{datafeed_id}/_update
					{
  "project_routing": "_alias:production-*"
}
		
	

Adjust project_routing to the scope you chose in the bulk flyout. The legacy migration default is _alias:_origin.

Scope changed, and the model is reacting

If the scope change was intentional, allow several extraction cycles for the model to adapt. Monitor Job messages and annotations until elevated-score warnings stop.

If the change was unintentional (for example a project was unlinked in Elastic Cloud or migration defaulted routing to _alias:_origin), restore the link or update project_routing to the intended expression. See Project scope problems for routing syntax.

If detection quality degrades and anomalies are not meaningful, roll back to the model snapshot Elasticsearch retained before the scope change (see the preconditions block) or to an earlier snapshot:

Close the anomaly detection job.
In Kibana, open Machine Learning → Anomaly Detection → your job → Model snapshots, select a snapshot from before the scope change (or the automatic rollback snapshot), and revert.
Confirm project_routing on the datafeed matches your intended scope with GET _ml/datafeeds/{datafeed_id}.
Start the datafeed.

Verify recovery

Failed bulk updates: every selected job shows a check icon in the flyout, or the success toast reports all jobs updated.
Model adaptation: new search_scope_changed annotations stop appearing and Job messages no longer report scope-change or elevated-score warnings on every cycle.
GET _ml/datafeeds/{datafeed_id}/_stats shows successful extraction cycles with remote_cluster_stats listing only the intended projects.