Field mapping conflicts
A cross-project search datafeed merges search results across linked projects, so a field name must represent the same analytical type everywhere. When mappings disagree, Elasticsearch can log a report-only warning after a confirmed scope change, exclude one project from the current run, or refuse to start the datafeed, depending on whether the field is optional or the required time field.
Where to look
anomaly detection job job messages in Kibana: Open Machine Learning → Anomaly Detection, select the job, and review the Job messages tab for audit entries and warnings about linked projects, credentials, or scope changes. On the Datafeed tab, View datafeed counts opens the datafeed chart flyout for extraction timing.
The same entries are stored in
.ml-notifications-*.GET _ml/datafeeds/{datafeed_id}: Shows the effectiveproject_routingvalue and, when an internal cloud credential exists,authorization.cloud_api_key.id.GET _ml/datafeeds/{datafeed_id}/_stats: While the datafeed runs, showsremote_cluster_statswithtotal_clusters,available_clusters,skipped_clusters,availability_ratio,stabilized_cluster_aliases, andper_cluster_consecutive_skips. The object is absent until the first search cycle establishes a baseline..ml-annotations-*: Scope-change annotations for the job. The annotationeventfield carriessearch_scope_changed(not the separatetypefield).GET /_project/tags: Lists linked projects and their tags so you can compare them with a routing expression.Elastic Cloud console: Review linked projects in Link and manage projects.
During active extraction problems, Job messages are authoritative. remote_cluster_stats from get datafeed stats reflects the last completed cycle.
Error messages
Search Job messages (or .ml-notifications-*) for these patterns. The bracketed datafeed id, field name, project aliases, and type detail vary with your configuration.
Optional field: after project scope stabilizes following a link or routing change, Elasticsearch re-checks field capabilities and logs a report-only warning. The datafeed continues:
Cross-project field conflict for datafeed [my-datafeed]: field [status] has incompatible types across linked projects: keyword in [prod-us], long in [prod-eu]. Align index mappings across projects or narrow project_routing to projects with a consistent schema.
Required time field: fail-fast when Elasticsearch builds the extractor (start or preview). The datafeed does not run:
Cannot run datafeed [my-datafeed]: required time field [@timestamp] has conflicting types across projects in scope: date in [prod-us], long in [prod-eu]. Fix mappings so [@timestamp] uses the same type in every project in scope, or exclude the conflicting project(s) via project_routing.
Required time field: mid-run project exclusion after a scope change. The datafeed keeps running on the remaining projects:
Datafeed [my-datafeed] excluded project [prod-eu] from this run: required time field [@timestamp] has conflicting types: date in [prod-us], long in [prod-eu]. Fix mappings in [prod-eu] to resume searching it, or remove it from project_routing.
The type detail after the colon lists each conflicting type and the project aliases that use it, for example keyword in [prod-us], long in [_origin, prod-eu].
Type compatibility
For the time field, only date and date_nanos are treated as compatible.
For optional fields, Elasticsearch groups types into analytical families and warns when a field's types span more than one family:
| Family | Types |
|---|---|
| Integral | long, integer, short, byte |
| Floating point | double, float, half_float, scaled_float |
| Date/time | date, date_nanos |
| Keyword-like | keyword, constant_keyword, wildcard |
| Text-like | text, match_only_text |
| Boolean | boolean |
| IP | ip |
| Geo point | geo_point |
| Geo shape | geo_shape |
| Object/nested | object, nested |
Types within the same family do not trigger a warning (for example long in one project and integer in another). Types from different families do (for example keyword and long). If any type is outside these families, Elasticsearch suppresses the optional-field warning.
Compare mappings across projects
Run field capabilities on the indices the datafeed queries, one call per project in scope:
GET _origin:logs-*/_field_caps?fields=@timestamp,status&include_unmapped
GET prod-us:logs-*/_field_caps?fields=@timestamp,status&include_unmapped
Use the _origin: qualifier for the origin project and the linked project's alias for remote projects. In each response, inspect the field entry's type keys and compare them across calls using the compatibility families above. Multiple keys in the same family (for example long and integer) are compatible. Keys spanning different families indicate a conflict.
A mapping changed after the datafeed was created
Index templates, ingest pipelines, or explicit mapping updates in one linked project can introduce conflicts that were not present at create time. A mapping rollout alone does not trigger Elasticsearch's field-conflict recheck. Symptoms of mapping drift include:
- Job messages report a new extraction error even though routing and credentials are unchanged.
- The datafeed fails to start or preview after a restart because the time field now conflicts at extractor build time.
Optional-field conflict warnings and time-field project exclusions appear only when Elasticsearch re-checks field capabilities after project scope stabilizes (for example after a project is linked or project_routing changes), not from a mapping change by itself. When that recheck runs, you might also see:
- An optional-field conflict warning (see above).
- A project-exclusion message for the time field (see above).
Example extraction error after mapping drift:
Datafeed is encountering errors extracting data: Cannot parse field [status] of type [long] in document with id 'abc123'
The text after the colon is the underlying cause and varies (parse failure, missing field, incompatible type, and so on).
Align the mappings
Standardize the conflicting field to a compatible type in every linked project, especially the time field (use date with a consistent format, or date_nanos everywhere). Update index templates or reindex where needed, then restart or preview the datafeed.
Exclude the conflicting project
When mappings cannot be aligned immediately, narrow project_routing to projects with a consistent schema. For routing syntax and stale-alias problems, see Project scope problems.
Stop the datafeed before updating it. If the datafeed is still running, the update fails with:
Cannot update datafeed [my-datafeed] while its status is started
The bracketed datafeed id and status vary with your configuration.
When a project_routing update changes the effective search scope, Elasticsearch retains the job's current model snapshot as a rollback point before applying the change. By default, that rollback gate requires the anomaly detection job to be closed. If the job is still open, the update fails with:
Cannot update project_routing for datafeed [my-datafeed] while job [my-job] is opened. Close the job so a rollback model snapshot can be retained.
The bracketed datafeed id, job id, and job state vary with your configuration.
The rollback gate also requires an existing model snapshot. If the job has never produced one, the update fails with:
Cannot update project_routing for datafeed [my-datafeed] because job [my-job] has no model snapshot to use as a rollback point. Open the job, ingest data, then close it before changing scope.
A job that has never run has no snapshot. Open it, let it process data, then close it before changing project scope.
Elasticsearch records the retained rollback snapshot in the job's Job messages (or .ml-notifications-*), for example:
Rollback model snapshot [1720000000] retained before project_routing scope change: Automatic rollback snapshot retained before project_routing scope change [] -> [_alias:prod-*]
When the datafeed already had a stored routing value, the bracketed pair shows the old and new expressions instead of an empty left side:
Rollback model snapshot [1720000000] retained before project_routing scope change: Automatic rollback snapshot retained before project_routing scope change [_alias:_origin] -> [_alias:prod-*]
You can revert to that snapshot if detection quality degrades after a scope change.
Exception: assigning _alias:_origin for the first time to a datafeed that had no project_routing preserves the existing local-only scope. That update bypasses the rollback gate entirely. Neither the closed-job check nor the snapshot requirement applies.
POST _ml/datafeeds/{datafeed_id}/_update
{
"project_routing": "_alias:prod-us"
}
Replace the routing expression with one that omits the conflicting project while keeping the anomaly detection job's intended coverage elsewhere.
Re-validate after a mapping change
After mappings stabilize:
- Stop the datafeed.
- Update the job query or aggregations if field names or types changed.
- Preview the datafeed in Kibana or with
POST _ml/datafeeds/{datafeed_id}/_previewand confirm data returns from every project still in scope. - Start the datafeed.
Protect the model before a planned mapping change
Before rolling out breaking mapping changes to projects in an active anomaly detection job's scope, close the job so Elasticsearch retains a model snapshot. If detection quality degrades after the change, revert using the procedure in Project scope changes.
Verify
_field_capsshows one compatible type per field across every project in scope.- Job messages no longer report field conflicts or extraction errors caused by mapping drift.
- Preview returns documents from each project matched by
project_routing.