--- title: Mapping explosion description: In Elasticsearch, a mapping or mapped field defines the fields in an index and their data types, such as text for full-text search, keyword for exact... url: https://www.elastic.co/elastic/docs-builder/docs/3028/troubleshoot/elasticsearch/mapping-explosion products: - Elasticsearch applies_to: - Elastic Cloud Serverless: Generally available - Elastic Stack: Generally available --- # Mapping explosion In Elasticsearch, a mapping or [mapped field](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/mapping-reference/field-data-types) defines the fields in an index and their data types, such as text for full-text search, keyword for exact filtering, or boolean for true/false values. Think of it as a schema that tells Elasticsearch how to interpret and store each piece of data. By default, Elasticsearch allows for [dynamic mappings](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping/dynamic-mapping) which allows on-the-fly creation of fields. Runaway field growth is colloquially called "mapping explosion". Excessively mapped fields within or across indices can cause the cluster to experience performance degradation such as slower searches, [high JVM memory pressure](https://www.elastic.co/elastic/docs-builder/docs/3028/troubleshoot/elasticsearch/high-jvm-memory-pressure), and prolonged startup times. To guard against mapping explosion, Elasticsearch default enforces field limit of `1000` by way of its [`index.mapping.total_fields.limit`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/index-settings/mapping-limit) setting. Elasticsearch will reject ingestion requests which would induce further mapped fields for any specific index once its limit threshold is reached with error ```text Limit of total fields [] has been exceeded while adding new fields [] ``` Advanced users may choose to bypass this rejection with [`index.mapping.total_fields.ignore_dynamic_beyond_limit`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/index-settings/mapping-limit) setting after weighing its effects against their use case. Mapping explosion typically signals an upstream data formatting issue that should be addressed client-side as [described in this blog](https://www.elastic.co/blog/3-ways-to-prevent-mapping-explosion-in-elasticsearch). Mapping explosion may surface as the following performance symptoms: - [CAT nodes](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-nodes) reporting high heap or CPU on the main node and/or nodes hosting the indices shards. This may potentially escalate to temporary node unresponsiveness and/or main overwhelm. - [CAT tasks](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-tasks) reporting long search durations only related to this index or indices, even on simple searches. - [CAT tasks](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-tasks) reporting long index durations only related to this index or indices. This usually relates to [pending tasks](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-pending-tasks) reporting that the coordinating node is waiting for all other nodes to confirm they are on mapping update request. - [CAT pending tasks](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-pending-tasks) reporting a task queue backlog with a lot of `put-mapping [MY_INDEX_NAME/MY_INDEX_UUID]` messages. - Discover’s **Fields for wildcard** page-loading API command or [Dev Tools](https://www.elastic.co/elastic/docs-builder/docs/3028/explore-analyze/query-filter/tools/console) page-refreshing Autocomplete API commands are taking a long time (more than 10 seconds) or timing out in the browser’s Developer Tools Network tab. For more information, refer to our [walkthrough on troubleshooting Discover](https://www.elastic.co/blog/troubleshooting-guide-common-issues-kibana-discover-load). - Discover’s **Available fields** taking a long time to compile Javascript in the browser’s Developer Tools Performance tab. This may potentially escalate to temporary browser page unresponsiveness. - Kibana’s [alerting](https://www.elastic.co/elastic/docs-builder/docs/3028/explore-analyze/alerting/alerts) or [security rules](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/security/detect-and-alert) may error `The content length (X) is bigger than the maximum allowed string (Y)` where `X` is attempted payload and `Y` is Kibana's [`server-maxPayload`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/kibana/configuration-reference/general-settings#server-maxpayload). - Long Elasticsearch start-up durations. - Kibana Javascript erring within browser while loading a Dashboard or Discover with message `Maximum call stack size exceeded`. ## Prevent or prepare [Mappings](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping) cannot be field-reduced once initialized. Elasticsearch indices default to [dynamic mappings](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping) which doesn’t normally cause problems unless it’s combined with overriding [`index.mapping.total_fields.limit`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/index-settings/mapping-limit). The default `1000` limit is considered generous, though overriding to `10000` doesn’t cause noticeable impact depending on use case. However, to give a bad example, overriding to `100000` and this limit being hit by mapping totals would usually have strong performance implications. If your index mapped fields expect to contain a large, arbitrary set of keys, you may instead consider: - Setting [`index.mapping.total_fields.ignore_dynamic_beyond_limit`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/index-settings/mapping-limit) to `true`. Instead of rejecting documents that exceed the field limit, this will ignore dynamic fields once the limit is reached. - Using the [flattened](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/mapping-reference/flattened) data type. Note, however, that flattened objects is [not fully supported in Kibana](https://github.com/elastic/kibana/issues/25820) yet. For example, this could apply to sub-mappings like { `host.name` , `host.os`, `host.version` }. Desired fields are still accessed by [runtime fields](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping/define-runtime-fields-in-search-request). - Disable [dynamic mappings](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping). This cannot effect current index mapping, but can apply going forward via an [index template](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/templates). Modifying to the [nested](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/mapping-reference/nested) data type would not resolve the core issue. ## Check for issue To confirm the field totals of an index to check for mapping explosion: - Check Elasticsearch cluster logs for errors similar to `Limit of total fields [X] in index [Y] has been exceeded` where `X` is the value of `index.mapping.total_fields.limit` and `Y` is the index name. The correlated client-side ingesting source HTTP 400 `mapper_parsing_exception` log error would be `Limit of total fields [X] has been exceeded while adding new fields [Z]` where `Z` is attempted new fields. - Review the elected-master node logs for a flood of `[elasticsearch.server][INFO] update_mapping [_doc]` messages. If they time out, a large amount of `org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-mapping [/]) within 30s` messages display as well. - For top-level fields, poll [field capabilities](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-field-caps) for `fields=*`. - Search the output of [get mapping](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping) for `"type"`. - If you’re inclined to use the [third-party tool JQ](https://stedolan.github.io/jq), you can process the [get mapping](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping) `mapping.json` output. ```sh cat mapping.json | jq -c 'to_entries[]| .key as $index| [.value.mappings| to_entries[]|select(.key=="properties") | {(.key):([.value|..|.type?|select(.!=null)]|length)}]| map(to_entries)| flatten| from_entries| ([to_entries[].value]|add)| {index: $index, field_count: .}' ``` You can use [analyze index disk usage](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-disk-usage) to find fields which are never or rarely populated as easy wins. ## Complex explosions Mapping explosions also covers when an individual index field totals are within limits but combined indices fields totals are very high. It’s very common for symptoms to first be noticed on a [data view](https://www.elastic.co/elastic/docs-builder/docs/3028/explore-analyze/find-and-organize/data-views) and be traced back to an individual index or a subset of indices via the [resolve index API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-resolve-index). However, though less common, it is possible to only experience mapping explosions on the combination of backing indices. For example, if a [data stream](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/data-streams)'s backing indices are all at field total limit but each contain unique fields from one another. This situation most easily surfaces by adding a [data view](https://www.elastic.co/elastic/docs-builder/docs/3028/explore-analyze/find-and-organize/data-views) and checking its **Fields** tab for its total fields count. This statistic does tells you overall fields and not only where [`index:true`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/mapping-reference/mapping-index), but serves as a good baseline. If your issue only surfaces via a [data view](https://www.elastic.co/elastic/docs-builder/docs/3028/explore-analyze/find-and-organize/data-views), you may consider this menu’s **Field filters** if you’re not using [multi-fields](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/elasticsearch/mapping-reference/field-data-types). Alternatively, you may consider a more targeted index pattern or using a negative pattern to filter-out problematic indices. For example, if `logs-*` has too high a field count because of problematic backing indices `logs-lotsOfFields-*`, then you could update to either `logs-*,-logs-lotsOfFields-*` or `logs-iMeantThisAnyway-*`. ## Resolve Mapping explosion is not easily resolved, so it is better prevented via the above. Encountering it usually indicates unexpected upstream data changes or planning failures. If encountered, we recommend reviewing your data architecture. The following options are additional to the ones discussed earlier on this page; they should be applied as best use-case applicable: - Disable [dynamic mappings](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping). - [Reindex](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-reindex) into an index with a corrected mapping, either via [index template](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/templates) or [explicitly set](https://www.elastic.co/elastic/docs-builder/docs/3028/manage-data/data-store/mapping). - If index is unneeded and/or historical, consider [deleting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-delete). - [Export](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/lsr/plugins-inputs-elasticsearch) and [re-import](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/lsr/plugins-outputs-elasticsearch) data into a mapping-corrected index after [pruning](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/lsr/plugins-filters-prune) problematic fields via Logstash. [Splitting index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-split) would not resolve the core issue.