Mapping

Elastic Stack Serverless

Mapping is the process of defining how a document and the fields it contains are stored and indexed.

Each document is a collection of fields, which each have their own data type. When mapping your data, you create a mapping definition, which contains a list of fields that are pertinent to the document. A mapping definition also includes metadata fields, like the _source field, which customize how a document’s associated metadata is handled.

Depending on where you are in your data journey, use dynamic mapping and explicit mapping to define your data. For example, you can explicitly map fields where you don’t want to use the defaults, or to gain greater control over which fields are created. Then you can allow Elasticsearch to dynamically map other fields. Using a combination of dynamic and explicit mapping on the same index is especially useful when you have a mix of known and unknown fields in your data.

When you use dynamic mapping, Elasticsearch automatically detects the data types of fields in your documents and creates mappings for you. If you index additional documents with new fields, Elasticsearch will add these fields automatically. You can add fields to the top-level mapping, and to inner object and nested fields. Dynamic mapping helps you get started quickly, but might yield suboptimal results for your specific use case due to automatic field type inference.

Use dynamic templates to define custom mappings that are applied to dynamically added fields based on the matching condition.

Use explicit mapping to define mappings by specifying data types for each field. This is recommended for production use cases, because you have full control over how your data is indexed to suit your specific use case.

Defining your own mappings enables you to:

• Define which string fields should be treated as full-text fields.
• Define which fields contain numbers, dates, or geolocations.
• Use data types that cannot be automatically detected (such as geo_point and geo_shape.)
• Choose date value formats, including custom date formats.
• Create custom rules to control the mapping for dynamically added fields.
• Optimize fields for partial matching.
• Perform language-specific text analysis.

Use runtime fields to make schema changes without reindexing. You can use runtime fields in conjunction with indexed fields to balance resource usage and performance. Your index will be smaller, but with slower search performance.

Explicit mappings should be defined at index creation for fields you know in advance. You can still add new fields to mappings at any time, as your data evolves.

Use the Update mapping API to update an existing mapping.

In most cases, you can’t change mappings for fields that are already mapped. These changes require reindexing.

However, you can update mappings under certain conditions:

• You can add new fields to an existing mapping at any time, dynamically or explicitly.

• You can add new multi-fields for existing fields.

  • Documents indexed before the mapping update will not have values for the new multi-fields until they are updated or reindexed. Documents indexed after the mapping change will automatically have values for the new multi-fields.

• Some mapping parameters can be updated for existing fields of certain data types.

Defining too many fields in an index can lead to a mapping explosion, which can cause out of memory errors and difficult situations from which to recover.

Consider a situation where every new document inserted introduces new fields, such as with dynamic mapping. Each new field is added to the index mapping, which can become a problem as the mapping grows.

Use the mapping limit settings to limit the number of field mappings (created manually or dynamically) and prevent documents from causing a mapping explosion.