Loading

Columnar index mode

Columnar index mode turns Elasticsearch into an analytical and search columnar store for the indices where you enable it. Instead of keeping multiple copies of each field for different query paths, columnar mode stores fields once as doc values by default, which can reduce storage cost for high-volume, analytics-heavy data while keeping the same APIs, dashboards, and integrations.

Columnar mode ships alongside existing index modes such as standard, logsdb, and time_series. You choose it per index (or in a template) at creation time; you can't change the mode after the index exists.

This page explains what columnar index mode is, when to use it, and how it fits with the rest of the Elasticsearch data store.

Choose columnar index mode when your data is written in volume, queried analytically (filters, aggregations, and dashboards), and retained for a long time. Typical fits include:

  • Logs and observability at scale: High ingest rates where storage and aggregation cost dominate, and you still need full-text search on message fields.
  • Security telemetry and threat hunting: Faceted exploration and historical lookups over large event stores, without giving up search behavior for pivot and lookup workflows.
  • Operational and business analytics: Dashboards and aggregations over application events, transactions, or IoT readings that would otherwise push you to a separate analytics store.

Columnar mode isn't a universal default. Prefer a standard index (or another specialized mode) when your workload is document-centric search that needs the original JSON _source as submitted, or when you rely on inverted indexes and related structures on most fields by default.

For metrics that need time series dimensions and metric field semantics, use a time series data stream instead. For logs that should keep the current logsdb defaults without switching to a fully columnar store, use a logs data stream.

Two values of index.mode enable columnar storage:

columnar
A general-purpose columnar store with no use-case-specific defaults.
Use it for bare indices and data streams that aren't log-oriented.
logsdb_columnar
The same columnar storage behavior with logging-oriented defaults, including a default @timestamp mapping and index sorting when @timestamp and host.name are present.
Use it for log data when you want columnar storage rather than the default logsdb mode.

Both modes apply to indices and to data stream backing indices configured through templates.

At a high level, columnar index mode changes storage defaults while leaving the document and query model familiar:

  • Store once, by field: Non-text fields are stored as doc values and aren't indexed by default, which avoids paying for structures your analytics workload may not need.
  • Keep search where it matters: Text fields remain indexed by default so full-text search continues to work on fields such as log messages.
  • Reconstruct _source when needed: The original JSON document body isn't stored as a separate copy. Elasticsearch returns a reconstructed or columnar _source representation at query time (or can store a columnar _source for faster retrieval).
  • Flat field layout: Object and passthrough mappings are flattened to leaf fields, which matches how columnar systems organize data for efficient scanning and compression.

Index sorting remains important for compression and query performance. logsdb_columnar sets sensible sort defaults for logs; for columnar, you choose sort fields that match your access patterns.

Columnar mode doesn't replace the core data store concepts:

  • You still store JSON documents in indices (or data streams), define mappings, and target indices by name, alias, or stream.
  • Shards, replicas, and near real-time search behave as they do for other index modes.
  • Existing query languages, Kibana visualizations, alerts, and ingest integrations continue to work against columnar indices.

What changes is how Elasticsearch lays data out on disk and which structures it builds by default, not how you interact with the cluster day to day.

  • Logs data streams: Use logsdb for efficient log storage with current defaults, or move to logsdb_columnar when you're ready for the columnar logs profile.
  • Time series data streams: Choose TSDS when your data is metrics with dimensions, not a general columnar analytics workload.
  • Index fundamentals: Review documents, mappings, settings, and shards if you're new to Elasticsearch storage concepts.