Loading

Logs data streams

Logs data streams store log data more efficiently. In benchmarks, logsdb index mode reduced the storage footprint of log data by up to 60%, with a small impact (10-20%) to indexing performance. Results vary depending on your data set and Elasticsearch version.

Logs data streams are created when the index.mode in the relevant template is set to logsdb, either automatically or manually.

Logsdb index mode is enabled by default for logs in Elastic Cloud Serverless, and for new logs data streams in Elastic Stack 9.0 and later.

Logsdb index mode is automatically enabled for the following data streams:

  • Elastic Cloud Serverless: Logsdb mode is automatically set on new and existing data streams with names matching the logs-*-* pattern.
  • Elastic Stack: Automatic logsdb mode depends on your version and configuration:
    • As of Elasticsearch version 9.0, logsdb mode is automatically set on new data streams with names matching the logs-*-* pattern.
    • In clusters that were upgraded from 8.x to 9.x:
      • If the instance had no existing logs-*-* data streams when you upgraded, new logs-*-* data streams are set to logsdb mode.
      • Existing data streams, including those used for integrations or APM in 8.x instances, are not automatically set to logsdb mode.

You can enable logsdb on existing data streams as needed, by editing the relevant index templates. For integrations, use @custom component templates.

In most cases, you won't need to enable logsdb mode manually. If you do need to enable it, you can either create a new template or update an existing one.

Set index.mode to logsdb in the relevant index template or @custom component template:

  • New data streams: Create a new template, specifying logsdb for the index mode. New data streams matching the template's index pattern use logsdb mode automatically.
  • Existing data streams: Update the templates the data stream references. Logsdb mode will take effect on the next rollover. For integrations, refer to Enable logsdb for integrations.

To create or edit an index template in Kibana:

  1. Go to Index Management using the navigation menu or the global search field.

  2. On the Index Templates tab, click Create template to create a new template, or click the name of an existing template to edit it.

    Tip

    If you have existing templates labeled Managed, refer to Enable logsdb for integrations or Default logs index template.

  3. Complete the steps in the wizard. In the Logistics step:

    • Click the Set index mode toggle to show the Index mode field.
    • Select LogsDB as the Index mode.

To create or update an index template:

First retrieve the current configuration, if any, so you can preserve existing settings. The PUT request overwrites any existing template.

				GET _index_template/my-index-template

Set index.mode to logsdb in the template request:

				PUT _index_template/my-index-template
					{
  "index_patterns": ["my-data-*"],
  "data_stream": { },
  "template": {
     "settings": {
        "index.mode": "logsdb"
     }
  },
  "priority": 200
}

To enable logsdb mode for integration data streams, create or update a @custom component template for each logs dataset. For details, refer to Enable logsdb for integrations.