Configure logs data retention

Serverless Unavailable Stack

Your data retention policies define how long Elasticsearch keeps your log data before automatically removing it. Setting an appropriate data retention period helps manage storage costs and keeps your log data manageable.

Manage log data retention in the following ways:

• Manage data retention using Streams
• Store logs in data streams
• Customize the built-in logs@lifecycle policy
• Automate rollover based on log volume
• Inspect and manage ILM policies using the ILM API

Streams provides a single, centralized UI within Kibana that simplifies common tasks, including setting data retention. The Retention tab lets you manage how your stream retains data and provides insight into data ingestion and storage size.

For more on managing data retention through the Streams UI, refer to Manage data retention for Streams.

A data stream lets you store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams also provide the following benefits:

• ILM out of the box to automate the management of the backing indices.
• Automatic rollover to ensure backing indices stay within optimal size and performance limits.
• Tiered storage (hot, warm, and cold phases) to optimize storage and performance.

Refer to the data stream docs for more information.

The logs@lifecycle ILM policy is preconfigured for common logging use cases. View or duplicate the policy at Stack Management → Index Lifecycle Policies or find Index Lifecycle Policies in the global search field.

The logs ILM policy provides a foundation for your logs data streams, but you might need to tailor it to fit your situation. Common modifications include:

• Adjust hot, warm, and cold phase transitions.
• Set retention durations for different phases.
• Update rollover conditions.

Refer to the Customize built-in policies tutorial for more on modifying the logs ILM policy.

When continuously indexing timestamped documents, you need to periodically roll over to a new index to ensure that backing indices stay within optimal size and performance limits.

Refer to the Automate rollover tutorial for more information.

You can also review ILM policies using the Get lifecycle policies API.

For example, running GET /_ilm/policy/logs@lifecycle pulls up the default logs ILM policy, and shows something like the following:

{
  "logs@lifecycle": {
    "version": 1,
    "modified_date": "2025-05-19T16:45:58.754Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_age": "30d",
              "max_primary_shard_size": "50gb"
            }
          }
        }
      },
      "_meta": {
        "description": "default policy for the logs index template installed by x-pack",
        "managed": true
      },
      "deprecated": false
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  }
}
		

Refer to the ILM API documentation for more information.