---
title: Manage time series data with data streams
description: To automate rollover and management of a data stream with ILM, you: Create a lifecycle policy that defines the appropriate phases and actions.Create an...
url: https://www.elastic.co/elastic/docs-builder/docs/3016/manage-data/lifecycle/index-lifecycle-management/tutorial-time-series-with-data-streams
products:
- Elasticsearch
applies_to:
- Elastic Stack: Generally available
---
# Manage time series data with data streams
To automate rollover and management of a data stream with ILM, you:
1. [Create a lifecycle policy](#ilm-gs-create-policy) that defines the appropriate [phases](https://www.elastic.co/elastic/docs-builder/docs/3016/manage-data/lifecycle/index-lifecycle-management/index-lifecycle) and [actions](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3016/reference/elasticsearch/index-lifecycle-actions).
2. [Create an index template](#ilm-gs-apply-policy) to [create the data stream](#ilm-gs-create-the-data-stream) and apply the ILM policy and the indices settings and mappings configurations for the backing indices.
3. [Verify that indices are moving through the lifecycle phases](#ilm-gs-check-progress) as expected.
When you enable index lifecycle management for Beats or the Logstash Elasticsearch output plugin, lifecycle policies are set up automatically. You do not need to take any other actions. You can modify the default policies through [Kibana Management](https://www.elastic.co/elastic/docs-builder/docs/3016/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies) or the ILM APIs.
## Create a lifecycle policy
A lifecycle policy specifies the phases in the index lifecycle and the actions to perform in each phase. A lifecycle can have up to five phases: `hot`, `warm`, `cold`, `frozen`, and `delete`.
For example, you might define a policy named `timeseries_policy` that has the following two phases:
- A `hot` phase that defines a rollover action to specify that an index rolls over when it reaches either a `max_primary_shard_size` of 50 gigabytes or a `max_age` of 30 days.
- A `delete` phase that sets `min_age` to remove the index 90 days after rollover.
The `min_age` value is relative to the rollover time, not the index creation time. [Learn more](/elastic/docs-builder/docs/3016/troubleshoot/elasticsearch/index-lifecycle-management-errors#min-age-calculation).
You can create the policy in Kibana or with the [create or update policy](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-put-lifecycle) API.
To create the policy from Kibana:
1. Go to the **Index Lifecycle Policies** management page using the navigation menu or the [global search field](https://www.elastic.co/elastic/docs-builder/docs/3016/explore-analyze/find-and-organize/find-apps-and-objects).
2. Click **Create policy**.
By default, only the hot index lifecycle phase is enabled. Enable each additional lifecycle phase that you'd like.
Use the [Create or update policy](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-put-lifecycle) API to add an ILM policy to the Elasticsearch cluster:
```json
{
"policy": {
"phases": {
"hot": { <1>
"actions": {
"rollover": {
"max_primary_shard_size": "50GB", <2>
"max_age": "30d"
}
}
},
"delete": {
"min_age": "90d", <3>
"actions": {
"delete": {} <4>
}
}
}
}
}
```
1. The `min_age` defaults to `0ms`, so new indices enter the `hot` phase immediately.
2. Trigger the `rollover` action when either of the conditions are met.
3. Move the index into the `delete` phase 90 days after rollover.
4. Trigger the `delete` action when the index enters the delete phase.
Note that for each phase after the hot phase, you have the option to move the data into the next phase after a certain duration of time. This duration is calculated from the time of the index rollover and not from the time the index is created.
For more details about default ILM policy settings, refer to [Create a lifecycle policy](/elastic/docs-builder/docs/3016/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy#ilm-create-policy).
## Create an index template to create the data stream and apply the lifecycle policy
To set up a data stream, first create an index template to specify the lifecycle policy. Because the template is for a data stream, it must also include a `data_stream` definition.
For example, you might create a template named `timeseries_template` and use it for a future data stream named `timeseries`.
To configure ILM to manage the data stream, you specify the name of the lifecycle policy that you want to apply to the data stream with the `index.lifecycle.name` setting.
Use the Kibana **Create template** wizard to add a template or the [Create or update index template](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-index-template) API to add a template and apply the lifecycle policy to indices matching the template.
To add an index template to a cluster using the wizard:
1. Go to the **Index Management** page using the navigation menu or the [global search field](https://www.elastic.co/elastic/docs-builder/docs/3016/explore-analyze/find-and-organize/find-apps-and-objects).
2. In the **Index Templates** tab, click **Create template** and follow the instructions.
You specify the name of the lifecycle policy that you want to apply to the data stream on the **Index settings** page.
To learn about which index template options you can specify, refer to [Create an index template to apply the lifecycle policy](/elastic/docs-builder/docs/3016/manage-data/lifecycle/index-lifecycle-management/configure-lifecycle-policy#apply-policy-template).
Use the API to add an index template to your cluster:
```json
{
"index_patterns": ["timeseries"], <1>
"data_stream": { },
"template": {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1,
"index.lifecycle.name": "timeseries_policy" <2>
}
}
}
```
## Create the data stream
To get things started, index a document into the name or wildcard pattern defined in the `index_patterns` of the [index template](https://www.elastic.co/elastic/docs-builder/docs/3016/manage-data/data-store/templates). As long as an existing data stream, index, or index alias does not already use the name, the index request automatically creates a corresponding data stream with a single backing index. Elasticsearch automatically indexes the request’s documents into this backing index, which also acts as the stream’s [write index](/elastic/docs-builder/docs/3016/manage-data/data-store/data-streams#data-stream-write-index).
For example, the following request creates the `timeseries` data stream and the first generation backing index called `.ds-timeseries-2099.03.08-000001`.
```json
{
"message": "logged the request",
"@timestamp": "1591890611"
}
```
When a rollover condition in the lifecycle policy is met, the `rollover` action:
- Creates the second generation backing index, named `.ds-timeseries-2099.03.08-000002`. Because it is a backing index of the `timeseries` data stream, the configuration from the `timeseries_template` index template is applied to the new index.
- As it is the latest generation index of the `timeseries` data stream, the newly created backing index `.ds-timeseries-2099.03.08-000002` becomes the data stream’s write index.
This process repeats each time a rollover condition is met. You can search across all of the data stream’s backing indices, managed by the `timeseries_policy`, with the `timeseries` data stream name. Write operations should be sent to the data stream name, which will route them to its current write index. Read operations against the data stream will be handled by all its backing indices.
## Check lifecycle progress
Use Kibana to [view the current status of your managed indices](https://www.elastic.co/elastic/docs-builder/docs/3016/manage-data/lifecycle/index-lifecycle-management/policy-view-status) and details about the ILM policy, or the ILM explain API. Find out things like:
- What phase an index is in and when it entered that phase.
- The current action and what step is being performed.
- If any errors have occurred or progress is blocked.
For example, the following request gets information about the `timeseries` data stream’s backing indices:
```json
```
The following response shows the data stream’s first generation backing index is waiting for the `hot` phase’s `rollover` action. It remains in this state and ILM continues to call `check-rollover-ready` until a rollover condition is met.
```json
{
"indices": {
".ds-timeseries-2099.03.07-000001": {
"index": ".ds-timeseries-2099.03.07-000001",
"index_creation_date_millis": 1538475653281,
"time_since_index_creation": "30s",
"managed": true,
"policy": "timeseries_policy",
"lifecycle_date_millis": 1538475653281,
"age": "30s",
"phase": "hot",
"phase_time_millis": 1538475653317,
"action": "rollover",
"action_time_millis": 1538475653317,
"step": "check-rollover-ready",
"step_time_millis": 1538475653317,
"phase_execution": {
"policy": "timeseries_policy",
"phase_definition": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_primary_shard_size": "50gb",
"max_age": "30d"
}
}
},
"version": 1,
"modified_date_in_millis": 1539609701576
}
}
}
}
```