---
title: Organize your data
description: Learn how to organize your data streams using routing and partitioning.
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/solutions/observability/streams-new/organize-your-data
products:
  - Elastic Cloud Enterprise
  - Elastic Cloud Hosted
  - Elastic Cloud Serverless
  - Elastic Cloud on Kubernetes
  - Elastic Observability
  - Elastic Stack
  - Elasticsearch
  - Kibana
applies_to:
  - Elastic Cloud Serverless: Preview
  - Elastic Stack: Preview since 9.2
---

# Organize your data
When logs from multiple sources flow into a single wired stream, partitioning lets you route subsets of that data into dedicated child streams. Each child stream can then be managed independently, with its own retention policy, processing rules, and field mappings, while automatically inheriting the parent's defaults.
For example, you can route firewall logs to a `logs.otel.firewall` child stream with a 7-day retention, and application logs to a `logs.otel.application` child stream with a 30-day retention, without duplicating any shared configuration.
<note>
  Partitioning is only available on [wired streams](/elastic/docs-content/pull/7062/solutions/observability/streams-new/get-data-in#get-data-in-wired). If you're using classic streams or all your logs need identical treatment, skip this step.
</note>


## Partitioning recommendations

Before creating partitions, keep the following in mind:
- **Partition by logical groupings**, not by high-cardinality fields. Group logs by team, technology type, or environment (for example, `web-servers`, `application`, `security`) rather than by individual service names or host identifiers, which can generate too many streams to manage effectively.
- **Aim for tens of partitions, not hundreds.** Each partition creates a dedicated data stream in Elasticsearch. There is a cost to each one, so keep the number manageable.
- **Only partition when you need different lifecycle policies.** If all your logs can share the same retention and processing rules, a single stream is simpler to operate.


## Partition your data

<stepper>
  <step title="Open the Partitioning tab">
    1. Open Streams from the navigation menu or use the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/explore-analyze/find-and-organize/find-apps-and-objects).
    2. Select your wired stream from the list.
    3. Go to the **Partitioning** tab.
  </step>

  <step title="Create a partition">
    Choose how to define partitions: manually using field-based conditions, or by letting AI analyze your data and suggest groupings.
    <dropdown title="Create partitions manually">
      1. Select **Create partition manually**.
      2. In the **Data preview**, hover over a field and select:
         - `plus_circle` to route data where the field matches the value.
      - `minus_circle` to route data where the field does not match the value.
      3. Under **Stream name**, give the child stream a name that reflects the condition.
      4. Select **Save** to create the child stream.
      Under **Condition**, you can also set the field, comparator, and value directly. Turn on the **Syntax editor** to enter conditions in YAML. For more on conditions, refer to [Streamlang conditions](/elastic/docs-content/pull/7062/solutions/observability/streams/management/streamlang#streams-streamlang-conditions).
    </dropdown>

    <dropdown title="Suggest partitions with AI">
      **Requires a [Generative AI connector](https://docs-v3-preview.elastic.dev/elastic/kibana/tree/main/reference/connectors-kibana/gen-ai-connectors).**<applies-to>Elastic Cloud Serverless: Preview</applies-to> <applies-to>Elastic Stack: Preview since 9.4</applies-to>
      1. Select **Suggest partitions with AI**. Streams analyzes your data and suggests groupings.
      2. Review the suggested partitions, then **Accept** or **Reject** each one.
      3. To refine the results, select **Modify suggestions**, provide guidance (for example, "Partition by service name and severity level"), and submit. Streams regenerates suggestions based on your input.
      4. Continue refining as needed, or select **Try again** to start over.
      5. After accepting, review the generated **Stream name** and **Condition**.
      6. Select **Create stream**.
      <applies-to>Elastic Stack: Preview from 9.2 to 9.3</applies-to>
      1. Select **Suggest partitions with AI**. Streams analyzes your data and suggests groupings.
      2. **Accept** or **Reject** the suggestions. After accepting, review the **Stream name** and **Condition**.
      3. Select **Create stream**.
    </dropdown>
  </step>

  <step title="Review the stream hierarchy">
    After saving, your stream list updates to show the parent-child relationship. For example:
    ```
    logs
    ├── logs.otel.application  [30d retention]
    └── logs.otel.firewall     [7d retention]
    ```
    Child streams automatically inherit the parent's field mappings, lifecycle settings, and processors. You can override any inherited setting at the child level without affecting the parent or other children.
  </step>
</stepper>

After partitioning, each child stream can be configured independently. You're ready to add [processing rules](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/solutions/observability/streams-new/parse-and-process) to extract fields, set [retention policies](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/solutions/observability/streams-new/configure-retention) per stream, or monitor [data quality](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/solutions/observability/streams-new/manage-data-quality) for individual streams.