---
title: Get data in
description: Learn how to get data into Streams.
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/solutions/observability/streams-new/get-data-in
products:
  - Elastic Cloud Enterprise
  - Elastic Cloud Hosted
  - Elastic Cloud Serverless
  - Elastic Cloud on Kubernetes
  - Elastic Observability
  - Elastic Stack
  - Elasticsearch
  - Kibana
applies_to:
  - Elastic Cloud Serverless: Generally available
  - Elastic Stack: Generally available since 9.2, Preview in 9.1
---

# Get data in
Streams supports two entry points depending on where your data is today:
- **[Ingest new data](#get-data-in-wired)**: Send logs to a managed endpoint for new ingestion. Best for new deployments, custom logs, and mixed-format sources.
- **[Work with existing data](#get-data-in-classic)**: Work with data already flowing into Elasticsearch. No migration or configuration changes required.


## Prerequisites

Streams requires the following permissions:
<applies-switch>
  <applies-item title="serverless:" applies-to="Elastic Cloud Serverless: Generally available">
    Streams requires these Elastic Cloud Serverless roles:
    - Admin: Ability to manage all Streams
    - Editor/Viewer: Limited access, cannot perform all actions
  </applies-item>

  <applies-item title="stack:" applies-to="Elastic Stack: Generally available">
    To manage all streams, you need the following permissions:
    - **Cluster permissions**: `manage_index_templates`, `manage_ingest_pipelines`, `manage_pipeline`, `read_pipeline`
    - **Data stream level permissions**: `read`, `write`, `create`, `manage`, `monitor`, `manage_data_stream_lifecycle`, `read_failure_store`, `manage_failure_store`, `manage_ilm`.
    To view streams, you need the following permissions:
    - **Data stream level**: `read`, `view_index_metadata`, `monitor`
    For more information, refer to [Cluster privileges](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/security-privileges#privileges-list-cluster) and [Granting privileges for data streams and aliases](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/deploy-manage/users-roles/cluster-or-deployment-auth/granting-privileges-for-data-streams-aliases)
  </applies-item>
</applies-switch>


## Ingest new data

<applies-to>
  - Elastic Cloud Serverless: Preview
  - Elastic Stack: Preview since 9.2
</applies-to>

Wired streams send your documents to a managed endpoint, from which you can route data into child streams based on partitioning rules. Child streams automatically inherit mappings, lifecycle settings, and processors from the parent, and configuration changes propagate through the hierarchy.
To send data to a wired stream, configure your shipper to point to the appropriate endpoint:
<tab-set>
  <tab-item title="OpenTelemetry">
    <note>
      Set the index based on your Elastic Stack version:
      - <applies-to>Elastic Stack: Preview from 9.2 to 9.3</applies-to> Set the index to `logs`. Only the `logs` endpoint is available in these versions.
      - <applies-to>Elastic Cloud Serverless: Preview</applies-to> <applies-to>Elastic Stack: Preview since 9.4</applies-to> Set the index to `logs.otel` or `logs.ecs`, depending on which endpoint you want to use.
    </note>

    ```yaml
    processors:
      transform/logs-streams:
        log_statements:
          - context: resource
            statements:
              - set(attributes["elasticsearch.index"], "logs.otel")
    service:
      pipelines:
        logs:
          receivers: [myreceiver]
          processors: [transform/logs-streams]
          exporters: [elasticsearch, otlp]
    ```
  </tab-item>

  <tab-item title="Filebeat">
    <note>
      Set the index based on your Elastic Stack version:
      - <applies-to>Elastic Stack: Preview from 9.2 to 9.3</applies-to> Set the index to `logs`. Only the `logs` endpoint is available in these versions.
      - <applies-to>Elastic Cloud Serverless: Preview</applies-to> <applies-to>Elastic Stack: Preview since 9.4</applies-to> Set the index to `logs.otel` or `logs.ecs`, depending on which endpoint you want to use.
    </note>

    ```yaml
    filebeat.inputs:
      - type: filestream
        id: my-filestream-id
        index: logs.otel
        enabled: true
        paths:
          - /var/log/*.log

    # No need to install templates for wired streams
    setup:
      template:
        enabled: false

    output.elasticsearch:
      hosts: ["<elasticsearch-host>"]
      api_key: "<your-api-key>"
    ```
  </tab-item>

  <tab-item title="Logstash">
    <note>
      Set the index based on your Elastic Stack version:
      - <applies-to>Elastic Stack: Preview from 9.2 to 9.3</applies-to> Set the index to `logs`. Only the `logs` endpoint is available in these versions.
      - <applies-to>Elastic Cloud Serverless: Preview</applies-to> <applies-to>Elastic Stack: Preview since 9.4</applies-to> Set the index to `logs.otel` or `logs.ecs`, depending on which endpoint you want to use.
    </note>

    ```json
    output {
      elasticsearch {
        hosts => ["<elasticsearch-host>"]
        api_key => "<your-api-key>"
        index => "logs.otel"
        action => "create"
      }
    }
    ```
  </tab-item>

  <tab-item title="Fleet">
    Use the **Custom Logs (Filestream)** integration to send data to wired streams:
    1. Find **Fleet** in the navigation menu or use the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/explore-analyze/find-and-organize/find-apps-and-objects).
    2. Select the **Settings** tab.
    3. Under **Outputs**, find the output you want to use and select the `pencil` icon.
    4. Turn on **Write to logs streams**.
    5. Add the **Custom Logs (Filestream)** integration to an agent policy.
    6. Enable the **Use the "logs" data stream** setting under **Change defaults**.
    7. Under **Where to add this integration**, select an agent policy that uses the output configured in step 4.
  </tab-item>

  <tab-item title="API">
    <note>
      Set the endpoint based on your Elastic Stack version:
      - <applies-to>Elastic Stack: Preview from 9.2 to 9.3</applies-to> Set the endpoint to `logs`. Only the `logs` endpoint is available in these versions.
      - <applies-to>Elastic Cloud Serverless: Preview</applies-to> <applies-to>Elastic Stack: Preview since 9.4</applies-to> Set the endpoint to `logs.otel` or `logs.ecs`, depending on which endpoint you want to use.
    </note>
    Send data to the endpoint using the [Bulk API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk):
    ```json
    POST /logs.otel/_bulk
    { "create": {} }
    { "@timestamp": "2025-05-05T12:12:12", "body": { "text": "Hello world!" }, "resource": { "attributes": { "host.name": "my-host-name" } } }
    { "create": {} }
    { "@timestamp": "2025-05-05T12:12:12", "message": "Hello world!", "host.name": "my-host-name" }
    ```
  </tab-item>
</tab-set>


## Work with existing data

Use classic streams when you want the ease of extracting fields and configuring data retention while working with data that's already being ingested into Elasticsearch.
Classic streams:
- Are based on existing data streams, index templates, and component templates.
- Can follow the data retention policy set in the existing index template.
- Do not support hierarchical inheritance or cascading configuration updates.

No additional configuration is required. Open Streams from Kibana and your existing data streams appear automatically.
<stepper>
  <step title="Open Streams">
    Open Streams from the following places in Kibana:
    - Select **Streams** from the navigation menu or use the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/explore-analyze/find-and-organize/find-apps-and-objects).
    - Open the data stream for a specific document from **Discover**. To do this, expand the details flyout for a document stored in a data stream, and select **Stream** or an action associated with the document's data stream. Streams then opens filtered to the selected data stream.
    You can also access Streams features using the Streams API. <applies-to>Elastic Stack: Preview since 9.1</applies-to> <applies-to>Elastic Cloud Serverless: Preview</applies-to> Refer to the [Streams API documentation](https://www.elastic.co/docs/api/doc/kibana/group/endpoint-streams) for more information.
  </step>

  <step title="Verify data is flowing">
    After configuring your data source, confirm data is appearing in Discover.For wired streams, you first need to make the index pattern available:
    1. Manually [create a data view](/elastic/docs-content/pull/7062/explore-analyze/find-and-organize/data-views#settings-create-pattern) for the wired streams index pattern (`logs,logs.*`).
    2. Add the wired streams index pattern (`logs,logs.*`) to the `observability:logSources` Kibana advanced setting, which you can open from the navigation menu or by using the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/7062/explore-analyze/find-and-organize/find-apps-and-objects).
    Once data appears in Discover, you're ready to start organizing, parsing, and configuring retention for your streams.
  </step>
</stepper>