Loading

Streams-new

Streams allows you to automatically parse, structure, and organize your log data so you can query it immediately, without writing Grok expressions or maintaining custom pipelines.

When an incident hits, Streams gets you to answers faster. AI-powered detection continuously scans your logs for critical signals and surfaces what matters. Instead of manually scanning thousands of log lines, you get a prioritized list of what matters.

Organize logs automatically
Streams uses AI to partition your log data by source and component, without manual regex rules or pipeline configuration. As new log formats arrive, Streams continues to learn and extend its partitioning automatically.
Get meaning from logs
The AI-powered processing pipeline detects log formats and generates parsing rules that extract structured fields from unstructured text. You get clean, queryable data without writing a single GROK expression.
Solve incidents in minutes, not hours
Significant Events detection continuously scans your streams for critical signals: out-of-memory errors, crash loops, certificate expirations, and anomalies.
Reduce time spent on managing pipelines
Streams uses AI to simplify parsing, enrichment, partitioning, and schema updates. You can start investigating issues within minutes, rather than spending weeks on pipeline setup and data engineering.
Control storage costs
By surfacing the most critical logs and automatically structuring data for efficient storage, Streams allows you to retain high-value data without discarding important information, reducing overall storage costs.

This is a quick overview of the main steps to get started with Streams in Kibana. It covers how to get data in, organize it into streams, parse and enrich your logs, set retention policies, and monitor data quality.

This tour is an ideal way to familiarize yourself with the Streams UI and its core workflows. You can follow along directly in your Elastic Cloud or self-managed Elasticsearch environment.

  1. Get data in

    Send logs via OpenTelemetry, Fluentd, Fluentbit, or an Elastic integration. For agentless ingest, send directly to the /logs endpoint.

  2. Organize your data

  3. Parse and process

    Streams automatically organizes your logs by source and component. Accept, adjust, or add partitions manually. Use the Processing tab to parse and extract fields from log messages. Accept AI-generated GROK rules or write your own.

  4. Configure retention

    Use the Retention tab to define how long each stream stores data and to review ingestion volume.

  5. Manage data quality

    Use the Data quality column to filter your streams by data quality status.