﻿---
title: Log monitoring
description: Elastic Observability allows you to deploy and manage logs at a petabyte scale, giving you insights into your logs in minutes. You can also search across...
url: https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs
products:
  - Elastic Cloud Serverless
  - Elastic Observability
---

# Log monitoring
Elastic Observability allows you to deploy and manage logs at a petabyte scale, giving you insights into your logs in minutes. You can also search across your logs in one place, troubleshoot in real time, and detect patterns and outliers with categorization and anomaly detection. For more information, refer to the following links:
- [Get started with system logs](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/get-started-with-system-logs): Onboard system log data from a machine or server.
- [Stream any log file](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/stream-any-log-file): Send log files to your Observability project using a standalone Elastic Agent.
- [Parse and route logs](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/parse-route-logs): Parse your log data and extract structured fields that you can use to analyze your data.
- [Filter and aggregate logs](/elastic/docs-builder/docs/2755/solutions/observability/logs/filter-aggregate-logs#logs-filter): Filter and aggregate your log data to find specific information, gain insight, and monitor your systems more efficiently.
- [Explore logs](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/discover-logs): Find information on visualizing and analyzing logs.
- [Run pattern analysis on log data](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/run-pattern-analysis-on-log-data): Find patterns in unstructured log messages and make it easier to examine your data.
- [Troubleshoot logs](https://www.elastic.co/elastic/docs-builder/docs/2755/troubleshoot/observability/troubleshoot-logs): Find solutions for errors you might encounter while onboarding your logs.


## Send log data to your project

You can send log data to your project in different ways depending on your needs. When choosing between these options, consider the different features and functionalities between them.
Refer to [Ingest tools overview](https://www.elastic.co/elastic/docs-builder/docs/2755/manage-data/ingest/tools) for more information on which option best fits your situation.
<tab-set>
  <tab-item title="Elastic Distribution of OpenTelemetry">
    The Elastic Distribution of OpenTelemetry (EDOT) Collector and SDKs provide native OpenTelemetry support for collecting logs, metrics, and traces. This approach is ideal for:
    - Native OpenTelemetry: When you want to use OpenTelemetry standards and are already using OpenTelemetry in your environment.
    - Full observability: When you need to collect logs, metrics, and traces from a single collector.
    - Modern applications: When building new applications with OpenTelemetry instrumentation.
    - Standards compliance: When you need to follow OpenTelemetry specifications.
    For more information, refer to [Elastic Distribution of OpenTelemetry](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/2755/reference/opentelemetry).
  </tab-item>

  <tab-item title="Elastic Agent">
    Elastic Agent uses [integrations](https://www.elastic.co/integrations/data-integrations) to ingest logs from Kubernetes, MySQL, and many more data sources. You have the following options when installing and managing an Elastic Agent:

    #### Fleet-managed Elastic AgentInstall an Elastic Agent and use Fleet to define, configure, and manage your agents in a central location.
    See [install Fleet-managed Elastic Agent](https://www.elastic.co/elastic/docs-builder/docs/2755/reference/fleet/install-fleet-managed-elastic-agent).

    #### Standalone Elastic AgentInstall an Elastic Agent and manually configure it locally on the system where it's installed. You are responsible for managing and upgrading the agents.
    See [install standalone Elastic Agent](https://www.elastic.co/elastic/docs-builder/docs/2755/reference/fleet/install-standalone-elastic-agent).

    #### Elastic Agent in a containerized environmentRun an Elastic Agent inside of a container — either with Fleet Server or standalone.
    See [install Elastic Agent in containers](https://www.elastic.co/elastic/docs-builder/docs/2755/reference/fleet/install-elastic-agents-in-containers).
  </tab-item>

  <tab-item title="Filebeat">
    Filebeat is a lightweight shipper for forwarding and centralizing log data. Installed as a service on your servers, Filebeat monitors the log files or locations that you specify, collects log events, and forwards them to your Observability project for indexing.
    - [Filebeat overview](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/2755/reference/beats/filebeat): General information on Filebeat and how it works.
    - [Filebeat quick start](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/2755/reference/beats/filebeat/filebeat-installation-configuration): Basic installation instructions to get you started.
    - [Set up and run Filebeat](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/2755/reference/beats/filebeat/setting-up-running): Information on how to install, set up, and run Filebeat.
  </tab-item>

  <tab-item title="Logstash">
    Logstash is a powerful data processing pipeline that can collect, transform, and enrich log data before sending it to Elasticsearch. It's ideal for:
    - Complex data processing: When you need to parse, filter, and transform logs before indexing.
    - Multiple data sources: When you need to collect logs from various sources and normalize them.
    - Advanced use cases: When you need data enrichment, aggregation, or routing to multiple destinations.
    - Extending Elastic integrations: When you want to add custom processing to data collected by Elastic Agent or Beats.
    For more information, refer to [Logstash](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/2755/reference/logstash) and [Using Logstash with Elastic integrations](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/2755/reference/logstash/using-logstash-with-elastic-integrations).
  </tab-item>
</tab-set>


## Configure logs

The following resources provide information on configuring your logs:
- [Data streams](https://www.elastic.co/elastic/docs-builder/docs/2755/manage-data/data-store/data-streams): Efficiently store append-only time series data in multiple backing indices partitioned by time and size.
- [Data views](https://www.elastic.co/elastic/docs-builder/docs/2755/explore-analyze/find-and-organize/data-views): Query log entries from the data streams of specific datasets or namespaces.
- [Index lifecycle management](https://www.elastic.co/elastic/docs-builder/docs/2755/manage-data/lifecycle/index-lifecycle-management/tutorial-customize-built-in-policies): Configure the built-in logs policy based on your application’s performance, resilience, and retention requirements.
- [Ingest pipeline](https://www.elastic.co/elastic/docs-builder/docs/2755/manage-data/ingest/transform-enrich/ingest-pipelines): Parse and transform log entries into a suitable format before indexing.
- [Mapping](https://www.elastic.co/elastic/docs-builder/docs/2755/manage-data/data-store/mapping): Define how data is stored and indexed.


## View and monitor logs

Use **Discover** to search, filter, and tail all your logs ingested into your project in one place.
The following resources provide information on viewing and monitoring your logs:
- [Discover and explore](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/discover-logs): Discover and explore all of the log events flowing in from your servers, virtual machines, and containers in a centralized view.
- [Detect log anomalies](https://www.elastic.co/elastic/docs-builder/docs/2755/explore-analyze/machine-learning/anomaly-detection): Use machine learning to detect log anomalies automatically.


## Monitor data sets

The **Data Set Quality** page provides an overview of your data sets and their quality. Use this information to get an idea of your overall data set quality, and find data sets that contain incorrectly parsed documents.
[Monitor data sets](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/data-set-quality-monitoring)

## Application logs

Application logs provide valuable insight into events that have occurred within your services and applications. See [Application logs](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/stream-application-logs).

## Log threshold alert

You can create a rule to send an alert when the log aggregation exceeds a threshold.
Refer to [Log threshold](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/incident-management/create-log-threshold-rule).

## Default logs template

Configure the default `logs` template using the `logs@custom` component template.
Refer to the [Logs index template reference](https://www.elastic.co/elastic/docs-builder/docs/2755/solutions/observability/logs/logs-index-template-reference).