---
title: Send any log file using OTel Collector
description: This guide shows you how to manually configure the Elastic Distribution of OpenTelemetry (EDOT) Collector to send your log data to Elasticsearch by configuring...
url: https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/logs/stream-any-log-file-using-edot-collector
products:
  - Elastic Cloud Serverless
  - Elastic Observability
applies_to:
  - Elastic Cloud Serverless: Generally available
  - Elastic Stack: Generally available
---

# Send any log file using OTel Collector
This guide shows you how to manually configure the Elastic Distribution of OpenTelemetry (EDOT) Collector to send your log data to Elasticsearch by configuring the `otel.yml` file. For an Elastic Agent equivalent, refer to [Send any log file using Elastic Agent](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/logs/stream-any-log-file).
For more OpenTelemetry quickstarts, refer to [EDOT quickstarts](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/get-started/opentelemetry/quickstart).

## Prerequisites

<applies-switch>
  <applies-item title="stack:" applies-to="Elastic Stack: Generally available">
    To follow the steps in this guide, you need an Elastic Stack deployment that includes:
    - Elasticsearch for storing and searching data
    - Kibana for visualizing and managing data
    - Kibana user with `All` privileges on Fleet and Integrations. Because many Integrations assets are shared across spaces, users need the Kibana privileges in all spaces.
    - Integrations Server (included by default in every Elastic Cloud Hosted deployment)
    To get started quickly, create an Elastic Cloud Hosted deployment and host it on AWS, GCP, or Azure. [Try it out for free](https://cloud.elastic.co/registration?page=docs&placement=docs-body).
  </applies-item>

  <applies-item title="serverless:" applies-to="Elastic Cloud Serverless: Generally available">
    The **Admin** role or higher is required to onboard log data. To learn more, refer to [Assign user roles and privileges](/elastic/docs-builder/docs/3028/deploy-manage/users-roles/cloud-organization/user-roles#general-assign-user-roles).
  </applies-item>
</applies-switch>


## Install the EDOT Collector

Complete these steps to install and configure the EDOT Collector and send your log data to Elastic Observability.
<stepper>
  <step title="Download and install the EDOT Collector">
    On your host, download the EDOT Collector installation package that corresponds with your system:
    <tab-set>
      <tab-item title="Linux">
        ```shell
        curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-9.3.2-linux-x86_64.tar.gz
        tar xzvf elastic-agent-9.3.2-linux-x86_64.tar.gz
        ```
      </tab-item>

      <tab-item title="macOS">
        ```shell
        curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-9.3.2-darwin-x86_64.tar.gz
        tar xzvf elastic-agent-9.3.2-darwin-x86_64.tar.gz
        ```
      </tab-item>

      <tab-item title="Windows">
        ```powershell
        # PowerShell 5.0+
        wget https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-9.3.2-windows-x86_64.zip -OutFile elastic-agent-9.3.2-windows-x86_64.zip
        Expand-Archive .\elastic-agent-9.3.2-windows-x86_64.zip
        ```
      </tab-item>
    </tab-set>
  </step>

  <step title="Configure the EDOT Collector">
    Follow these steps to retrieve the managed OTLP endpoint URL for your Serverless project:
    1. In Elastic Cloud Serverless, open your Observability project.
    2. Go to **Add data** → **Application** → **OpenTelemetry**.
    3. Select **Managed OTLP Endpoint** in the second step.
    4. Copy the OTLP endpoint configuration value.
    5. Select **Create API Key** to generate an API key.
    Replace `<ELASTIC_OTLP_ENDPOINT>` and `<ELASTIC_API_KEY>` before applying the following commands:
    <tab-set>
      <tab-item title="Linux">
        ```bash
        ELASTIC_OTLP_ENDPOINT=<ELASTIC_OTLP_ENDPOINT> && \
        ELASTIC_API_KEY=<ELASTIC_API_KEY> && \
        cp ./otel_samples/managed_otlp/logs_metrics_traces.yml ./otel.yml && \
        mkdir -p ./data/otelcol && \
        sed -i "s#\${env:STORAGE_DIR}#${PWD}/data/otelcol#g" ./otel.yml && \
        sed -i "s#\${env:ELASTIC_OTLP_ENDPOINT}#${ELASTIC_OTLP_ENDPOINT}#g" ./otel.yml && \
        sed -i "s#\${env:ELASTIC_API_KEY}#${ELASTIC_API_KEY}#g" ./otel.yml
        ```
      </tab-item>

      <tab-item title="macOS">
        ```bash
        ELASTIC_OTLP_ENDPOINT=<ELASTIC_OTLP_ENDPOINT> && \
        ELASTIC_API_KEY=<ELASTIC_API_KEY> && \
        cp ./otel_samples/managed_otlp/logs_metrics_traces.yml ./otel.yml && \
        mkdir -p ./data/otelcol && \
        sed -i '' "s#\${env:STORAGE_DIR}#${PWD}/data/otelcol#g" ./otel.yml && \
        sed -i '' "s#\${env:ELASTIC_OTLP_ENDPOINT}#${ELASTIC_OTLP_ENDPOINT}#g" ./otel.yml && \
        sed -i '' "s#\${env:ELASTIC_API_KEY}#${ELASTIC_API_KEY}#g" ./otel.yml
        ```
      </tab-item>

      <tab-item title="Windows">
        ```powershell
        Remove-Item -Path .\otel.yml -ErrorAction SilentlyContinue
        Copy-Item .\otel_samples\managed_otlp\logs_metrics_traces.yml .\otel.yml
        New-Item -ItemType Directory -Force -Path .\data\otelcol | Out-Null

        $content = Get-Content .\otel.yml
        $content = $content -replace '\${env:STORAGE_DIR}', "$PWD\data\otelcol"
        $content = $content -replace '\${env:ELASTIC_OTLP_ENDPOINT}', "<ELASTIC_OTLP_ENDPOINT>"
        $content = $content -replace '\${env:ELASTIC_API_KEY}', "<ELASTIC_API_KEY>"
        $content | Set-Content .\otel.yml
        ```
      </tab-item>
    </tab-set>
  </step>

  <step title="Configure log file collection">
    To collect logs from specific log files, you need to modify the `otel.yml` configuration file. The configuration includes receivers, processors, and exporters that handle log data.
    <applies-switch>
      <applies-item title="stack:" applies-to="Elastic Stack: Generally available">
        Here's an example configuration for collecting log files with Elastic Stack:
        <dropdown title="otel.yml for logs collection (Elastic Stack)">
          ```yaml
          receivers:
            # Receiver for platform specific log files
            filelog/platformlogs:
              include: [ /var/log/*.log ]
              retry_on_failure:
                enabled: true
              start_at: end
              storage: file_storage
          #   start_at: beginning

          extensions:
            file_storage:
              directory: ${env:STORAGE_DIR}

          processors:
            resourcedetection:
              detectors: ["system"]
              system:
                hostname_sources: ["os"]
                resource_attributes:
                  host.name:
                    enabled: true
                  host.id:
                    enabled: false
                  host.arch:
                    enabled: true
                  host.ip:
                    enabled: true
                  host.mac:
                    enabled: true
                  host.cpu.vendor.id:
                    enabled: true
                  host.cpu.family:
                    enabled: true
                  host.cpu.model.id:
                    enabled: true
                  host.cpu.model.name:
                    enabled: true
                  host.cpu.stepping:
                    enabled: true
                  host.cpu.cache.l2.size:
                    enabled: true
                  os.description:
                    enabled: true
                  os.type:
                    enabled: true

          exporters:
            # Exporter to print the first 5 logs/metrics and then every 1000th
            debug:
              verbosity: detailed
              sampling_initial: 5
              sampling_thereafter: 1000

            # Exporter to send logs and metrics to Elasticsearch
            elasticsearch/otel:
              endpoints: ["${env:ELASTIC_ENDPOINT}"]
              api_key: ${env:ELASTIC_API_KEY}
              mapping:
                mode: otel

          service:
            extensions: [file_storage]
            pipelines:
              logs/platformlogs:
                receivers: [filelog/platformlogs]
                processors: [resourcedetection]
                exporters: [debug, elasticsearch/otel]
          ```
        </dropdown>
      </applies-item>

      <applies-item title="serverless:" applies-to="Elastic Cloud Serverless: Generally available">
        Here's an example configuration for collecting log files with Elastic Cloud Serverless:
        <dropdown title="otel.yml for logs collection (Serverless)">
          ```yaml
          receivers:
            # Receiver for platform specific log files
            filelog/platformlogs:
              include: [/var/log/*.log]
              retry_on_failure:
                enabled: true
              start_at: end
              storage: file_storage
          #   start_at: beginning

          extensions:
            file_storage:
              directory: ${env:STORAGE_DIR}

          processors:
            resourcedetection:
              detectors: ["system"]
              system:
                hostname_sources: ["os"]
                resource_attributes:
                  host.name:
                    enabled: true
                  host.id:
                    enabled: false
                  host.arch:
                    enabled: true
                  host.ip:
                    enabled: true
                  host.mac:
                    enabled: true
                  host.cpu.vendor.id:
                    enabled: true
                  host.cpu.family:
                    enabled: true
                  host.cpu.model.id:
                    enabled: true
                  host.cpu.model.name:
                    enabled: true
                  host.cpu.stepping:
                    enabled: true
                  host.cpu.cache.l2.size:
                    enabled: true
                  os.description:
                    enabled: true
                  os.type:
                    enabled: true

          exporters:
            # Exporter to print the first 5 logs/metrics and then every 1000th
            debug:
              verbosity: detailed
              sampling_initial: 5
              sampling_thereafter: 1000

            # Exporter to send logs and metrics to Elasticsearch Managed OTLP Input
            otlp/ingest:
              endpoint: ${env:ELASTIC_OTLP_ENDPOINT}
              headers:
                Authorization: ApiKey ${env:ELASTIC_API_KEY}

          service:
            extensions: [file_storage]
            pipelines:
              logs/platformlogs:
                receivers: [filelog/platformlogs]
                processors: [resourcedetection]
                exporters: [debug, otlp/ingest]
          ```
        </dropdown>
      </applies-item>
    </applies-switch>
    Key configuration elements:
    - `receivers.filelog/platformlogs.include`: Specifies the path to your log files. You can use patterns like `/var/log/*.log`.
    - `processors.resourcedetection`: Automatically detects and adds host system information to your logs.
    - `extensions.file_storage`: Provides persistent storage for the Collector's state.
    - `exporters`: Configures how data is sent to Elasticsearch (Elastic Stack) or OTLP endpoint (Serverless).
  </step>

  <step title="Run the EDOT Collector">
    Run the following command to run the EDOT Collector:
    <tab-set>
      <tab-item title="Linux and macOS">
        ```bash
        sudo ./otelcol --config otel.yml
        ```
      </tab-item>

      <tab-item title="Windows">
        ```powershell
        .\elastic-agent.exe otel --config otel.yml
        ```
      </tab-item>
    </tab-set>

    <note>
      The Collector opens ports `4317` and `4318` to receive application data from locally running OTel SDKs without authentication. This allows the SDKs to send data without any further configuration needed as they use this endpoint by default.
    </note>
  </step>
</stepper>


## Troubleshoot your EDOT Collector configuration

If you're not seeing your log files in the UI, verify the following:
- The path to your logs file under `include` is correct.
- Your API key is properly set in the environment variables.
- The OTLP endpoint URL is correct and accessible.
- The Collector is running without errors (check the console output).

If you're still running into issues, see [EDOT Collector troubleshooting](https://www.elastic.co/elastic/docs-builder/docs/3028/troubleshoot/ingest/opentelemetry/edot-collector) and [Configure EDOT Collector](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/reference/edot-collector/config).

## Next steps

After you have your EDOT Collector configured and are streaming log data to Elasticsearch:
- Refer to the [Explore log data](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/logs/discover-logs) documentation for information on exploring your log data in the UI, including searching and filtering your log data, getting information about the structure of log fields, and displaying your findings in a visualization.
- Refer to the [Parse and organize logs](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/logs/parse-route-logs) documentation for information on extracting structured fields from your log data, rerouting your logs to different data streams, and filtering and aggregating your log data.
- Refer to the [Filter and aggregate logs](https://www.elastic.co/elastic/docs-builder/docs/3028/solutions/observability/logs/filter-aggregate-logs) documentation for information on filtering and aggregating your log data to find specific information, gain insight, and monitor your systems more efficiently.
- To collect telemetry from applications and use the EDOT Collector as a gateway, instrument your target applications following the setup instructions:
  - [Android](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/android/)
- [.NET](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/dotnet/setup/)
- [iOS](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/ios/)
- [Java](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/java/setup/)
- [Node.js](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/nodejs/setup/)
- [PHP](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/php/setup/)
- [Python](https://www.elastic.co/docs/reference/opentelemetry/edot-sdks/python/setup/)