---
title: Run Filebeat on Kubernetes
description: You can use Filebeat Docker images on Kubernetes to retrieve and ship container logs. You deploy Filebeat as a DaemonSet to ensure there’s a running instance...
url: https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/running-on-kubernetes
products:
  - Beats
  - Filebeat
applies_to:
  - Elastic Cloud Serverless: Generally available
  - Elastic Stack: Generally available
---

# Run Filebeat on Kubernetes
You can use Filebeat [Docker images](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/running-on-docker) on Kubernetes to retrieve and ship container logs.
<tip>
  Running Elastic Cloud on Kubernetes? See [Run Beats on ECK](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3028/deploy-manage/deploy/cloud-on-k8s/beats).
</tip>


## Kubernetes deploy manifests for Filebeat

You deploy Filebeat as a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) to ensure there’s a running instance on each node of the cluster.
The container logs host folder (`/var/log/containers`) is mounted on the Filebeat container. Filebeat starts an input for the files and begins harvesting them as soon as they appear in the folder.
Everything is deployed under the `kube-system` namespace by default. To change the namespace, modify the manifest file.
To download the manifest file, run:
```sh
curl -L -O https://raw.githubusercontent.com/elastic/beats/9.3/deploy/kubernetes/filebeat-kubernetes.yaml
```

<warning>
  **If you are using Kubernetes 1.7 or earlier:** Filebeat uses a hostPath volume to persist internal data. It’s located under `/var/lib/filebeat-data`. The manifest uses folder autocreation (`DirectoryOrCreate`), which was introduced in Kubernetes 1.8. You need to remove `type: DirectoryOrCreate` from the manifest, and create the host folder yourself.
</warning>

To support runtime environments different from Docker, like CRI-O or containerd, configure the `paths` as follows:
<tab-set>
  <tab-item title="Single input">
    Use a single [filestream](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-input-filestream) input to ingest all container logs.
    ```yaml
    filebeat.inputs:
    - type: filestream
      id: container-logs 
      prospector.scanner.symlinks: true 
      parsers:
        - container: ~
      paths:
        - /var/log/containers/*.log 
      processors:
        - add_kubernetes_metadata:
             host: ${NODE_NAME}
             default_indexers.enabled: false
             default_matchers.enabled: false
             indexers:
                - pod_uid:
             matchers:
                - logs_path:
                     logs_path: "/var/log/pods/" 
                     resource_type: "pod" 
    ```
  </tab-item>

  <tab-item title="One input per container">
    Use [autodiscover](/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover#_kubernetes) to generate a
    [filestream](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-input-filestream) input per
    container.
    ```yaml
     filebeat.autodiscover:
       providers:
         - type: kubernetes
           node: ${NODE_NAME}
           hints.enabled: true
           hints.default_config:
             type: filestream
             id: container-${data.kubernetes.container.id} 
             prospector.scanner.symlinks: true 
             parsers:
               - container: ~
             paths:
               - /var/log/containers/*-${data.kubernetes.container.id}.log 
    ```
  </tab-item>
</tab-set>


## Settings

By default, Filebeat sends events to an existing Elasticsearch deployment, if present. To specify a different destination, change the following parameters in the manifest file:
```yaml
- name: ELASTICSEARCH_HOST
  value: elasticsearch
- name: ELASTICSEARCH_PORT
  value: "9200"
- name: ELASTICSEARCH_USERNAME
  value: elastic
- name: ELASTICSEARCH_PASSWORD
  value: changeme
```


### Running Filebeat on control plane nodes

Kubernetes control plane nodes can use [taints](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) to limit the workloads that can run on them. To run Filebeat on control plane nodes, you may need to update the Daemonset spec to include proper tolerations:
```yaml
spec:
 tolerations:
 - key: node-role.kubernetes.io/control-plane
   effect: NoSchedule
```


### Red Hat OpenShift configuration

If you are using Red Hat OpenShift, you need to specify additional settings in the manifest file and enable the container to run as privileged. Filebeat needs to run as a privileged container to mount logs written on the node (hostPath) and read them.
1. Modify the `DaemonSet` container spec in the manifest file:
   ```yaml
     securityContext:
       runAsUser: 0
       privileged: true
   ```
2. Grant the `filebeat` service account access to the privileged SCC:
   ```shell
   oc adm policy add-scc-to-user privileged system:serviceaccount:kube-system:filebeat
   ```
   This command enables the container to be privileged as an administrator for OpenShift.
3. Override the default node selector for the `kube-system` namespace (or your custom namespace) to allow for scheduling on any node:
   ```shell
   oc patch namespace kube-system -p \
   '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}'
   ```
   This command sets the node selector for the project to an empty string. If you don’t run this command, the default node selector will skip control plane nodes.


## Log rotation

Filebeat supports reading from rotating log files, [including GZIP files](/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-input-filestream#reading-gzip-files).
However, some log rotation strategies can result in lost or duplicate events
when using Filebeat to forward messages. For more information, refer to
[Log rotation results in lost or duplicate events](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/file-log-rotation).
Kubernetes stores logs on `/var/log/pods` and uses symlinks on `/var/log/containers`
for active log files. For full details, refer to the official
[Kubernetes documentation on log rotation](https://kubernetes.io/docs/concepts/cluster-administration/logging/#log-rotation).
Ingest rotated logs by enabling decompression of GZIP files and changing the monitored
path to `/var/log/pods/` instead of `/var/log/containers`, which only contains
active log files.
<important>
  When you change the path on an existing deployment,
  Filebeat reads all existing files in the new directory from the beginning.
  This action causes a one-time re-ingestion of the log files.After the initial scan, Filebeat tracks files normally and will only
  ingest new log data.
</important>

The following are examples of configurations for ingesting rotated log files:

### Single input

Use a single [filestream](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-input-filestream) input to ingest all container logs.
<applies-switch>
  <applies-item title="stack: ga 9.3" applies-to="Elastic Stack: Generally available since 9.3">
    ```yaml
        filebeat.inputs:
           - type: filestream
             id: kubernetes-container-logs
             compression: auto 
             parsers:
                - container: ~
             paths:
                - /var/log/pods/*/*/*.log* 
             prospector:
                scanner:
                   fingerprint.enabled: true
             file_identity.fingerprint: ~
             processors:
                - add_kubernetes_metadata:
                     host: ${NODE_NAME}
                     default_indexers.enabled: false
                     default_matchers.enabled: false
                     indexers:
                        - pod_uid:
                     matchers:
                        - logs_path:
                             logs_path: "/var/log/pods/" 
                             resource_type: "pod" 
    ```

    <note>
      With this configuration, [add_kubernetes_metadata](/elastic/docs-builder/docs/3028/reference/beats/filebeat/add-kubernetes-metadata#_logs_path)
      adds pod metadata, which does not include
      container data (such as `kubernetes.container.name`). If you need container
      metadata, you must consider using autodiscover instead. Refer to the
      [autodiscover documentation](/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover#_kubernetes) for details.
    </note>
  </applies-item>

  <applies-item title="stack: beta 9.2" applies-to="Elastic Stack: Beta since 9.2">
    ```yaml
        filebeat.inputs:
           - type: filestream
             id: kubernetes-container-logs
             gzip_experimental: true 
             parsers:
                - container: ~
             paths:
                - /var/log/pods/*/*/*.log* 
             prospector:
                scanner:
                   fingerprint.enabled: true
             file_identity.fingerprint: ~
             processors:
                - add_kubernetes_metadata:
                     host: ${NODE_NAME}
                     default_indexers.enabled: false
                     default_matchers.enabled: false
                     indexers:
                        - pod_uid:
                     matchers:
                        - logs_path:
                             logs_path: "/var/log/pods/" 
                             resource_type: "pod" 
    ```

    <note>
      With this configuration, [add_kubernetes_metadata](/elastic/docs-builder/docs/3028/reference/beats/filebeat/add-kubernetes-metadata#_logs_path)
      adds pod metadata, which does not include
      container data (such as `kubernetes.container.name`). If you need container
      metadata, you must consider using autodiscover instead. Refer to the
      [autodiscover documentation](/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover#_kubernetes) for details.
    </note>
  </applies-item>
</applies-switch>


### One input per container

Use [autodiscover](/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover#_kubernetes) to generate a
[filestream](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-input-filestream) input per
container.
<applies-switch>
  <applies-item title="stack: ga 9.3" applies-to="Elastic Stack: Generally available since 9.3">
    ```yaml
         filebeat.autodiscover:
            id: kubernetes-container-logs-${data.kubernetes.pod.uid}-${data.kubernetes.container.name}
            compression: auto 
            paths:
              - /var/log/pods/${data.kubernetes.namespace}_${data.kubernetes.pod.name}_${data.kubernetes.pod.uid}/${data.kubernetes.container.name}/*.log* 

            parsers:
              - container: ~
            prospector:
              scanner:
                fingerprint.enabled: true
            file_identity.fingerprint: ~
    ```
  </applies-item>

  <applies-item title="stack: beta 9.2" applies-to="Elastic Stack: Beta since 9.2">
    ```yaml
         filebeat.autodiscover:
            id: kubernetes-container-logs-${data.kubernetes.pod.uid}-${data.kubernetes.container.name}
            gzip_experimental: true 
            paths:
              - /var/log/pods/${data.kubernetes.namespace}_${data.kubernetes.pod.name}_${data.kubernetes.pod.uid}/${data.kubernetes.container.name}/*.log* 

            parsers:
              - container: ~
            prospector:
              scanner:
                fingerprint.enabled: true
            file_identity.fingerprint: ~
    ```
  </applies-item>
</applies-switch>


## Load Kibana dashboards

Filebeat comes packaged with various pre-built Kibana dashboards that you can use to visualize logs from your Kubernetes environment.
If these dashboards are not already loaded into Kibana, you must [install Filebeat](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-installation-configuration) on any system that can connect to the Elastic Stack, and then run the `setup` command to load the dashboards. To learn how, see [Load Kibana dashboards](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/load-kibana-dashboards).
The `setup` command does not load the ingest pipelines used to parse log lines. By default, ingest pipelines are set up automatically the first time you run Filebeat and connect to Elasticsearch.
<important>
  If you are using an output other than Elasticsearch, such as Logstash, you need to:
  - [Load the index template manually](/elastic/docs-builder/docs/3028/reference/beats/filebeat/filebeat-template#load-template-manually)
  - [Load Kibana dashboards](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/load-kibana-dashboards)
  - [Load ingest pipelines](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/load-ingest-pipelines)
</important>


## Deploy

To deploy Filebeat to Kubernetes, run:
```sh
kubectl create -f filebeat-kubernetes.yaml
```

To check the status, run:
```sh
$ kubectl --namespace=kube-system get ds/filebeat

NAME       DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE-SELECTOR   AGE
filebeat   32        32        0         32           0           <none>          1m
```

Log events should start flowing to Elasticsearch. The events are annotated with metadata added by the [add_kubernetes_metadata](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/add-kubernetes-metadata) processor.

## Parsing JSON logs

The application logs from workloads running on Kubernetes are usually in JSON format. In such cases, special handling can be applied to parse the JSON logs properly and decode them into fields.
You can configure [Filebeat autodiscover](https://www.elastic.co/elastic/docs-builder/docs/3028/reference/beats/filebeat/configuration-autodiscover) to identify and parse JSON logs in two different ways:
- [Using templates and `ndjson` parser options](#templates-and-parser-options)
- [Using hints and annotations](#hints-and-annotations)

We will illustrate this using an example of one pod with two containers where only the logs of one container are in JSON format.
Example log:
```
{"type":"log","@timestamp":"2020-11-16T14:30:13+00:00","tags":["warning","plugins","licensing"],"pid":7,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
```


### Using templates and `ndjson` parser options

To use this method to parse the JSON logs in our example, configure autodiscover:
```yaml
filebeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      templates:
        - condition:
            contains:
              kubernetes.container.name: "no-json-logging"
          config:
            - type: filestream
              id: container-${data.kubernetes.container.id}
              prospector.scanner.symlinks: true
              parsers:
                - container: ~
              paths:
                - /var/log/containers/*-${data.kubernetes.container.id}.log
        - condition:
            contains:
              kubernetes.container.name: "json-logging"
          config:
            - type: filestream
              id: container-${data.kubernetes.container.id}
              prospector.scanner.symlinks: true
              parsers:
                - container: ~
                - ndjson:
                    target: ""
                    add_error_key: true
                    message_key: message
              paths:
                - /var/log/containers/*-${data.kubernetes.container.id}.log
```


### Using hints and annotations

To configure autodiscover to parse JSON logs using this method, it is important to annotate the pod to only parse logs of the correct container as JSON logs. To achieve this, construct the annotations like this:
```yaml
co.elastic.logs.<kubernetes_container_name>/json.keys_under_root: "true"
co.elastic.logs.<kubernetes_container_name>/json.add_error_key: "true"
co.elastic.logs.<kubernetes_container_name>/json.message_key: "message"
```

For the example we're using:
1. Configure autodiscover:
   ```yaml
   filebeat.autodiscover:
     providers:
       - type: kubernetes
         node: ${NODE_NAME}
         hints.enabled: true
         hints.default_config:
           type: filestream
           id: container-${data.kubernetes.container.id}
           prospector.scanner.symlinks: true
           parsers:
             - container: ~
           paths:
             - /var/log/containers/*-${data.kubernetes.container.id}.log
   ```
2. Then annotate the pod:
   ```yaml
   annotations:
     co.elastic.logs.json-logging/json.keys_under_root: "true"
     co.elastic.logs.json-logging/json.add_error_key: "true"
     co.elastic.logs.json-logging/json.message_key: "message"
   ```