Quickstart: Running Beats on Elastic Cloud on Kubernetes

1. Apply the following specification to deploy Filebeat and collect the logs of all containers running in the Kubernetes cluster. ECK automatically configures the secured connection to an Elasticsearch cluster named quickstart, created in Deploy an Elasticsearch cluster.

   cat <<EOF | kubectl apply -f -
   apiVersion: beat.k8s.elastic.co/v1beta1
   kind: Beat
   metadata:
     name: quickstart
   spec:
     type: filebeat
     version: 9.4.1
     elasticsearchRef:
       name: quickstart
     config:
       filebeat.inputs:
       - type: container
         paths:
         - /var/log/containers/*.log
     daemonSet:
       podTemplate:
         spec:
           dnsPolicy: ClusterFirstWithHostNet
           hostNetwork: true
           securityContext:
             runAsUser: 0
           containers:
           - name: filebeat
             volumeMounts:
             - name: varlogcontainers
               mountPath: /var/log/containers
             - name: varlogpods
               mountPath: /var/log/pods
             - name: varlibdockercontainers
               mountPath: /var/lib/docker/containers
           volumes:
           - name: varlogcontainers
             hostPath:
               path: /var/log/containers
           - name: varlogpods
             hostPath:
               path: /var/log/pods
           - name: varlibdockercontainers
             hostPath:
               path: /var/lib/docker/containers
   EOF
   		

   Check Configuration Examples for more ready-to-use manifests.

2. Monitor Beats.

   Retrieve details about the Filebeat.

   kubectl get beat
   		

   NAME                  HEALTH   AVAILABLE   EXPECTED   TYPE       VERSION   AGE
   quickstart            green    3           3          filebeat   9.4.1     2m
   		

3. List all the Pods belonging to a given Beat.

   kubectl get pods --selector='beat.k8s.elastic.co/name=quickstart-beat-filebeat'
   		

   NAME                                      READY   STATUS    RESTARTS   AGE
   quickstart-beat-filebeat-tkz65            1/1     Running   0          3m45s
   quickstart-beat-filebeat-kx5jt            1/1     Running   0          3m45s
   quickstart-beat-filebeat-nb6qh            1/1     Running   0          3m45s
   		

4. Access logs for one of the Pods.

   kubectl logs -f quickstart-beat-filebeat-tkz65
   		

5. Access logs ingested by Filebeat.

   You have two options:

   • Follow the Elasticsearch deployment guide and run:

     Tip

     If the remote endpoint uses a certificate that is not publicly trusted (for example, one signed by a private or corporate CA), provide the corresponding CA certificate using --cacert /path/to/ca.pem so that curl can verify it.

For testing only, you can use --insecure (or -k) to skip certificate verification. This flag turns off TLS trust checks and should not be used in production. ::::

sh curl -u "elastic:$PASSWORD" "https://localhost:9200/filebeat-*/_search"

* Follow the Kibana deployment guide, log in and go to Kibana > Discover.