Elastic
Loading

Quickstart: Monitor hosts with Elastic Agent

In this quickstart guide, you’ll learn how to scan your host to detect and collect logs and metrics, then navigate to dashboards to further analyze and explore your observability data. You’ll also learn how to get value out of your observability data.

To scan your host, you’ll run an auto-detection script that downloads and installs Elastic Agent, which is used to collect observability data from the host and send it to Elastic.

The script also generates an Elastic Agent configuration file that you can use with your existing Infrastructure-as-Code tooling.

Prerequisites

  • An Elasticsearch cluster for storing and searching your data, and Kibana for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out Elastic Cloud.

  • A user with the superuser built-in role or the privileges required to onboard data.

    Expand to view required privileges
    • Cluster: ['monitor', 'manage_own_api_key']
    • Index: { names: ['logs-*-*', 'metrics-*-*'], privileges: ['auto_configure', 'create_doc'] }
    • Kibana: { spaces: ['*'], feature: { fleet: ['all'], fleetv2: ['all'] } }

  • Root privileges on the host—required to run the auto-detection script used in this quickstart.

  • An Elastic Observability Serverless project. To learn more, refer to Create an Observability project.
  • A user with the Admin role or higher—required to onboard system logs and metrics. To learn more, refer to Assign user roles and privileges.
  • Root privileges on the host—required to run the auto-detection script used in this quickstart.

Limitations

  • The auto-detection script works on Linux and MacOS only. Support for the lsof command is also required if you want to detect custom log files.
  • If you’ve installed Apache or Nginx in a non-standard location, you’ll need to specify log file paths manually when you run the scan.
  • Because Docker Desktop runs in a VM, its logs are not auto-detected.

Collect your data

  1. In Kibana, go to the Observability UI and click Add Data.

  2. Under What do you want to monitor? select Host, and then select Elastic Agent: Logs & Metrics.

  1. Copy the install command.

    You’ll run this command to download the auto-detection script, scan your system for observability data, and install Elastic Agent.

  2. Open a terminal on the host you want to scan, and run the command.

  3. Review the list of log files:

    • Enter Y to ingest all the log files listed.
    • Enter n to either exclude log files or specify additional log paths. Enter Y to confirm your selections.

  1. Create a new Elastic Observability Serverless project, or open an existing one.

  2. In your Elastic Observability Serverless project, go to Add Data.

  3. Under What do you want to monitor? select Host, and then select Elastic Agent: Logs & Metrics.

  1. Copy the install command.

    You’ll run this command to download the auto-detection script, scan your system for observability data, and install Elastic Agent.

  2. Open a terminal on the host you want to scan, and run the command.

  3. Review the list of log files:

    • Enter Y to ingest all the log files listed.
    • Enter n to either exclude log files or specify additional log paths. Enter Y to confirm your selections.

When the script is done, you’ll see a message like "Elastic Agent is configured and running."

There might be a slight delay before logs and other data are ingested.

Note

Need to scan your host again?

The auto-detection script (auto_detect.sh) is downloaded to the directory where you ran the installation command. You can re-run the script on the same host to detect additional logs. The script will scan the host and reconfigure Elastic Agent with any additional logs that are found. If the script misses any custom logs, you can add them manually by entering n after the script has finished scanning the host.

Visualize your data

After installation is complete and all relevant data is flowing into Elastic, the Visualize your data section will show links to assets you can use to analyze your data. Depending on what type of observability data was collected, the page may link to the following integration assets:

Integration asset Description
Apache Prebuilt dashboard for monitoring Apache HTTP server health using error and access log data.
Custom .log files Discover for analyzing custom logs.
Docker Prebuilt dashboard for monitoring the status and health of Docker containers.
MySQL Prebuilt dashboard for monitoring MySQl server health using error and access log data.
Nginx Prebuilt dashboard for monitoring Nginx server health using error and access log data.
System Prebuilt dashboard for monitoring host status and health using system metrics.
Other prebuilt dashboards Prebuilt dashboards are also available for systems and services not described here,including PostgreSQL, Redis, HAProxy, Kafka, RabbitMQ, Prometheus, Apache Tomcat, and MongoDB.

For example, you can navigate the Host overview dashboard to explore detailed metrics about system usage and throughput. Metrics that indicate a possible problem are highlighted in red.

Host overview dashboard

Get value out of your data

After using the dashboards to examine your data and confirm you’ve ingested all the host logs and metrics you want to monitor, you can use Elastic Observability to gain deeper insight into your data.

For host monitoring, the following capabilities and features are recommended:

Refer to the OBservability overview for a description of other useful features.