Configure inputs for standalone Elastic Agents

The inputs section of the elastic-agent.yml file specifies how Elastic Agent locates and processes input data.

• Find the settings available for an input
• Sample metrics input configuration
• Sample log files input configuration

Elastic Agent does not define a separate set of settings for each input. Apart from a small number of fields that Elastic Agent handles itself, the settings you specify on an input or a stream are passed through unchanged to the underlying collector, such as Filebeat or Metricbeat.

This means you can use any setting documented for the corresponding Beats input or module, even if it doesn't appear in the examples on this page. To find the settings available for an input, look up its type in Elastic Agent inputs, then follow the link to the Beats documentation for that input or module.

Elastic Agent handles the following input fields itself rather than passing them through:

For example, the Metricbeat System module documents a cpu.metrics setting, and period is a standard Metricbeat module option. Both can be set directly on a stream:

- type: system/metrics
  id: unique-system-metrics-id
  data_stream.namespace: default
  use_output: default
  streams:
    - metricsets:
        - cpu
      data_stream.dataset: system.cpu
      period: 10s
      cpu.metrics: [percentages, normalized_percentages]
		

1. How often the metricsets are collected.
2. Which CPU metrics to report. Supported values are percentages, normalized_percentages, and ticks. Defaults to [percentages].

By default Elastic Agent collects system metrics, such as CPU, memory, network, and file system metrics, and sends them to the default output. For example, to define data streams for cpu, memory, network and filesystem metrics, this is the configuration:

- type: system/metrics
  id: unique-system-metrics-id
  data_stream.namespace: default
  use_output: default
  streams:
    - metricsets:
      - cpu
      data_stream.dataset: system.cpu
    - metricsets:
      - memory
      data_stream.dataset: system.memory
    - metricsets:
      - network
      data_stream.dataset: system.network
    - metricsets:
      - filesystem
      data_stream.dataset: system.filesystem
		

1. The name of the input. Refer to Elastic Agent inputs for the list of what’s available.

2. A unique ID for the input.

3. A user-defined namespace.

4. The name of the output to use. If not specified, default will be used.

5. The set of enabled module metricsets. Refer to the Metricbeat System module for a list of available options. Any of these options can be set on the stream, as described in Find the settings available for an input.

6. A user-defined dataset. It can contain anything that makes sense to signify the source of the data.

To enable Elastic Agent to collect log files, you can use a configuration like the following.

- type: filestream
  id: your-input-id
  streams:
    - id: your-filestream-stream-id
      data_stream:
        dataset: generic
      paths:
        - /var/log/*.log
		

1. The name of the input. Refer to Elastic Agent inputs for the list of what’s available.
2. A unique ID for the input.
3. A unique ID for the data stream to track the state of the ingested files.
4. The streams block is required only if multiple streams are used on the same input. Refer to the Filebeat filestream documentation for a list of available options. Also, specifically for the filestream input type, refer to the simplified log ingestion for an example of ingesting a set of logs specified as an array.

The input in this example harvests all files in the path /var/log/*.log, that is, all logs in the directory /var/log/ that end with .log. All patterns supported by Go Glob are also supported here.

To fetch all files from a predefined level of subdirectories, use this pattern: /var/log/*/*.log. This fetches all .log files from the subfolders of /var/log. It does not fetch log files from the /var/log folder itself. Currently it is not possible to recursively fetch all files in all subdirectories of a directory.