System Integration

<div class="condensed-table">
| | |
| --- | --- |
| Version | 1.64.0 (View all) |
| Compatible Kibana version(s) | 8.17.0 or higher
9.0.0 or higher |
| Supported Serverless project types
What’s this? | Security
Observability |
| Subscription level
What’s this? | Basic |
| Level of support
What’s this? | Elastic |

</div>
The System integration allows you to monitor servers, personal computers, and more.

Use the System integration to collect metrics and logs from your machines. Then visualize that data in Kibana, create alerts to notify you if something goes wrong, and reference data when troubleshooting an issue.

For example, if you wanted to be notified when less than 10% of the disk space is still available, you could install the System integration to send file system metrics to Elastic. Then, you could view real-time updates to disk space used on your system in Kibana’s [Metrics System] Overview dashboard. You could also set up a new rule in the Elastic Observability Metrics app to alert you when the percent free is less than 10% of the total disk space.

The System integration collects two types of data: logs and metrics.

Logs help you keep a record of events that happen on your machine. Log data streams collected by the System integration include application, system, and security events on machines running Windows and auth and syslog events on machines running macOS or Linux. See more details in the Logs reference.

Metrics give you insight into the state of the machine. Metric data streams collected by the System integration include CPU usage, load statistics, memory usage, information on network behavior, and more. See more details in the Metrics reference.

You can enable and disable individual data streams. If all data streams are disabled and the System integration is still enabled, Fleet uses the default data streams.

You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.

Each data stream collects different kinds of metric data, which may require dedicated permissions to be fetched and which may vary across operating systems. Details on the permissions needed for each data stream are available in the Metrics reference.

For step-by-step instructions on how to set up an integration, see the Getting started guide.

Note that certain data streams may access /proc to gather process information, and the resulting ptrace_may_access() call by the kernel to check for permissions can be blocked by AppArmor and other LSM software, even though the System module doesn’t use ptrace directly.

In addition, when running inside a container the proc filesystem directory of the host should be set using system.hostfs setting to /hostfs.

If you specify more than 22 query conditions (event IDs or event ID ranges), some versions of Windows will prevent the integration from reading the event log due to limits in the query system. If this occurs, a similar warning as shown below:

The specified query is invalid.

In some cases, the limit may be lower than 22 conditions. For instance, using a mixture of ranges and single event IDs, along with an additional parameter such as ignore older, results in a limit of 21 conditions.

If you have more than 22 conditions, you can work around this Windows limitation by using a drop_event processor to do the filtering after filebeat has received the events from Windows. The filter shown below is equivalent to event_id: 903, 1024, 2000-2004, 4624 but can be expanded beyond 22 event IDs.

- drop_event.when.not.or:
  - equals.winlog.event_id: "903"
  - equals.winlog.event_id: "1024"
  - equals.winlog.event_id: "4624"
  - range:
      winlog.event_id.gte: 2000
      winlog.event_id.lte: 2004

The Windows application data stream provides events from the Windows Application event log.

• Windows

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The Windows system data stream provides events from the Windows System event log.

• Windows

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The Windows security data stream provides events from the Windows Security event log.

• Windows

{
    "@timestamp": "2019-11-07T10:37:04.226Z",
    "agent": {
        "ephemeral_id": "7b61ba2a-a1b9-4711-87d0-1b3aad5afb85",
        "id": "a152fcd9-5b11-4ed3-9958-e3a95043132d",
        "name": "docker-fleet-agent",
        "type": "filebeat",
        "version": "8.8.0"
    },
    "data_stream": {
        "dataset": "system.security",
        "namespace": "ep",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "a152fcd9-5b11-4ed3-9958-e3a95043132d",
        "snapshot": false,
        "version": "8.8.0"
    },
    "event": {
        "action": "logging-service-shutdown",
        "agent_id_status": "verified",
        "category": [
            "process"
        ],
        "code": "1100",
        "created": "2023-07-18T12:31:50.439Z",
        "dataset": "system.security",
        "ingested": "2023-07-18T12:31:51Z",
        "kind": "event",
        "original": "<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Eventlog' Guid='{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}'/><EventID>1100</EventID><Version>0</Version><Level>4</Level><Task>103</Task><Opcode>0</Opcode><Keywords>0x4020000000000000</Keywords><TimeCreated SystemTime='2019-11-07T10:37:04.226092500Z'/><EventRecordID>14257</EventRecordID><Correlation/><Execution ProcessID='1144' ThreadID='4532'/><Channel>Security</Channel><Computer>WIN-41OB2LO92CR.wlbeat.local</Computer><Security/></System><UserData><ServiceShutdown xmlns='http://manifests.microsoft.com/win/2004/08/windows/eventlog'></ServiceShutdown></UserData></Event>",
        "outcome": "success",
        "provider": "Microsoft-Windows-Eventlog",
        "type": [
            "end"
        ]
    },
    "host": {
        "name": "WIN-41OB2LO92CR.wlbeat.local"
    },
    "input": {
        "type": "httpjson"
    },
    "log": {
        "level": "information"
    },
    "tags": [
        "forwarded",
        "preserve_original_event"
    ],
    "winlog": {
        "channel": "Security",
        "computer_name": "WIN-41OB2LO92CR.wlbeat.local",
        "event_id": "1100",
        "keywords": [
            "Audit Success"
        ],
        "level": "information",
        "opcode": "Info",
        "outcome": "success",
        "process": {
            "pid": 1144,
            "thread": {
                "id": 4532
            }
        },
        "provider_guid": "{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}",
        "provider_name": "Microsoft-Windows-Eventlog",
        "record_id": "14257",
        "time_created": "2019-11-07T10:37:04.226Z"
    }
}

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The auth data stream provides auth logs. It can collect logs from traditional log files (e.g: /var/log/syslog*) or from journald. Both inputs can be enabled at the same time and Conditions can be used to select in which OSes/hosts that input should run.

• macOS prior to 10.8
• Linux

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The syslog data stream provides system logs. It can collect logs from traditional log files (e.g: /var/log/syslog*) or from journald. Both inputs can be enabled at the same time and Conditions can be used to select in which OSes/hosts that input should run.

• macOS
• Linux

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System core data stream provides usage statistics for each CPU core.

• FreeBSD
• Linux
• macOS
• OpenBSD
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System cpu data stream provides CPU statistics.

• FreeBSD
• Linux
• macOS
• OpenBSD
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System diskio data stream provides disk IO metrics collected from the operating system. One event is created for each disk mounted on the system.

• Linux
• macOS (requires 10.10+)
• Windows
• FreeBSD (amd64)

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System filesystem data stream provides file system statistics. For each file system, one document is provided.

• FreeBSD
• Linux
• macOS
• OpenBSD
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System fsstat data stream provides overall file system statistics.

• FreeBSD
• Linux
• macOS
• OpenBSD
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System load data stream provides load statistics.

• FreeBSD
• Linux
• macOS
• OpenBSD

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System memory data stream provides memory statistics.

• FreeBSD
• Linux
• macOS
• OpenBSD
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System network data stream provides network IO metrics collected from the operating system. One event is created for each network interface.

• FreeBSD
• Linux
• macOS
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System process data stream provides process statistics. One document is provided for each process.

• FreeBSD
• Linux
• macOS
• Windows

Process execution data should be available for an authorized user. If running as less privileged user, it may not be able to read process data belonging to other users.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The process_summary data stream collects high level statistics about the running processes.

• FreeBSD
• Linux
• macOS
• Windows

General process summary data should be available without elevated permissions. If the process data belongs to the other users, it will be counted as unknown value.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System socket_summary data stream provides the summary of open network sockets in the host system.

It collects a summary of metrics with the count of existing TCP and UDP connections and the count of listening ports.

• FreeBSD
• Linux
• macOS
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.

The System uptime data stream provides the uptime of the host operating system.

• Linux
• macOS
• OpenBSD
• FreeBSD
• Windows

This data should be available without elevated permissions.

ECS Field Reference

Please refer to the following document for detailed information on ECS fields.