Beats for Elasticsearch Serverless
Beats are lightweight data shippers that send operational data to Elasticsearch. Elastic provides separate Beats for different types of data, such as logs, metrics, and uptime. Depending on what data you want to collect, you might need to install multiple shippers on a single host.
Beats are not hosted by Elastic. You deploy and manage them on your own infrastructure, such as on-premises servers, virtual machines, or containers. Beats work with all Elasticsearch Serverless project types, including Elasticsearch, Observability, and Security projects.
If you're looking for a hosted data collection option that doesn't require managing infrastructure, consider agentless integrations, which run on Elastic's infrastructure and require no agent deployment or maintenance.
| Data | Beats |
|---|---|
| Audit data | Auditbeat |
| Log files and journals | Filebeat |
| Availability | Heartbeat |
| Metrics | Metricbeat |
| Network traffic | Packetbeat |
| Windows event logs | Winlogbeat |
Beats can send data to Elasticsearch directly or through Logstash, where you can further process and enhance the data before visualizing it in Kibana.
To send data to an Elasticsearch Serverless project, configure your Beat to connect using the project's Elasticsearch endpoint URL and an API key.
- Log in to Elastic Cloud.
- Find your Elasticsearch endpoint URL. Select Manage next to your project, then find the Elasticsearch endpoint under Application endpoints, cluster and component IDs. Alternatively, open your project, select the help icon, then select Connection details.
- Create an API key with the appropriate privileges. Refer to Create API key for detailed steps. For information on the required privileges, refer to Grant access using API keys.
In your Beat configuration file (for example, filebeat.yml), set the output.elasticsearch section with your endpoint URL and API key:
output.elasticsearch:
hosts: ["ELASTICSEARCH_ENDPOINT_URL"]
api_key: "YOUR_API_KEY"
Do not use cloud.id or cloud.auth for Elasticsearch Serverless projects. Those settings are for Elastic Cloud Hosted deployments only.
Follow the quick start guide for the Beat you want to use:
- Auditbeat quick start
- Filebeat quick start
- Heartbeat quick start
- Metricbeat quick start
- Packetbeat quick start
- Winlogbeat quick start
When you reach the connection setup step, use the Elasticsearch Serverless configuration from Configure the output instead of the cloud.id or hosts examples shown for other deployment types.
When using Beats with Elasticsearch Serverless, keep the following differences in mind:
- Authentication: Elasticsearch Serverless requires API key authentication. Username and password authentication,
cloud.id, andcloud.authare not supported. - Data stream lifecycle: Elasticsearch Serverless uses data stream lifecycle (DSL) instead of index lifecycle management (ILM). ILM settings in your Beat configuration are ignored. Refer to the data stream lifecycle documentation for details.
- Ingest pipelines: Ingest pipelines work the same way as in other deployment types.