--- title: Use internal collection to send monitoring data description: Use internal collectors to send Beats monitoring data directly to your monitoring cluster. Or as an alternative to internal collection, use Use Metricbeat... url: https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/use-internal-collection-to-send-monitoring-data products: - APM - Elastic Observability applies_to: - Elastic Stack: Generally available --- # Use internal collection to send monitoring data Use internal collectors to send Beats monitoring data directly to your monitoring cluster. Or as an alternative to internal collection, use [Use Metricbeat collection](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/use-metricbeat-to-send-monitoring-data). The benefit of using internal collection instead of Metricbeat is that you have fewer pieces of software to install and maintain. 1. Create an API key or user that has appropriate authority to send system-level monitoring data to Elasticsearch. For example, you can use the built-in `apm_system` user or assign the built-in `apm_system` role to another user. For more information on the required privileges, see [Create a *monitoring* role](/elastic/docs-builder/docs/3016/solutions/observability/apm/create-assign-feature-roles-to-apm-server-users#apm-privileges-to-publish-monitoring). For more information on how to use API keys, see [Grant access using API keys](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/grant-access-using-api-keys). 2. Add the `monitoring` settings in the APM Server configuration file. If you configured the Elasticsearch output and want to send APM Server monitoring events to the same Elasticsearch cluster, specify the following minimal configuration: ```yaml monitoring: enabled: true elasticsearch: api_key: id:api_key username: apm_system password: somepassword ``` If you want to send monitoring events to an [Elastic Cloud](https://cloud.elastic.co/) monitoring cluster, you can use two simpler settings. When defined, these settings overwrite settings from other parts in the configuration. For example: ```yaml monitoring: enabled: true cloud.id: 'staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw==' cloud.auth: 'elastic:YOUR_PASSWORD' ``` If you configured a different output, such as Logstash or you want to send APM Server monitoring events to a separate Elasticsearch cluster (referred to as the *monitoring cluster*), you must specify additional configuration options. For example: ```yaml monitoring: enabled: true cluster_uuid: PRODUCTION_ES_CLUSTER_UUID elasticsearch: hosts: [":", ":"] api_key: id:api_key username: apm_system password: somepassword ``` If you want to use PKI authentication to send monitoring events to Elasticsearch, you must specify a different set of configuration options. For example: ```yaml monitoring: enabled: true cluster_uuid: PRODUCTION_ES_CLUSTER_UUID elasticsearch: hosts: [":", ":"] username: "" ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] ssl.certificate: "/etc/pki/client/cert.pem" ssl.key: "/etc/pki/client/cert.key" ``` You must specify the `username` as `""` explicitly so that the username from the client certificate (`CN`) is used. See [SSL/TLS output settings](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/ssl-tls-output-settings) for more information about SSL settings. 3. Start APM Server. 4. [View the monitoring data in Kibana](https://www.elastic.co/elastic/docs-builder/docs/3016/deploy-manage/monitor/stack-monitoring/kibana-monitoring-data). ## Settings for internal collection Use the following settings to configure internal collection when you are not using Metricbeat to collect monitoring data. You specify these settings in the X-Pack monitoring section of the `apm-server.yml` config file: ### `monitoring.enabled` The `monitoring.enabled` config is a boolean setting to enable or disable monitoring features. If set to `true`, monitoring is enabled. The default value is `false`. ### `monitoring.elasticsearch` The Elasticsearch instances that you want to ship your APM Server metrics to. This configuration option contains the following fields: #### `api_key` The detail of the API key to be used to send monitoring information to Elasticsearch. See [Grant access using API keys](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/grant-access-using-api-keys) for more information. #### `bulk_max_size` The maximum number of metrics to bulk in a single Elasticsearch bulk API index request. The default is `50`. For more information, see [Elasticsearch](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/configure-elasticsearch-output). #### `backoff.init` The number of seconds to wait before trying to reconnect to Elasticsearch after a network error. After waiting `backoff.init` seconds, APM Server tries to reconnect. If the attempt fails, the backoff timer is increased exponentially up to `backoff.max`. After a successful connection, the backoff timer is reset. The default is `1s`. #### `backoff.max` The maximum number of seconds to wait before attempting to connect to Elasticsearch after a network error. The default is `60s`. #### `compression_level` The gzip compression level. Setting this value to `0` disables compression. The compression level must be in the range of `1` (best speed) to `9` (best compression). The default value is `0`. Increasing the compression level reduces the network usage but increases the CPU usage. #### `headers` Custom HTTP headers to add to each request. For more information, see [Elasticsearch](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/configure-elasticsearch-output). #### `hosts` The list of Elasticsearch nodes to connect to. Monitoring metrics are distributed to these nodes in round robin order. For more information, see [Elasticsearch](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/configure-elasticsearch-output). #### `max_retries` The number of times to retry sending the monitoring metrics after a failure. After the specified number of retries, the metrics are typically dropped. The default value is `3`. For more information, see [Elasticsearch](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/configure-elasticsearch-output). #### `parameters` Dictionary of HTTP parameters to pass within the URL with index operations. #### `password` The password that APM Server uses to authenticate with the Elasticsearch instances for shipping monitoring data. #### `metrics.period` The time interval (in seconds) when metrics are sent to the Elasticsearch cluster. A new snapshot of APM Server metrics is generated and scheduled for publishing each period. The default value is 10 * time.Second. #### `state.period` The time interval (in seconds) when state information are sent to the Elasticsearch cluster. A new snapshot of APM Server state is generated and scheduled for publishing each period. The default value is 60 * time.Second. #### `protocol` The name of the protocol to use when connecting to the Elasticsearch cluster. The options are: `http` or `https`. The default is `http`. If you specify a URL for `hosts`, however, the value of protocol is overridden by the scheme you specify in the URL. #### `proxy_url` The URL of the proxy to use when connecting to the Elasticsearch cluster. For more information, see [Elasticsearch](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/configure-elasticsearch-output). #### `timeout` The HTTP request timeout in seconds for the Elasticsearch request. The default is `90`. #### `ssl` Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer (SSL) parameters like the certificate authority (CA) to use for HTTPS-based connections. If the `ssl` section is missing, the host CAs are used for HTTPS connections to Elasticsearch. For more information, see [SSL/TLS output settings](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/ssl-tls-output-settings). #### `username` The user ID that APM Server uses to authenticate with the Elasticsearch instances for shipping monitoring data.