--- title: Configure the Elasticsearch output for APM Server description: The Elasticsearch output sends events directly to Elasticsearch using the Elasticsearch HTTP API. Example configuration: When sending data to a secured... url: https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/configure-elasticsearch-output products: - APM - Elastic Observability applies_to: - Elastic Stack: Generally available --- # Configure the Elasticsearch output for APM Server ![supported deployment methods](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/images/observability-binary-yes-fm-yes.svg)The configuration options described here apply to both APM Server binary and Fleet-managed users, but how they are specified varies: - **APM Server binary users**: Use the syntax in YAML configuration examples shown below in your configuration file. - **Fleet-managed users**: Use the same APM-specific Elasticsearch output settings outlined below, but [configure them in the Fleet UI](/elastic/docs-builder/docs/3016/reference/fleet/fleet-settings#output-settings). The Elasticsearch output sends events directly to Elasticsearch using the Elasticsearch HTTP API. Example configuration: ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] ``` When sending data to a secured cluster through the `elasticsearch` output, APM Server can use any of the following authentication methods: - Basic authentication credentials (username and password). - Token-based (API key) authentication. - Public Key Infrastructure (PKI) certificates. **Basic authentication:** ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] username: "apm_writer" password: "YOUR_PASSWORD" ``` **API key authentication:** ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] api_key: "ZCV7VnwBgnX0T19fN8Qe:KnR6yE41RrSowb0kQ0HWoA" ``` **PKI certificate authentication:** ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] ssl.certificate: "/etc/pki/client/cert.pem" ssl.key: "/etc/pki/client/cert.key" ``` See [Secure communication with Elasticsearch](#apm-securing-communication-elasticsearch) for details on each authentication method. ## Compatibility This output works with all compatible versions of Elasticsearch. See the [Elastic Support Matrix](https://www.elastic.co/support/matrix#matrix_compatibility). ## Configuration options You can specify the following options in the `elasticsearch` section of the `apm-server.yml` config file: ### `enabled` The enabled config is a boolean setting to enable or disable the output. If set to `false`, the output is disabled. The default value is `true`. ### `hosts` The list of Elasticsearch nodes to connect to. The events are distributed to these nodes in round robin order. If one node becomes unreachable, the event is automatically sent to another node. Each Elasticsearch node can be defined as a `URL` or `IP:PORT`. For example: `http://192.15.3.2`, `https://es.found.io:9230` or `192.24.3.2:9300`. If no port is specified, `9200` is used. When a node is defined as an `IP:PORT`, the *scheme* and *path* are taken from the [`protocol`](#apm-protocol-option) and [`path`](#apm-path-option) config options. ```yaml output.elasticsearch: hosts: ["10.45.3.2:9220", "10.45.3.1:9230"] protocol: https path: /elasticsearch ``` In the previous example, the Elasticsearch nodes are available at `https://10.45.3.2:9220/elasticsearch` and `https://10.45.3.1:9230/elasticsearch`. ### `compression_level` The gzip compression level. Setting this value to `0` disables compression. The compression level must be in the range of `1` (best speed) to `9` (best compression). Increasing the compression level will reduce the network usage but will increase the CPU usage. The default value is `5`. ### `escape_html` Configure escaping of HTML in strings. Set to `true` to enable escaping. The default value is `false`. ### `api_key` Instead of using a username and password, you can use API keys to secure communication with Elasticsearch. The value must be the ID of the API key and the API key joined by a colon: `id:api_key`. You *must* set the API key to be configured to **Beats**. Base64 encoded API keys are not currently supported in this configuration. For details on how to create and configure a compatible API key, refer to [Create an API key for writing events](/elastic/docs-builder/docs/3016/solutions/observability/apm/grant-access-using-api-keys#apm-beats-api-key-publish). ![API key dropdown highlighting the Beats option](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/images/observability-apm-api-key-beats.png) ### `username` The basic authentication username for connecting to Elasticsearch. This user needs the privileges required to publish events to Elasticsearch. To create a user like this, see [Create a *writer* role](/elastic/docs-builder/docs/3016/solutions/observability/apm/create-assign-feature-roles-to-apm-server-users#apm-privileges-to-publish-events). ### `password` The basic authentication password for connecting to Elasticsearch. ### `parameters` Dictionary of HTTP parameters to pass within the URL with index operations. ### `protocol` The name of the protocol Elasticsearch is reachable on. The options are: `http` or `https`. The default is `http`. However, if you specify a URL for [`hosts`](#apm-hosts-option), the value of `protocol` is overridden by whatever scheme you specify in the URL. ### `path` An HTTP path prefix that is prepended to the HTTP API calls. This is useful for the cases where Elasticsearch listens behind an HTTP reverse proxy that exports the API under a custom prefix. ### `headers` Custom HTTP headers to add to each request created by the Elasticsearch output. Example: ```yaml output.elasticsearch.headers: X-My-Header: Header contents ``` It is possible to specify multiple header values for the same header name by separating them with a comma. ### `proxy_url` The URL of the proxy to use when connecting to the Elasticsearch servers. The value may be either a complete URL or a "host[:port]", in which case the "http" scheme is assumed. If a value is not specified through the configuration file then proxy environment variables are used. See the [Go documentation](https://golang.org/pkg/net/http/#ProxyFromEnvironment) for more information about the environment variables. ### `max_retries` The number of times to retry publishing an event after a publishing failure. After the specified number of retries, the events are typically dropped. Set `max_retries` to a value less than 0 to retry until all events are published. The default is 3. ### `flush_bytes` The bulk request size threshold, in bytes, before flushing to Elasticsearch. If compression is enabled, this is compressed bytes. The value must have a suffix, e.g. `"2MB"`. The default is `1MB`. ### `flush_interval` The maximum duration to accumulate events for a bulk request before being flushed to Elasticsearch. The value must have a duration suffix, e.g. `"5s"`. The default is `1s`. ### `backoff.init` The number of seconds to wait before trying to reconnect to Elasticsearch after a network error. After waiting `backoff.init` seconds, APM Server tries to reconnect. If the attempt fails, the backoff timer is increased exponentially up to `backoff.max`. After a successful connection, the backoff timer is reset. The default is `1s`. ### `backoff.max` The maximum number of seconds to wait before attempting to connect to Elasticsearch after a network error. The default is `60s`. ### `timeout` The HTTP request timeout in seconds for the Elasticsearch request. The default is 90. ### `ssl` Configuration options for SSL parameters like the certificate authority to use for HTTPS-based connections. If the `ssl` section is missing, the host CAs are used for HTTPS connections to Elasticsearch. See the [secure communication with Elasticsearch](#apm-securing-communication-elasticsearch) guide or [SSL configuration reference](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/ssl-tls-output-settings) for more information. ## Secure communication with Elasticsearch When sending data to a secured cluster through the `elasticsearch` output, APM Server can use any of the following authentication methods: - Basic authentication credentials (username and password). - Token-based API authentication. - A client certificate. Authentication is specified in the APM Server configuration file: - To use **basic authentication**, specify the `username` and `password` settings under `output.elasticsearch`. For example: ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] username: "apm_writer" password: "YOUR_PASSWORD" ``` - To use token-based **API key authentication**, specify the `api_key` under `output.elasticsearch`. For example: ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] api_key: "KnR6yE41RrSowb0kQ0HWoA" ``` - To use **Public Key Infrastructure (PKI) certificates** to authenticate users, specify the `certificate` and `key` settings under `output.elasticsearch`. For example: ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] ssl.certificate: "/etc/pki/client/cert.pem" ssl.key: "/etc/pki/client/cert.key" ``` These settings assume that the distinguished name (DN) in the certificate is mapped to the appropriate roles in the `role_mapping.yml` file on each node in the Elasticsearch cluster. For more information, see [Using role mapping files](/elastic/docs-builder/docs/3016/deploy-manage/users-roles/cluster-or-deployment-auth/mapping-users-groups-to-roles#mapping-roles-file). By default, APM Server uses the list of trusted certificate authorities (CA) from the operating system where APM Server is running. If the certificate authority that signed your node certificates is not in the host system’s trusted certificate authorities list, you need to add the path to the `.pem` file that contains your CA’s certificate to the APM Server configuration. This will configure APM Server to use a specific list of CA certificates instead of the default list from the OS. Here is an example configuration: ```yaml output.elasticsearch: hosts: ["https://myEShost:9200"] ssl.certificate_authorities: - /etc/pki/my_root_ca.pem - /etc/pki/my_other_ca.pem ssl.certificate: "/etc/pki/client.pem" ssl.key: "/etc/pki/key.pem" ``` For any given connection, the SSL/TLS certificates must have a subject that matches the value specified for `hosts`, or the SSL handshake fails. For example, if you specify `hosts: ["foobar:9200"]`, the certificate MUST include `foobar` in the subject (`CN=foobar`) or as a subject alternative name (SAN). Make sure the hostname resolves to the correct IP address. If no DNS is available, then you can associate the IP address with your hostname in `/etc/hosts` (on Unix) or `C:\Windows\System32\drivers\etc\hosts` (on Windows). ### Learn more about secure communication More information on sending data to a secured cluster is available in the configuration reference: - [SSL/TLS output settings](https://www.elastic.co/elastic/docs-builder/docs/3016/solutions/observability/apm/apm-server/ssl-tls-output-settings)