Configure Elasticsearch

Self Managed

Elasticsearch ships with good defaults and requires very little configuration. Most settings can be changed on a running cluster using the Cluster update settings API.

The configuration files should contain settings which are node-specific (such as node.name and paths), or settings which a node requires in order to be able to join a cluster, such as cluster.name and network.host.

For a complete list of settings that you can apply to your Elasticsearch cluster, refer to the Elasticsearch configuration reference.

For a list of settings that must be configured before using your cluster in production, refer to Important settings configuration.

Elasticsearch has three configuration files:

• elasticsearch.yml for configuring Elasticsearch
• jvm.options for configuring Elasticsearch JVM settings
• log4j2.properties for configuring Elasticsearch logging

These files are located in the config directory, whose default location depends on whether or not the installation is from an archive distribution (tar.gz or zip) or a package distribution (Debian or RPM packages).

For the archive distributions, the config directory location defaults to $ES_HOME/config. The location of the config directory can be changed using the ES_PATH_CONF environment variable:

ES_PATH_CONF=/path/to/my/config ./bin/elasticsearch

Alternatively, you can export the ES_PATH_CONF environment variable through the command line or through your shell profile.

For the package distributions, the config directory location defaults to /etc/elasticsearch.

The location of the config directory can be changed by setting the ES_PATH_CONF environment variable, however, setting the environment variable in your shell is not sufficient. Instead, this variable is sourced from one the following locations:

• Debian: /etc/default/elasticsearch
• RPM: /etc/sysconfig/elasticsearch

You need to edit the ES_PATH_CONF=/etc/elasticsearch entry in the relevant file for your package to change the config directory location.

The configuration format is YAML. Here is an example of changing the path of the data and logs directories:

path:
    data: /var/lib/elasticsearch
    logs: /var/log/elasticsearch

Settings can also be flattened as follows:

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

In YAML, you can format non-scalar values as sequences:

discovery.seed_hosts:
   - 192.168.1.10:9300
   - 192.168.1.11
   - seeds.mydomain.com

Though less common, you can also format non-scalar values as arrays:

discovery.seed_hosts: ["192.168.1.10:9300", "192.168.1.11", "seeds.mydomain.com"]

Environment variables referenced with the ${...} notation within the configuration file will be replaced with the value of the environment variable. For example:

node.name:    ${HOSTNAME}
network.host: ${ES_NETWORK_HOST}

Values for environment variables must be simple strings. Use a comma-separated string to provide values that Elasticsearch will parse as a list. For example, Elasticsearch will split the following string into a list of values for the ${HOSTNAME} environment variable:

export HOSTNAME="host1,host2"

Cluster and node settings can be categorized based on how they are configured:

You can configure and update dynamic settings on a running cluster using the cluster update settings API. You can also configure dynamic settings locally on an unstarted or shut down node using elasticsearch.yml.

Updates made using the cluster update settings API can be persistent, which apply across cluster restarts, or transient, which reset after a cluster restart. You can also reset transient or persistent settings by assigning them a null value using the API.

If you configure the same setting using multiple methods, Elasticsearch applies the settings in following order of precedence:

1. Transient setting
2. Persistent setting
3. elasticsearch.yml setting
4. Default setting value

For example, you can apply a transient setting to override a persistent setting or elasticsearch.yml setting. However, a change to an elasticsearch.yml setting will not override a defined transient or persistent setting.

Use the cluster update settings API to configure dynamic cluster settings. Only use elasticsearch.yml for static cluster settings and node settings. The API doesn’t require a restart and ensures a setting’s value is the same on all nodes.

Static settings can only be configured on an unstarted or shut down node using elasticsearch.yml.

Static settings must be set on every relevant node in the cluster.

Refer to the following documentation to learn how to perform key configuration tasks for Elasticsearch:

• Security: Learn about security features that prevent bad actors from tampering with your data, and encrypt communications to, from, and within your cluster.
• Cluster or deployment users: Set up authentication and authorization for your cluster, and learn about the underlying security technologies that Elasticsearch uses to authenticate and authorize requests internally and across services.
• API keys: Authenticate and authorize programmatic access to your deployments and Elasticsearch resources.
• Connectors: Manage connection information between Elastic and third-party systems.
• Remote clusters: Enable communication between Elasticsearch clusters to support cross-cluster replication and cross-cluster search.
• Add and Remove Elasticsearch nodes: Learn how to add or remove nodes to change the size and capacity of your cluster.
• Production guidance: Review tips and guidance that you can use to design a production environment that matches your workloads, policies, and deployment needs.