Secure your settings

ECE ECK Elastic Cloud Hosted Self Managed

Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. Depending on the settings you need to protect, you can use:

The Elasticsearch keystore and the elasticsearch-keystore tool to manage Elasticsearch settings.
The Kibana keystore and the kibana-keystore tool to manage Kibana settings.
Kubernetes secrets, if you are using Elastic Cloud on Kubernetes.

Important

Only some settings are designed to be read from the keystore. However, the keystore has no validation to block unsupported settings. Adding unsupported settings to the keystore causes reload_secure_settings to fail and if not addressed, Elasticsearch will fail to start. To check whether a setting is supported in the keystore, look for a "Secure" qualifier in the setting reference.

The Elasticsearch keystore

With the Elasticsearch keystore, you can add a key and its secret value, then use the key in place of the secret value when you configure your sensitive settings.

Elastic Cloud

There are three types of secrets that you can use:

Single string - Associate a secret value to a setting.
Multiple strings - Associate multiple keys to multiple secret values.
JSON block/file - Associate multiple keys to multiple secret values in JSON format.

Add secret values

Add keys and secret values to the keystore.

Log in to the Elastic Cloud Console.
Find your deployment on the home page in the Hosted deployments card and select Manage to access it directly. Or, select Hosted deployments to go to the Deployments page to view all of your deployments.

On the Deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.
From your deployment menu, select Security.
Locate Elasticsearch keystore and select Add settings.
On the Create setting window, select the secret Type.
Configure the settings, then select Save.
All the modifications to the non-reloadable keystore take effect only after restarting Elasticsearch. Reloadable keystore changes take effect after issuing a reload_secure_settings API request.

Delete secret values

When your keys and secret values are no longer needed, delete them from the keystore.

Log in to the Elastic Cloud Console.
Find your deployment on the home page in the Hosted deployments card and select Manage to access it directly. Or, select Hosted deployments to go to the Deployments page to view all of your deployments.

On the Deployments page you can narrow your deployments by name, ID, or choose from several other filters. To customize your view, use a combination of filters, or change the format from a grid to a list.
From your deployment menu, select Security.
From the Existing keystores list, use the delete icon next to the Setting Name that you want to delete.
On the Confirm to delete window, select Confirm.
All modifications to the non-reloadable keystore take effect only after restarting Elasticsearch. Reloadable keystore changes take effect after issuing a reload_secure_settings API request.

Elastic Cloud Enterprise

There are three types of secrets that you can use:

Single string - Associate a secret value to a setting.
Multiple strings - Associate multiple keys to multiple secret values.
JSON block/file - Associate multiple keys to multiple secret values in JSON format.

Add secret values

Add keys and secret values to the keystore.

Log into the Cloud UI.
On the Deployments page, select your deployment.

Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
From your deployment menu, select Security.
Locate Elasticsearch keystore and select Add settings.
On the Create setting window, select the secret Type.
Configure the settings, then select Save.
All the modifications to the non-reloadable keystore take effect only after restarting Elasticsearch. Reloadable keystore changes take effect after issuing a reload_secure_settings API request.

Delete secret values

When your keys and secret values are no longer needed, delete them from the keystore.

Log into the Cloud UI.
On the Deployments page, select your deployment.

Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
From your deployment menu, select Security.
From the Existing keystores list, use the delete icon next to the Setting Name that you want to delete.
On the Confirm to delete window, select Confirm.
All modifications to the non-reloadable keystore take effect only after restarting Elasticsearch. Reloadable keystore changes take effect after issuing a reload_secure_settings API request.

Steps

Create the keystore:

		curl -k -X PATCH -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/$DEPLOYMENT_ID/elasticsearch/$REF_ID/keystore \
{
  "secrets": {
    "s3.client.CLIENT_NAME.access_key": {
      "as_file": false
      "value": "ACCESS_KEY_VALUE"
    }
    "s3.client.CLIENT_NAME.secret_key": {
      "value": "SECRET_KEY_VALUE"
    }
  }
}

	

ELASTICSEARCH_CLUSTER_ID: The Elasticsearch cluster ID as shown in the Cloud UI or obtained through the API

List the keys defined in the keystore:

		{
  "secrets": {
    "s3.client.CLIENT_NAME.access_key": {
      "as_file": false
    },
    "s3.client.CLIENT_NAME.secret_key": {
      "as_file": false
    }
  }
}

	

Create the credentials for an S3 or Minio repository:

		curl -k -X PUT -H "Authorization: ApiKey $ECE_API_KEY" https://$COODINATOR_HOST:12443/api/v1/clusters/elasticsearch/$ELASTICSEARCH_CLUSTER_ID/_snapshot/s3-repo
{
  "type": "s3",
  "settings": {
    "bucket": "s3_REPOSITORY_NAME",
    "client": "CLIENT_NAME",
    "base_path": "PATH_NAME"
  }
}

	

Create the credentials for a GCS repository:

		curl -k -X PUT -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/clusters/elasticsearch/ELASTICSEARCH_CLUSTER_ID/_snapshot/s3-repo
{
  "type": "gcs",
  "settings": {
    "bucket": "BUCKET_NAME",
    "base_path": "BASE_PATH_NAME",
    "client": "CLIENT_NAME"
  }
}

	

To use GCS snapshots, the cluster must have the repository-gcs plugin enabled.

Remove keys that are defined in the keystore:

		curl -k -X PATCH -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/$DEPLOYMENT_ID/elasticsearch/$REF_ID/keystore \
{
  "secrets": {
    "KEY_TO_REMOVE": {}
  }
}

	

Verify your credentials

If your credentials are invalid, an administrator can verify that they are correct by checking the keystore field in the cluster metadata.

If the credential values are correct, but do not work, the keystore file could be out of sync on one or more nodes. To sync the keystore file, update the value for the key by using the patch API to delete the key from keystore, then add it back again.

Self-managed

All the modifications to the keystore take effect only after restarting Elasticsearch.

These settings, just like the regular ones in the elasticsearch.yml config file, need to be specified on each node in the cluster. Currently, all secure settings are node-specific settings that must have the same value on every node.

Reloadable secure settings

Just like the settings values in elasticsearch.yml, changes to the keystore contents are not automatically applied to the running Elasticsearch node. Re-reading settings requires a node restart. However, certain secure settings are marked as reloadable. Such settings can be re-read and applied on a running node.

You can define these settings before the node is started, or call the Nodes reload secure settings API after the settings are defined to apply them to a running node.

The values of all secure settings, reloadable or not, must be identical across all cluster nodes. After making the desired secure settings changes, using the bin/elasticsearch-keystore add command, call:

		 POST _nodes/reload_secure_settings {
  "secure_settings_password": "keystore-password" 1
}

	

The password that the Elasticsearch keystore is encrypted with.

This API decrypts, re-reads the entire keystore and validates all settings on every cluster node, but only the reloadable secure settings are applied. Changes to other settings do not go into effect until the next restart. Once the call returns, the reload has been completed, meaning that all internal data structures dependent on these settings have been changed. Everything should look as if the settings had the new value from the start.

When changing multiple reloadable secure settings, modify all of them on each cluster node, then issue a reload_secure_settings call instead of reloading after each modification.

There are reloadable secure settings for:

The Kibana keystore

Self Managed

Some settings are sensitive, and relying on filesystem permissions to protect their values is not sufficient. For this use case, Kibana provides a keystore, and the kibana-keystore tool to manage the settings in the keystore.

Note

Run all commands as the user who runs Kibana.
Any valid Kibana setting can be stored in the keystore securely. Unsupported, extraneous or invalid settings will cause Kibana to fail to start up.

Create the keystore

To create the kibana.keystore, use the create command:

bin/kibana-keystore create

The file kibana.keystore will be created in the config directory defined by the environment variable KBN_PATH_CONF.

To create a password protected keystore use the --password flag.

List settings in the keystore

A list of the settings in the keystore is available with the list command:

bin/kibana-keystore list

Add string settings

Note

Your input will be JSON-parsed to allow for object/array input configurations. To enforce string values, use "double quotes" around your input.

Sensitive string settings, like authentication credentials for Elasticsearch can be added using the add command:

bin/kibana-keystore add the.setting.name.to.set

Once added to the keystore, these setting will be automatically applied to this instance of Kibana when started. For example if you do

bin/kibana-keystore add elasticsearch.username

you will be prompted to provide the value for elasticsearch.username. (Your input will show as asterisks.)

The tool will prompt for the value of the setting. To pass the value through stdin, use the --stdin flag:

		cat /file/containing/setting/value | bin/kibana-keystore add the.setting.name.to.set --stdin

	

Remove settings

To remove a setting from the keystore, use the remove command:

bin/kibana-keystore remove the.setting.name.to.remove

Read settings

To display the configured setting values, use the show command:

bin/kibana-keystore show setting.key

Change password

To change the password of the keystore, use the passwd command:

bin/kibana-keystore passwd

Has password

To check if the keystore is password protected, use the has-passwd command. An exit code of 0 will be returned if the keystore is password protected, and the command will fail otherwise.

bin/kibana-keystore has-passwd