Custom configuration files and plugins

ECK

To run Elasticsearch with specific plugins or configuration files installed on ECK, you have multiple options. Each option has its own pros and cons.

1. Create a custom container image with the required plugins and configuration files.

   • Pros

     • Deployment is reproducible and reusable.
     • Does not require internet access at runtime.
     • Saves bandwidth and is quicker to start.

   • Cons

     • Requires a container registry and build infrastructure to build and host the custom image.
     • Version upgrades require building a new container image.

2. Use init containers to install plugins and configuration files.

   • Pros

     • Easier to get started and upgrade versions.

   • Cons

     • Requires pods to have internet access. Check the note about using Istio.
     • Adding new Elasticsearch nodes could randomly fail due to network issues or bad configuration.
     • Each Elasticsearch node needs to repeat the download, wasting bandwidth and slowing startup.
     • Deployment manifests are more complicated.

3. Use ConfigMaps or Secrets together with volumes and volume mounts for configuration files.

   • Pros

     • Best choice for injecting configuration files into your Elasticsearch nodes.
     • Follows standard Kubernetes methodology to mount files into Pods.

   • Cons

     • Not valid for plugins installation.
     • Requires to maintain the ConfigMaps or Secrets with the content of the files.

The following sections provide examples for each of the mentioned options.

Refer to Creating custom images for instructions on how to build custom Docker images based on the official Elastic images.

The following example describes option 2, using a repository plugin. To install the plugin before the Elasticsearch nodes start, use an init container to run the plugin installation tool.

spec:
  nodeSets:
  - name: default
    count: 3
    podTemplate:
      spec:
        initContainers:
        - name: install-plugins
          command:
          - sh
          - -c
          - |
            bin/elasticsearch-plugin remove --purge repository-azure
            bin/elasticsearch-plugin install --batch repository-azure

When using Istio, init containers do not have network access, as the Envoy sidecar that provides network connectivity is not started yet. In this scenario, custom containers are the best option. If custom containers are simply not a viable option, then it is possible to adjust the startup command for the Elasticsearch container itself to run the plugin installation before starting Elasticsearch, as the following example describes. Note that this approach will require updating the startup command if it changes in the Elasticsearch image, which could potentially cause failures during upgrades.

spec:
  nodeSets:
  - name: default
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch
          command:
          - /usr/bin/env
          - bash
          - -c
          - |
            #!/usr/bin/env bash
            set -e
            bin/elasticsearch-plugin remove --purge repository-s3 || true
            bin/elasticsearch-plugin install --batch repository-s3
            /bin/tini -- /usr/local/bin/docker-entrypoint.sh

To install custom configuration files you can:

1. Add the configuration data into a ConfigMap or Secret.
2. Use volumes and volume mounts in your manifest to mount the contents of the ConfigMap or Secret as files in your Elasticsearch nodes.

The next example shows how to add a synonyms file for the synonym token filter in Elasticsearch. But you can use the same approach for any kind of file you want to mount into the configuration directory of Elasticsearch, like adding CA certificates of external systems.

1. Create the ConfigMap or Secret with the data:

There are multiple ways to create and mount ConfigMaps and Secrets on Kubernetes. Refer to the official documentation for more details.

This example shows how to create a ConfigMap named synonyms with the content of a local file named my-synonyms.txt added into the synonyms-elasticsearch.txt key of the ConfigMap.

kubectl create configmap synonyms -n <namespace> --from-file=my-synonyms.txt=synonyms-elasticsearch.txt

2. Declare the ConfigMap as a volume and mount it in the Elasticsearch containers.

In this example, modify your Elasticsearch manifest to mount the contents of the synonyms ConfigMap into /usr/share/elasticsearch/config/dictionaries on the Elasticsearch nodes.

spec:
  nodeSets:
  - name: default
    count: 3
    podTemplate:
      spec:
        containers:
        - name: elasticsearch 1
          volumeMounts:
          - name: synonyms
            mountPath: /usr/share/elasticsearch/config/dictionaries 2
        volumes:
        - name: synonyms
          configMap: 3
            name: synonyms 4

1. Elasticsearch runs by convention in a container called elasticsearch. Do not change that value.
2. Use always a path under /usr/share/elasticsearch/config.
3. Use secret instead of configMap if you used a secret to store the data.
4. The ConfigMap name must be the same as the ConfigMap created in the previous step.

After the changes are applied, Elasticsearch nodes should be able to access dictionaries/synonyms-elasticsearch.txt and use it in any configuration setting.