Secure HTTP communications

ECE ECK Self Managed

Securing HTTP communications is essential for protecting data transmitted between:

• Kibana and Elasticsearch
• Kibana and web browsers
• External monitoring systems and Elasticsearch
• Any HTTP client and your Elastic Stack deployment

Different deployment types have different requirements:

Self Managed

For self-managed deployments, you need to generate and configure certificates for secure HTTP communications.

Complete all steps in:

• Set up basic security
• Set up basic security plus HTTPS

Secure Sockets Layer (SSL) and Transport Layer Security (TLS) provide encryption for data-in-transit. While these terms are often used interchangeably, Kibana supports only TLS, which supersedes the old SSL protocols.

TLS requires X.509 certificates to authenticate the communicating parties and perform encryption of data-in-transit. Each certificate contains a public key and has and an associated — but separate — private key; these keys are used for cryptographic operations. Kibana supports certificates and private keys in PEM or PKCS#12 format.

In a standard TLS configuration, the server presents a signed certificate to authenticate itself to the client. In a mutual TLS configuration, the client also presents a signed certificate to authenticate itself to the server.

Elasticsearch security features are enabled on your cluster by default, so each request that Kibana (the client) makes to Elasticsearch (the server) is authenticated. Most requests made by end users through Kibana to Elasticsearch are authenticated by using the credentials of the logged-in user.

To enroll Kibana with an Elasticsearch cluster, you pass a generated enrollment token. This token configures Kibana to authenticate with Elasticsearch using a service account token. Kibana also supports mutual TLS authentication with Elasticsearch via a Public Key Infrastructure (PKI) realm. With this setup, Elasticsearch needs to verify the signature on the Kibana client certificate, and it also needs to map the client certificate’s distinguished name (DN) to the appropriate kibana_system role.

If you haven’t already, start Kibana and connect it to Elasticsearch using the enrollment token.

1. Obtain a client certificate and private key for Kibana.

   Kibana uses the client certificate and corresponding private key when connecting to Elasticsearch.

   Note

   This is not the same as the server certificate that Kibana will present to web browsers.

   You may choose to generate a client certificate and private key using the elasticsearch-certutil tool. If you followed the Elasticsearch documentation for generating the certificates authority, then you already have a certificate authority (CA) to sign the Elasticsearch server certificate. You may choose to use the same CA to sign the Kibana client certificate. For example:

   bin/elasticsearch-certutil cert -ca elastic-stack-ca.p12 -name kibana-client -dns <your_kibana_hostname>
   

   This will generate a client certificate and private key in a PKCS#12 file named kibana-client.p12. In this example, the client certificate has a Common Name (CN) of "kibana-client" and a subject alternative name (SAN) of "<your_kibana_hostname>". The SAN may be required if you have hostname verification enabled on Elasticsearch.

2. Obtain the certificate authority (CA) certificate chain for Kibana.

   Elasticsearch needs the appropriate CA certificate chain to properly establish trust when receiving connections from Kibana.

   If you followed the instructions to generate a client certificate, then you will have a PKCS#12 file for Kibana. You can extract the CA certificate chain from this file. For example:

   openssl pkcs12 -in kibana-client.p12 -cacerts -nokeys -out kibana-ca.crt
   

   This will produce a PEM-formatted file named kibana-ca.crt that contains the CA certificate from the PKCS#12 file.

3. Configure Elasticsearch with a PKI realm and a native realm.

   By default, Elasticsearch provides a native realm for authenticating with a username and password. However, to support both a PKI realm (for Kibana) and a native realm (for end users), you must configure each realm in elasticsearch.yml:

   xpack.security.authc.realms.pki.realm1.order: 1
   xpack.security.authc.realms.pki.realm1.certificate_authorities: "/path/to/kibana-ca.crt"
   xpack.security.authc.realms.native.realm2.order: 2
   

4. Configure Elasticsearch to request client certificates.

   By default, Elasticsearch will not request a client certificate when establishing a TLS connection. To change this, you must set up optional client certificate authentication in elasticsearch.yml:

   xpack.security.http.ssl.client_authentication: "optional"
   

5. Restart Elasticsearch.

6. Use Kibana to create a role mapping in Elasticsearch for the client certificate.

   This role mapping will assign the kibana_system role to any user that matches the included mapping rule, which is set to equal the client certificate’s DN attribute:

   

   For more information, see Map external users and groups to roles.

7. Configure Kibana to use the client certificate and private key.

   You need to specify the information required to access your client certificate and corresponding private key.

   1. If your certificate and private key are contained in a PKCS#12 file:

      Specify your PKCS#12 file in kibana.yml:

      elasticsearch.ssl.keystore.path: "/path/to/kibana-client.p12"
      

      If your PKCS#12 file is encrypted, add the decryption password to your Kibana keystore:

      bin/kibana-keystore add elasticsearch.ssl.keystore.password
      

      Tip

      If your PKCS#12 file isn’t protected with a password, depending on how it was generated, you may need to set elasticsearch.ssl.keystore.password to an empty string.

   2. Otherwise, if your certificate and private key are in PEM format:

      Specify your certificate and private key in kibana.yml:

      elasticsearch.ssl.certificate: "/path/to/kibana-client.crt"
      elasticsearch.ssl.key: "/path/to/kibana-client.key"
      

      If your private key is encrypted, add the decryption password to your Kibana keystore:

      bin/kibana-keystore add elasticsearch.ssl.keyPassphrase
      

8. Configure Kibana not to use a username and password for Elasticsearch.

   You must remove the elasticsearch.username and elasticsearch.password settings from kibana.yml. If these are present, Kibana will attempt to use them to authenticate to Elasticsearch via the native realm.

9. Restart Kibana.

These steps enable Kibana to authenticate to Elasticsearch using a certificate. However, end users will only be able to authenticate to Kibana with a username and password. To allow end users to authenticate to Kibana using a client certificate, see Kibana PKI authentication.

When connecting client applications to Elasticsearch, use these best practices:

• Always use HTTPS for all connections
• Validate server certificates to prevent man-in-the-middle attacks
• Use API keys or token-based authentication rather than basic auth where possible
• Implement appropriate connection pooling and retry mechanisms
• Consider mutual TLS for high-security environments

For code examples and library-specific guidance, see HTTP/REST client security.

When sending data from Beats to Elasticsearch, you need to secure those communications with TLS and proper authentication.

Key configuration requirements:

• Configure Beats to use HTTPS when connecting to Elasticsearch
• Set up appropriate authentication (API keys recommended)
• Verify Elasticsearch server certificates

For detailed instructions on configuring Beats security settings, see Configure Beats security.

ECE

For ECE deployments, certificate management and TLS configuration are handled at the platform level. Refer to these guides for ECE-specific security configurations:

• Manage security certificates in ECE
• Allow x509 Certificates Signed with SHA-1
• Configure TLS version

ECK

By default, the operator manages a self-signed certificate with a custom CA for each resource. The CA, the certificate and the private key are each stored in a separate Secret.

> kubectl get secret | grep es-http
hulk-es-http-ca-internal         Opaque                                2      28m
hulk-es-http-certs-internal      Opaque                                2      28m
hulk-es-http-certs-public        Opaque                                1      28m

The public certificate is stored in a secret named <name>-[es|kb|apm|ent|agent]-http-certs-public.

> kubectl get secret hulk-es-http-certs-public -o go-template='{{index .data "tls.crt" | base64decode }}'
-----BEGIN CERTIFICATE-----
MIIDQDCCAiigAwIBAgIQHC4O/RWX15a3/P3upsm3djANBgkqhkiG9w0BAQsFADA6
...
QLYL4zLEby3vRxq65+xofVBJAaM=
-----END CERTIFICATE-----

You can provide your own CA and certificates instead of the self-signed certificate to connect to Elastic stack applications through HTTPS using a Kubernetes secret.

Check Setup your own certificate to learn how to do that.

This example illustrates how to create your own self-signed certificate for the quickstart Elasticsearch cluster using the OpenSSL command line utility. Note the subject alternative name (SAN) entry for quickstart-es-http.default.svc.

$ openssl req -x509 -sha256 -nodes -newkey rsa:4096 -days 365 -subj "/CN=quickstart-es-http" -addext "subjectAltName=DNS:quickstart-es-http.default.svc" -keyout tls.key -out tls.crt
$ kubectl create secret generic quickstart-es-cert --from-file=ca.crt=tls.crt --from-file=tls.crt=tls.crt --from-file=tls.key=tls.key

This example illustrates how to issue a self-signed certificate for the quickstart Elasticsearch cluster using a cert-manager self-signed issuer.

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: quickstart-es-cert
spec:
  isCA: true
  dnsNames:
    - quickstart-es-http
    - quickstart-es-http.default.svc
    - quickstart-es-http.default.svc.cluster.local
  issuerRef:
    kind: Issuer
    name: selfsigned-issuer
  secretName: quickstart-es-cert
  subject:
    organizations:
      - quickstart

Here is how to issue multiple Elasticsearch certificates from a single self-signed CA. This is useful for example for Remote clusters which need to trust each other’s CA, in order to avoid mounting N CAs when a cluster is connected to N other clusters.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca
spec:
  isCA: true
  commonName: selfsigned-ca
  secretName: root-ca-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    kind: ClusterIssuer
    name: selfsigned-issuer
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: ca-issuer
spec:
  ca:
    secretName: root-ca-secret
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: quickstart-es-cert
spec:
  isCA: false
  dnsNames:
    - quickstart-es-http
    - quickstart-es-http.default.svc
    - quickstart-es-http.default.svc.cluster.local
  subject:
    organizations:
      - quickstart
  privateKey:
    algorithm: RSA
    encoding: PKCS1
    size: 2048
  issuerRef:
    kind: Issuer
    name: ca-issuer
  secretName: quickstart-es-cert

To use a custom domain with a self-signed certificate:

spec:
  http:
    service:
      spec:
        type: LoadBalancer
    tls:
      selfSignedCertificate:
        subjectAltNames:
        - ip: 160.46.176.15
        - dns: hulk.example.com

You can bring your own certificate to configure TLS to ensure that communication between HTTP clients and the Elastic Stack application is encrypted.

Create a Kubernetes secret with:

• ca.crt: CA certificate (optional if tls.crt was issued by a well-known CA).
• tls.crt: The certificate.
• tls.key: The private key to the first certificate in the certificate chain.

kubectl create secret generic my-cert --from-file=ca.crt --from-file=tls.crt --from-file=tls.key

Alternatively you can also bring your own CA certificate including a private key and let ECK issue certificates with it. Any certificate SANs you have configured as decribed in Reserve static IP and custom domain will also be respected when issuing certificates with this CA certificate.

Create a Kubernetes secret with:

• ca.crt: CA certificate.
• ca.key: The private key to the CA certificate.

kubectl create secret generic my-cert --from-file=ca.crt --from-file=ca.key

In both cases, you have to reference the secret name in the http.tls.certificate section of the resource manifest.

spec:
  http:
    tls:
      certificate:
        secretName: my-cert

You can explicitly disable TLS for Kibana, APM Server, and the HTTP layer of Elasticsearch.

spec:
  http:
    tls:
      selfSignedCertificate:
        disabled: true

By default a ClusterIP Service is created and associated with the Kibana deployment. If you want to expose Kibana externally with a load balancer, it is recommended to include a custom DNS name or IP in the self-generated certificate.

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana-sample
spec:
  version: 8.16.1
  count: 1
  elasticsearchRef:
    name: "elasticsearch-sample"
  http:
    service:
      spec:
        type: LoadBalancer1
    tls:
      selfSignedCertificate:
        subjectAltNames:
        - ip: 1.2.3.4
        - dns: kibana.example.com

1. default is ClusterIP

If you want to use your own certificate, the required configuration is identical to Elasticsearch. Refer to Set up HTTPS for the Elastic Stack.

You can disable the generation of the self-signed certificate and hence disable TLS. This is not recommended outside of testing clusters.

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana-sample
spec:
  version: 8.16.1
  count: 1
  elasticsearchRef:
    name: "elasticsearch-sample"
  http:
    tls:
      selfSignedCertificate:
        disabled: true