Air gapped install

Self Managed

Some components of the Elastic Stack require additional configuration and local dependencies in order to deploy in environments without internet access. This guide gives an overview of this setup scenario and connects you to existing documentation for individual parts of the stack.

Refer to the section for each Elastic component for air-gapped installation configuration and dependencies in a self-managed Linux environment.

Air-gapped install of Elasticsearch may require additional steps in order to access some of the features. General install and configuration guides are available in Deploy an Elasticsearch cluster.

Specifically:

• To be able to use the GeoIP processor, refer to the GeoIP processor documentation for instructions on downloading and deploying the required databases.
• Refer to Machine learning for instructions on deploying the Elastic Learned Sparse EncodeR (ELSER) natural language processing (NLP) model and other trained machine learning models.

Air-gapped install of Kibana may require a number of additional services in the local network in order to access some of the features. General install and configuration guides are available in the Kibana install documentation.

Specifically:

• To be able to use Kibana mapping visualizations, you need to set up and configure the Elastic Maps Service.
• To be able to use Kibana sample data, install or update hundreds of prebuilt alert rules, and explore available data integrations, you need to set up and configure the Elastic Package Registry.
• To provide detection rule updates for Endpoint Security agents, you need to set up and configure the Elastic Endpoint Artifact Repository.
• To access the APM integration, you need to set up and configure Elastic APM.
• To install and use the Elastic documentation for Kibana AI assistants, you need to set up and configure the Elastic product documentation for Kibana.

Elastic Beats are light-weight data shippers. They do not require any special configuration in air-gapped environments. To learn more, refer to the Beats documentation.

Logstash is a versatile data shipping and processing application. It does not require any special configuration in air-gapped environments. To learn more, refer to the Logstash documentation.

Air-gapped install of Elastic Agent depends on the Elastic Package Registry and the Elastic Artifact Registry for most use-cases. The agent itself is fairly lightweight and installs dependencies only as required by its configuration. In terms of connections to these dependencies, Elastic Agents need to be able to connect to the Elastic Artifact Registry directly, but Elastic Package Registry connections are handled through Kibana.

Additionally, if the Elastic Agent Elastic Defend integration is used, then access to the Elastic Endpoint Artifact Repository is necessary in order to deploy updates for some of the detection and prevention capabilities.

To learn more about install and configuration, refer to the Elastic Agent install documentation. Make sure to check the requirements specific to running Elastic Agents in an air-gapped environment.

Fleet Server is a required middleware component for any scalable deployment of the Elastic Agent. The air-gapped dependencies of Fleet Server are the same as those of the Elastic Agent.

To learn more about installing Fleet Server, refer to the Fleet Server set up documentation.

Air-gapped setup of the APM server is possible in two ways:

• By setting up one of the Elastic Agent deployments with an APM integration, as described in Switch a self-installation to the APM integration. See air gapped installation guidance for Elastic Agent.
• Or, by installing a standalone Elastic APM Server, as described in the APM configuration documentation.

Refer to Connect to Elastic Maps Service in the Kibana documentation to learn how to configure your firewall to connect to Elastic Maps Service, host it locally, or disable it completely.

Air-gapped install of the EPR is possible using any OCI-compatible runtime like Podman (a typical choice for RHEL-like Linux systems) or Docker. Links to the official container image and usage guide is available on the Air-gapped environments page in the Fleet and Elastic Agent Guide.

#!/usr/bin/env bash

EPR_BIND_ADDRESS="0.0.0.0"
EPR_BIND_PORT="8443"
EPR_TLS_CERT="/etc/elastic/epr/epr.pem"
EPR_TLS_KEY="/etc/elastic/epr/epr-key.pem"
EPR_IMAGE="docker.elastic.co/package-registry/distribution:9.0.0"

podman create \
  --name "elastic-epr" \
  -p "$EPR_BIND_ADDRESS:$EPR_BIND_PORT:$EPR_BIND_PORT" \
  -v "$EPR_TLS_CERT:/etc/ssl/epr.crt:ro" \
  -v "$EPR_TLS_KEY:/etc/ssl/epr.key:ro" \
  -e "EPR_ADDRESS=0.0.0.0:$EPR_BIND_PORT" \
  -e "EPR_TLS_CERT=/etc/ssl/epr.crt" \
  -e "EPR_TLS_KEY=/etc/ssl/epr.key" \
  "$EPR_IMAGE"

## creates service file in the root directory
# podman generate systemd --new --files --name elastic-epr --restart-policy always

# container-elastic-epr.service
# autogenerated by Podman 4.1.1
# Wed Oct 19 13:12:33 UTC 2022

[Unit]
Description=Podman container-elastic-epr.service
Documentation=man:podman-generate-systemd(1)
Wants=network-online.target
After=network-online.target
RequiresMountsFor=%t/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=always
TimeoutStopSec=70
ExecStartPre=/bin/rm -f %t/%n.ctr-id
ExecStart=/usr/bin/podman run \
  --cidfile=%t/%n.ctr-id \
  --cgroups=no-conmon \
  --rm \
  --sdnotify=conmon \
  -d \
  --replace \
  --name elastic-epr \
  -p 0.0.0.0:8443:8443 \
  -v /etc/elastic/epr/epr.pem:/etc/ssl/epr.crt:ro \
  -v /etc/elastic/epr/epr-key.pem:/etc/ssl/epr.key:ro \
  -e EPR_ADDRESS=0.0.0.0:8443 \
  -e EPR_TLS_CERT=/etc/ssl/epr.crt \
  -e EPR_TLS_KEY=/etc/ssl/epr.key docker.elastic.co/package-registry/distribution:9.0.0
ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%n.ctr-id
ExecStopPost=/usr/bin/podman rm -f --ignore --cidfile=%t/%n.ctr-id
Type=notify
NotifyAccess=all

[Install]
WantedBy=default.target

Air-gapped install of the Elastic Artifact Registry is necessary in order to enable Elastic Agent deployments to perform self-upgrades and install certain components which are needed for some of the data integrations (that is, in addition to what is also retrieved from the EPR). To learn more, refer to Host your own artifact registry for binary downloads in the Fleet and Elastic Agent Guide.

#!/usr/bin/env bash
set -o nounset -o errexit -o pipefail

STACK_VERSION=9.0.0
ARTIFACT_DOWNLOADS_BASE_URL=https://artifacts.elastic.co/downloads

DOWNLOAD_BASE_DIR=${DOWNLOAD_BASE_DIR:?"Make sure to set DOWNLOAD_BASE_DIR when running this script"}

COMMON_PACKAGE_PREFIXES="apm-server/apm-server beats/auditbeat/auditbeat beats/elastic-agent/elastic-agent beats/filebeat/filebeat beats/heartbeat/heartbeat beats/metricbeat/metricbeat beats/osquerybeat/osquerybeat beats/packetbeat/packetbeat cloudbeat/cloudbeat endpoint-dev/endpoint-security fleet-server/fleet-server"

WIN_ONLY_PACKAGE_PREFIXES="beats/winlogbeat/winlogbeat"

RPM_PACKAGES="beats/elastic-agent/elastic-agent"
DEB_PACKAGES="beats/elastic-agent/elastic-agent"

function download_packages() {
  local url_suffix="$1"
  local package_prefixes="$2"

  local _url_suffixes="$url_suffix ${url_suffix}.sha512 ${url_suffix}.asc"
  local _pkg_dir=""
  local _dl_url=""

  for _download_prefix in $package_prefixes; do
    for _pkg_url_suffix in $_url_suffixes; do
          _pkg_dir=$(dirname ${DOWNLOAD_BASE_DIR}/${_download_prefix})
          _dl_url="${ARTIFACT_DOWNLOADS_BASE_URL}/${_download_prefix}-${_pkg_url_suffix}"
          (mkdir -p $_pkg_dir && cd $_pkg_dir && curl -O "$_dl_url")
    done
  done
}

# and we download
for _os in linux windows; do
  case "$_os" in
    linux)
      PKG_URL_SUFFIX="${STACK_VERSION}-${_os}-x86_64.tar.gz"
      ;;
    windows)
      PKG_URL_SUFFIX="${STACK_VERSION}-${_os}-x86_64.zip"
      ;;
    *)
      echo "[ERROR] Something happened"
      exit 1
      ;;
  esac

  download_packages "$PKG_URL_SUFFIX" "$COMMON_PACKAGE_PREFIXES"

  if [[ "$_os" = "windows" ]]; then
    download_packages "$PKG_URL_SUFFIX" "$WIN_ONLY_PACKAGE_PREFIXES"
  fi

  if [[ "$_os" = "linux" ]]; then
    download_packages "${STACK_VERSION}-x86_64.rpm" "$RPM_PACKAGES"
    download_packages "${STACK_VERSION}-amd64.deb" "$DEB_PACKAGES"
  fi
done


## selinux tweaks
# semanage fcontext -a -t "httpd_sys_content_t" '/opt/elastic-packages(/.*)?'
# restorecon -Rv /opt/elastic-packages

user  nginx;
worker_processes  2;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log          /var/log/nginx/access.log  main;
    sendfile            on;
    keepalive_timeout   65;

    server {
        listen                  9080 default_server;
        server_name             _;
        root                    /opt/elastic-packages;

        location / {

        }
    }

}

Air-gapped setup of this component is, essentially, identical to the setup of the Elastic Artifact Registry except that different artifacts are served. To learn more, refer to Configure offline endpoints and air-gapped environments in the Elastic Security guide.

Some machine learning features, like natural language processing (NLP), require you to deploy trained models. To learn about deploying machine learning models in an air-gapped environment, refer to:

• Deploy ELSER in an air-gapped environment.
• Install trained models in an air-gapped environment with Eland.

Detailed install and configuration instructions are available in the Kibana AI Assistants settings documentation.