Loading

Quickstart for hosts and VMs on Elastic Cloud Hosted

Learn how to set up the Elastic Agent and EDOT SDKs on hosts and VMs with Elastic Cloud Hosted (ECH) to collect host metrics, logs, and application traces. This quickstart uses the Elastic Cloud Managed OTLP Endpoint — the recommended ingestion path for ECH.

Use the Add data screen in Elastic Observability to generate install commands that are already configured with the values you need.

  1. Open Elastic Observability.
  2. Go to Add data.
  3. Select what you want to monitor.
  4. Follow the instructions.
  • An Elastic Cloud Hosted deployment running version 9.0 or later.
  • The host or VM running a supported operating system (Linux, macOS, or Windows).

Follow these steps to deploy the Elastic Agent and EDOT SDKs with ECH.

  1. Download the Elastic Agent

    Download the Elastic Agent for your operating system.

  2. Find your endpoint and create an API key

    Find your endpoint

    1. Log in to the Elastic Cloud Console.
    2. From the home page, find your deployment in Hosted deployments, and select Manage.
    3. In the Application endpoints, cluster and component IDs section, select Managed OTLP.
    4. Copy the public endpoint value.

    Create an API key

    Using Kibana

    1. Go to Stack Management → API keys.

    2. Click Create API key, enter a name, and enable Control security privileges.

    3. In the role descriptors box, enter the following:

      {
  "otlp_writer": {
    "applications": [
      {
        "application": "apm",
        "resources": ["*"],
        "privileges": ["event:write"]
      }
    ]
  }
}

    4. Click Create API key and copy the encoded value.

    Using the Elasticsearch API

    Use the Create API key API:

    				POST /_security/api_key
					{
  "name": "otlp-writer",
  "role_descriptors": {
    "otlp_writer": {
      "applications": [
        {
          "application": "apm",
          "resources": ["*"],
          "privileges": ["event:write"]
        }
      ]
    }
  }
}

    The event:write privilege for the apm application is the minimum required to send data through the Elastic Cloud Managed OTLP Endpoint.

  3. Configure the Elastic Agent

    Replace <ELASTIC_OTLP_ENDPOINT> and <ELASTIC_API_KEY> before applying the following commands.

    ELASTIC_OTLP_ENDPOINT=<ELASTIC_OTLP_ENDPOINT> && \
ELASTIC_API_KEY=<ELASTIC_API_KEY> && \
cp ./otel_samples/managed_otlp/logs_metrics_traces.yml ./otel.yml && \
mkdir -p ./data/otelcol && \
sed -i "s#\${env:STORAGE_DIR}#${PWD}/data/otelcol#g" ./otel.yml && \
sed -i "s#\${env:ELASTIC_OTLP_ENDPOINT}#${ELASTIC_OTLP_ENDPOINT}#g" ./otel.yml && \
sed -i "s#\${env:ELASTIC_API_KEY}#${ELASTIC_API_KEY}#g" ./otel.yml
		
    ELASTIC_OTLP_ENDPOINT=<ELASTIC_OTLP_ENDPOINT> && \
ELASTIC_API_KEY=<ELASTIC_API_KEY> && \
cp ./otel_samples/managed_otlp/logs_metrics_traces.yml ./otel.yml && \
mkdir -p ./data/otelcol && \
sed -i '' "s#\${env:STORAGE_DIR}#${PWD}/data/otelcol#g" ./otel.yml && \
sed -i '' "s#\${env:ELASTIC_OTLP_ENDPOINT}#${ELASTIC_OTLP_ENDPOINT}#g" ./otel.yml && \
sed -i '' "s#\${env:ELASTIC_API_KEY}#${ELASTIC_API_KEY}#g" ./otel.yml
		
    Remove-Item -Path .\otel.yml -ErrorAction SilentlyContinue
Copy-Item .\otel_samples\managed_otlp\logs_metrics_traces.yml .\otel.yml
New-Item -ItemType Directory -Force -Path .\data\otelcol | Out-Null

$content = Get-Content .\otel.yml
$content = $content -replace '\${env:STORAGE_DIR}', "$PWD\data\otelcol"
$content = $content -replace '\${env:ELASTIC_OTLP_ENDPOINT}', "<ELASTIC_OTLP_ENDPOINT>"
$content = $content -replace '\${env:ELASTIC_API_KEY}', "<ELASTIC_API_KEY>"
$content | Set-Content .\otel.yml

    For more details, refer to the configuration reference.

  4. Run the Elastic Agent

    Use the following command to start the Elastic Agent.

    sudo ./otelcol --config otel.yml
		
    .\elastic-agent.exe otel --config otel.yml
    Note

    By default, the Collector opens ports 4317 and 4318 to receive application data from locally running OTel SDKs.

  5. (Optional) Instrument your applications

    To collect telemetry from applications and use the Elastic Agent as a gateway, instrument your target applications following the setup instructions:

    Configure your SDKs to send the data to the local Elastic Agent using OTLP/gRPC (http://localhost:4317) or OTLP/HTTP (http://localhost:4318).

  6. Explore your data

    Go to Kibana and select Dashboards to explore your newly collected data.

If you need to write telemetry directly to Elasticsearch using the elasticsearch exporter — for example, for pipeline customizations not yet supported through Elastic Cloud Managed OTLP Endpoint — follow these steps.

Retrieve your Elasticsearch URL and your API key:

  1. Retrieve the Elasticsearch URL for your Elastic Cloud deployment:

    1. Go to the Elastic Cloud console.
    2. Next to your deployment, select Manage.
    3. Under Applications next to Elasticsearch, select Copy endpoint.

  2. Create an API Key following these instructions.

Replace <ELASTICSEARCH_ENDPOINT> and <ELASTIC_API_KEY> before applying the following commands.

ELASTICSEARCH_ENDPOINT=<ELASTICSEARCH_ENDPOINT> && \
ELASTIC_API_KEY=<ELASTIC_API_KEY> && \
cp ./otel_samples/logs_metrics_traces.yml ./otel.yml && \
mkdir -p ./data/otelcol && \
sed -i "s#\${env:STORAGE_DIR}#${PWD}/data/otelcol#g" ./otel.yml && \
sed -i "s#\${env:ELASTIC_ENDPOINT}#${ELASTICSEARCH_ENDPOINT}#g" ./otel.yml && \
sed -i "s#\${env:ELASTIC_API_KEY}#${ELASTIC_API_KEY}#g" ./otel.yml
		
ELASTICSEARCH_ENDPOINT=<ELASTICSEARCH_ENDPOINT> && \
ELASTIC_API_KEY=<ELASTIC_API_KEY> && \
cp ./otel_samples/logs_metrics_traces.yml ./otel.yml && \
mkdir -p ./data/otelcol && \
sed -i '' "s#\${env:STORAGE_DIR}#${PWD}/data/otelcol#g" ./otel.yml && \
sed -i '' "s#\${env:ELASTIC_ENDPOINT}#${ELASTICSEARCH_ENDPOINT}#g" ./otel.yml && \
sed -i '' "s#\${env:ELASTIC_API_KEY}#${ELASTIC_API_KEY}#g" ./otel.yml
		
Remove-Item -Path .\otel.yml -ErrorAction SilentlyContinue
Copy-Item .\otel_samples\logs_metrics_traces.yml .\otel.yml
New-Item -ItemType Directory -Force -Path .\data\otelcol | Out-Null

$content = Get-Content .\otel.yml
$content = $content -replace '\${env:STORAGE_DIR}', "$PWD\data\otelcol"
$content = $content -replace '\${env:ELASTIC_ENDPOINT}', "<ELASTICSEARCH_ENDPOINT>"
$content = $content -replace '\${env:ELASTIC_API_KEY}', "<ELASTIC_API_KEY>"
$content | Set-Content .\otel.yml

The following issues might occur.

The following error is due to an improperly formatted API key:

Exporting failed. Dropping data.
{"kind": "exporter", "data_type": }
"Unauthenticated desc = ApiKey prefix not found"

Format your API key as "Authorization": "ApiKey <api-key-value-here>" or "Authorization=ApiKey <api-key>" depending on whether you're using a Collector or SDK.

For additional troubleshooting, refer to Troubleshooting common issues with the Elastic Agent and Troubleshooting the EDOT SDKs.