Add cloud metadata

The add_cloud_metadata processor enriches each event with instance metadata from the machine’s hosting provider. At startup the processor queries a list of hosting providers and caches the instance metadata.

The following providers are supported:

• Amazon Web Services (AWS)
• Digital Ocean
• Google Compute Engine (GCE)
• Tencent Cloud (QCloud)
• Alibaba Cloud (ECS)
• Huawei Cloud (ECS)
• Azure Virtual Machine
• Openstack Nova

The Alibaba Cloud and Tencent providers are disabled by default, because they require to access a remote host. Use the providers setting to select a list of default providers to query.

This configuration enables the processor:

- add_cloud_metadata: ~
		

The metadata that is added to events varies by hosting provider. For examples, refer to Provider-specific metadata examples.

The add_cloud_metadata processor supports SSL options to configure the http client used to query cloud metadata.

For more information, refer to SSL/TLS, specifically the settings under Table 7, Common configuration options and Table 8, Client configuration options.

After the Azure provider detects an Azure VM, the processor automatically makes a best-effort Azure Resource Manager lookup for the AKS cluster name and ID. These fields are optional, but the processor attempts the lookup even when the VM is not an AKS node. Because inputs that collect logs and metrics enable this processor by default, the lookup can occur without an explicit processor configuration.

If TENANT_ID, CLIENT_ID, and CLIENT_SECRET are all present, the processor uses an explicit client secret. Otherwise, it uses the Azure SDK for Go DefaultAzureCredential chain. Depending on the Elastic Agent version, development-focused credentials in the bundled default chain may start Azure CLI, Azure Developer CLI, or Azure PowerShell on Windows. AzurePowerShellCredential is in the chain in Elastic Agent 9.1.8 and later 9.1 releases, 9.2.2 and later 9.2 releases, and all 9.3 and later releases; when included, it invokes PowerShell with an encoded command.

In production, you can exclude development credentials by setting AZURE_TOKEN_CREDENTIALS=prod in the Elastic Agent process environment and restarting Elastic Agent. This setting retains EnvironmentCredential, WorkloadIdentityCredential, and ManagedIdentityCredential. For details, refer to Microsoft's guidance on excluding a credential type category. On Windows, follow the Windows service environment procedure to set the variable and restart Elastic Agent.

The following sections show examples for each of the supported providers.

{
  "cloud": {
    "account.id": "123456789012",
    "availability_zone": "us-east-1c",
    "instance.id": "i-4e123456",
    "machine.type": "t2.medium",
    "image.id": "ami-abcd1234",
    "provider": "aws",
    "region": "us-east-1"
  }
}
		

{
  "cloud": {
    "instance.id": "1234567",
    "provider": "digitalocean",
    "region": "nyc2"
  }
}
		

{
  "cloud": {
    "availability_zone": "us-east1-b",
    "instance.id": "1234556778987654321",
    "machine.type": "f1-micro",
    "project.id": "my-dev",
    "provider": "gcp"
  }
}
		

{
  "cloud": {
    "availability_zone": "gz-azone2",
    "instance.id": "ins-qcloudv5",
    "provider": "qcloud",
    "region": "china-south-gz"
  }
}
		

{
  "cloud": {
    "availability_zone": "cn-east-2b",
    "instance.id": "37da9890-8289-4c58-ba34-a8271c4a8216",
    "provider": "huawei",
    "region": "cn-east-2"
  }
}
		

This metadata is only available when VPC is selected as the network type of the ECS instance.

{
  "cloud": {
    "availability_zone": "cn-shenzhen",
    "instance.id": "i-wz9g2hqiikg0aliyun2b",
    "provider": "ecs",
    "region": "cn-shenzhen-a"
  }
}
		

{
  "cloud": {
    "provider": "azure",
    "instance.id": "04ab04c3-63de-4709-a9f9-9ab8c0411d5e",
    "instance.name": "test-az-vm",
    "machine.type": "Standard_D3_v2",
    "region": "eastus2"
  }
}
		

{
  "cloud": {
    "instance.name": "test-998d932195.mycloud.tld",
    "instance.id": "i-00011a84",
    "availability_zone": "xxxx-az-c",
    "provider": "openstack",
    "machine.type": "m2.large"
  }
}