--- title: Logs description: Collect and forward Azure logs to Elasticsearch with the EDOT Cloud Forwarder for Azure. url: https://docs-v3-preview.elastic.dev/elastic/edot-cloud-forwarder-azure/tree/main/reference/edot-cf-azure/signals/logs products: - EDOT Cloud Forwarder - EDOT Cloud Forwarder for Azure - Elastic Cloud Serverless - Elastic Observability applies_to: - Serverless Observability projects: Preview - Serverless Security projects: Preview - Elastic Distribution of OpenTelemetry Cloud Forwarder for Azure: Preview since 0.7 --- # Logs EDOT Cloud Forwarder for Azure forwards Azure resource logs to Elasticsearch. This page describes the supported sources, the collection methods on the Azure side, the encoding modes available in ECF, and what the resulting documents look like in Elasticsearch. ## Sources EDOT Cloud Forwarder for Azure supports the following log sources: | Source | Description | |---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Activity logs | Subscription-level resource logs that record operations performed on Azure resources, including categories such as Administrative, Security, Service Health, and Resource Health. | | Resource logs | Per-resource diagnostic logs emitted by Azure services (for example, audit logs from Key Vault or query logs from a database). | ## Collection Azure streams logs to the `logs` Event Hub created by EDOT Cloud Forwarder for Azure through **Diagnostic Settings**. Set up a diagnostic setting on each subscription or resource you want to collect logs from, and point its destination at the `logs` event hub. ### Activity logs 1. In the Azure portal, open **Subscriptions** from the search bar. 2. Select the Azure subscription from which you want to collect Activity logs. 3. Go to **Activity log** → **Export Activity Logs** → **Add diagnostic setting**. - In **Diagnostic setting name**, enter a name for the diagnostic setting. - Select the categories you want to collect (for example, all categories under **Logs → Administrative**). - In **Destination details**, select **Stream to an event hub**. - In **Event hub namespace**, select the namespace created by EDOT Cloud Forwarder for Azure. - In **Event hub name (optional)**, select the `logs` event hub created by EDOT Cloud Forwarder for Azure. 4. Select **Save** to create the diagnostic setting. 5. After a few minutes, the diagnostic setting starts streaming the Activity logs to the `logs` event hub. ### Resource logs For per-resource logs, create a diagnostic setting on the Azure resource itself (for example, a Key Vault or a Storage Account) and point its destination at the same `logs` event hub. The available log categories depend on the resource type. ### Verify in Kibana Once the diagnostic setting is active: 1. Go to **Discover** in your Elastic Cloud deployment or Serverless project, and select the `logs-*` data view. 2. Filter by `data_stream.dataset` to browse the logs streamed to Elasticsearch. ## Encoding The encoding mode controls how Azure logs are transformed before being sent to Elasticsearch. | Mode | Description | Data stream dataset | Prerequisites | |---------------------------|-------------------------------------------------------------------------------------------------------------------|-------------------------------------|---------------------------------------------------| | **OTel Native** (default) | Parses logs into OpenTelemetry semantic conventions with structured attributes. | `azure..otel` | None | | **ECS** | Preserves the original Azure log as a string and routes it through the Azure Logs integration for ECS processing. | `azure.events`, routed by category. | [Azure Logs integration](#ecs-mode-prerequisites) | The encoding mode is selected at deployment time through the `logsEncodingExtension` Bicep parameter. Refer to [Configure EDOT Cloud Forwarder for Azure](https://docs-v3-preview.elastic.dev/elastic/edot-cloud-forwarder-azure/tree/main/reference/edot-cf-azure/configure) for details. ### ECS mode prerequisites ECS mode requires the **Azure Logs** integration on your Elasticsearch cluster. The integration provides the ingest pipelines that parse the raw event, route it to the correct data stream based on the `category` field, and apply ECS field mappings. To install the Azure Logs integration: 1. In your Elastic Cloud deployment or Serverless project, go to **Integrations**. 2. Search for **Azure Logs**. 3. Select **Install**. Without the integration, the documents remain in `logs-azure.events-default` unprocessed and might not be indexed correctly. ## How encoding works When an Azure resource log arrives at the Event Hub, EDOT Cloud Forwarder for Azure decodes and transforms it according to the selected encoding mode before sending it to Elasticsearch. The following example shows how the same Azure Activity Log appears in Elasticsearch depending on the encoding mode. This is a simplified Azure Activity Log as it arrives in the Event Hub: ```json { "time": "2025-06-10T14:30:00.1234567Z", "resourceId": "/SUBSCRIPTIONS/TEST-SUB-001/RESOURCEGROUPS/TEST-RG/PROVIDERS/MICROSOFT.COMPUTE/VIRTUALMACHINES/MY-VM", "operationName": "Microsoft.Compute/virtualMachines/write", "category": "Administrative", "resultType": "Success", "correlationId": "abc-123-def-456", "level": "Information", "location": "westeurope", "callerIpAddress": "203.0.113.50", "properties": { "message": "Virtual machine updated successfully" } } ``` The log is parsed into OpenTelemetry semantic conventions. Azure-specific fields are mapped to structured `attributes`, and cloud resource metadata is placed in `resource.attributes`: ```json { "@timestamp": "2025-06-10T14:30:00.123Z", "attributes": { "azure.category": "Administrative", "azure.correlation_id": "abc-123-def-456", "azure.operation.name": "Microsoft.Compute/virtualMachines/write", "azure.result.type": "Success", "network.peer.address": "203.0.113.50" }, "body": { "text": "Virtual machine updated successfully" }, "data_stream": { "dataset": "azure.activity.otel", "namespace": "default", "type": "logs" }, "resource": { "attributes": { "cloud.provider": "azure", "cloud.region": "westeurope", "cloud.resource_id": "/SUBSCRIPTIONS/TEST-SUB-001/RESOURCEGROUPS/TEST-RG/PROVIDERS/MICROSOFT.COMPUTE/VIRTUALMACHINES/MY-VM" } }, "severity_text": "Information" } ``` In ECS mode, EDOT Cloud Forwarder for Azure preserves the original Azure log as a string in the `message` field and sends it to the `logs-azure.events-default` data stream. What happens next depends on whether the Azure Logs integration is installed. The document is stored in `logs-azure.events-default` with the original event as an unparsed string. Without the integration, documents remain unprocessed and might not be indexed correctly or lack the expected field mappings. ```json { "message": "{\"time\":\"2025-06-10T14:30:00.1234567Z\",\"resourceId\":\"/SUBSCRIPTIONS/TEST-SUB-001/RESOURCEGROUPS/TEST-RG/PROVIDERS/MICROSOFT.COMPUTE/VIRTUALMACHINES/MY-VM\",\"operationName\":\"Microsoft.Compute/virtualMachines/write\",\"category\":\"Administrative\",\"resultType\":\"Success\",\"correlationId\":\"abc-123-def-456\",\"level\":\"Information\",\"location\":\"westeurope\",\"callerIpAddress\":\"203.0.113.50\",\"properties\":{\"message\":\"Virtual machine updated successfully\"}}", "data_stream": { "dataset": "azure.events", "namespace": "default", "type": "logs" }, "event": { "dataset": "azure.events" }, "input": { "type": "azure-eventhub" }, "tags": ["forwarded", "preserve_original_event"] } ``` The integration's ingest pipeline parses the `message` string, extracts the `category` field, and routes the document to the appropriate final data stream. Because this log has `"category": "Administrative"`, it is routed to `logs-azure.activitylogs-default` and processed into ECS fields: ```json { "@timestamp": "2025-06-10T14:30:00.123Z", "azure": { "activitylogs": { "category": "Administrative", "operation_name": "Microsoft.Compute/virtualMachines/write", "result_type": "Success" }, "correlation_id": "abc-123-def-456", "resource": { "group": "TEST-RG", "id": "/SUBSCRIPTIONS/TEST-SUB-001/RESOURCEGROUPS/TEST-RG/PROVIDERS/MICROSOFT.COMPUTE/VIRTUALMACHINES/MY-VM", "provider": "MICROSOFT.COMPUTE" }, "subscription_id": "TEST-SUB-001" }, "client": { "ip": "203.0.113.50" }, "cloud": { "provider": "azure" }, "data_stream": { "dataset": "azure.activitylogs", "namespace": "default", "type": "logs" }, "event": { "action": "Microsoft.Compute/virtualMachines/write", "kind": "event" }, "log": { "level": "Information" }, "tags": ["preserve_original_event"] } ``` ## Limitations - Logs are streamed exclusively through Diagnostic Settings. Data Collection Rules are not supported for logs. - ECS mode requires the Azure Logs integration to be installed on the destination Elasticsearch cluster. Without it, documents remain unparsed. - Failed deliveries are persisted as blobs in the `logs-error-container` storage container. Refer to [Troubleshooting](https://docs-v3-preview.elastic.dev/elastic/edot-cloud-forwarder-azure/tree/main/reference/edot-cf-azure/troubleshooting).