---
title: Enable stack monitoring on ECH and ECE deployments
description: The deployment logging and monitoring feature lets you monitor your deployment in Kibana by shipping logs and metrics to a monitoring deployment. You...
url: https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/stack-monitoring/ece-ech-stack-monitoring
products:
- Elastic Cloud Enterprise
- Elastic Cloud Hosted
applies_to:
- Elastic Cloud Hosted: Generally available
- Elastic Cloud Enterprise: Generally available
---
# Enable stack monitoring on ECH and ECE deployments
The deployment logging and monitoring feature lets you monitor your deployment in Kibana by shipping logs and metrics to a monitoring deployment. You can:
- View your deployment’s health and performance in real time and analyze past cluster, index, and node metrics.
- View your deployment’s logs to debug issues, discover slow queries, surface deprecations, and analyze access to your deployment.
Monitoring consists of two components:
- A monitoring and logging agent that is installed on each node in your deployment. The agents collect and index metrics to Elasticsearch, either on the same deployment or by sending logs and metrics to an external monitoring deployment. Elastic manages the installation and configuration of the monitoring agent for you, and you should not modify any of the settings.
- The stack monitoring application in Kibana that visualizes the monitoring metrics through a dashboard, and the logs application that allows you to search and analyze deployment logs.
With logging and monitoring enabled for a deployment, metrics are collected for Elasticsearch, Kibana, and APM with Fleet Server.
AutoOps is a [monitoring](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor) tool that simplifies cluster management through performance recommendations, resource utilization visibility, and real-time issue detection with resolution paths. Learn more about [AutoOps](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/autoops).
## Before you begin
- Some limitations apply when you use monitoring on ECH or ECE. To learn more, check the monitoring [restrictions and limitations](#restrictions-monitoring).
- Enabling logs and monitoring requires additional resources. For production systems where these features are enabled, we recommend allocating at least 4 GB of RAM per Elasticsearch instance. Review [Minimum size recommendations for production use](/elastic/docs-content/pull/8159/deploy-manage/deploy/elastic-cloud/elastic-cloud-hosted-planning#ec-minimum-recommendations) for more details.
## Monitoring for production use
For production use, you should send your deployment logs and metrics to a dedicated monitoring deployment. Monitoring indexes logs and metrics into Elasticsearch and these indexes consume storage, memory, and CPU cycles like any other index. By using a separate monitoring deployment, you avoid affecting your other production deployments and can view the logs and metrics even when a production deployment is unavailable.
How many monitoring deployments you use depends on your requirements:
- You can ship logs and metrics for many deployments to a single monitoring deployment, if your business requirements permit it.
- Although monitoring will work with a deployment running a single node, you need a minimum of three monitoring nodes to make monitoring highly available.
- You might need to create dedicated monitoring deployments for isolation purposes in some cases. For example:
- If you have many deployments and some of them are much larger than others, creating separate monitoring deployments prevents a large deployment from potentially affecting monitoring performance for smaller deployments.
- If you need to silo Elasticsearch data for different business departments. Deployments that have been configured to ship logs and metrics to a target monitoring deployment have access to indexing data and can manage monitoring index templates, which is addressed by creating separate monitoring deployments.
Logs and metrics that get sent to a dedicated monitoring Elasticsearch deployment [may not be cleaned up automatically](#logging-and-monitoring-retention) and might require some additional steps to remove excess data periodically.
## Retention of logging and monitoring indices
When sending monitoring and logging data to a deployment, an ILM policy is pre-configured to control data retention. To view or edit the policies, Go to the **Index Lifecycle Policies** management page in the navigation menu or use the [global search field](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/explore-analyze/find-and-organize/find-apps-and-objects).
On Elastic Cloud Hosted deployments, monitoring indices use the `elastic-cloud-logs` [index lifecycle management (ILM)](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/manage-data/lifecycle/index-lifecycle-management) policy. By default, this policy does not include a [delete phase](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/elasticsearch/index-lifecycle-actions/ilm-delete), which means that logs can grow infinitely and cause a potential disk storage issue in the future. To manage storage effectively, we recommend adding a delete phase with a retention period tailored to your requirements.
## Enable logging and monitoring
Elastic manages the installation and configuration of the monitoring agent for you. When you enable monitoring on a deployment, you are configuring where the monitoring agent for your current deployment should send its logs and metrics.
**Before you start**
- Enabling logging and monitoring increases the resource consumption of the deployment. For production systems, we recommend sizing deployments with logging and monitoring enabled to at least 4 GB of RAM on each Elasticsearch instance.
- Enabling logging and monitoring can trigger a plan change on your deployment. You can monitor the plan change progress from the deployment's **Activity** page.
The monitoring deployment and production deployment must be on the same major version, cloud provider, and region.
To enable monitoring on your deployment:
1. Log in to the [Elastic Cloud Console](https://cloud.elastic.co?page=docs&placement=docs-body) or ECE [Cloud UI](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui).
2. On the home page, find your deployment.
If you have many deployments, you can instead go to the **Hosted deployments** (Elastic Cloud Hosted) or **Deployments** (Elastic Cloud Enterprise) page. On that page, you can narrow your deployments by name, ID, or choose from several other filters.
3. Select **Manage**.
1. From the navigation menu, select **Logs and metrics**.
2. Under **Ship to a deployment**, select **Enable**.
3. Choose where to send your logs and metrics. Select **Save**.
If a deployment is not listed, make sure that it is running a compatible version. The monitoring deployment and production deployment must be on the same major version, cloud provider, and region.
Remember to send logs and metrics for production deployments to a dedicated monitoring deployment, so that your production deployments are not impacted by the overhead of indexing and storing monitoring data. A dedicated monitoring deployment also gives you more control over the retention period for monitoring data.
## Access the monitoring application in Kibana
With monitoring enabled for your deployment, you can access the [logs](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/solutions/observability) and [stack monitoring](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/monitoring-data/visualizing-monitoring-data) through Kibana.
1. Log in to the [Elastic Cloud Console](https://cloud.elastic.co?page=docs&placement=docs-body) or ECE [Cloud UI](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/deploy/cloud-enterprise/log-into-cloud-ui).
2. On the home page, find your deployment.
If you have many deployments, you can instead go to the **Hosted deployments** (Elastic Cloud Hosted) or **Deployments** (Elastic Cloud Enterprise) page. On that page, you can narrow your deployments by name, ID, or choose from several other filters.
3. Select **Manage**.
1. From the navigation menu, select **Logs and metrics**.
2. Select the corresponding **View** button to check the logs or metrics data.
Alternatively, you can access logs and metrics directly on the Kibana **Logs** and **Stack Monitoring** pages in the target monitoring deployment.
You can also create an `elastic-cloud-logs-*` data view (formerly *index pattern*) to view your deployment’s logs in the Kibana **Discover** tab.
Several fields are available for you to view logs based on key details, such as the source deployment:
| Field | Description | Example value |
|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|
| `service.id` | The ID of the deployment that generated the log. This is the most reliable field for identifying the source deployment, it's populated consistently across every component and log category | `6ff525333d2844539663f3b1da6c04b6` |
| `service.name` | The name of the deployment or component that generated the log. Coverage varies by component: not populated for some Elasticsearch log categories (for example, audit and GC logs), and reflects the specific component rather than the deployment name for APM and Fleet Server | `My Production Deployment`, or `apm-server` / `fleet-server` on those instances |
| `cloud.availability_zone` | The availability zone in which the instance that generated the log is deployed | `ap-northeast-1d` |
| `service.node.name` | The ID of the instance that generated the log | `instance-0000000008` |
| `service.type` | The type of instance that generated the log. Can hold more than one value on a single log entry, for example an instance running both Elastic Agent and Fleet Server may log `[agent, fleet-server]` | `elasticsearch` |
| `service.version` | The version of the stack resource that generated the log | `9.0.0` |
| `event.dataset` | The logging category for the instance that generated the log | `elasticsearch.server` |
`service.type` narrows results to a component (for example, all Elasticsearch logs), but it won't distinguish between server logs, audit logs, slow logs, and GC logs from that same component. Use `event.dataset` alongside it to filter down to a specific log category, this matters most on a busy cluster or one with audit logging enabled, where server logs can otherwise bury what you're looking for.
If you need to filter to a specific deployment reliably across all components and log categories, use `service.id` rather than `service.name`, since `service.name` isn't consistently populated for every Elasticsearch log category and takes on component-specific values for APM and Fleet Server rather than the deployment name.
## Logging features
When shipping logs to a monitoring deployment there are more logging features available to you. These features include:
### For Elasticsearch
Standard Elasticsearch logs use `event.dataset: elasticsearch.server`. Enabling the following features adds further log categories:
- [Audit logging](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/security/logging-configuration/enabling-audit-logs) - logs security-related events on your deployment (`event.dataset: elasticsearch.audit`)
- [Slow query and index logging](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/logging-configuration/slow-logs) - helps find and debug slow queries and indexing (`event.dataset: elasticsearch.index_search_slowlog` for search, `elasticsearch.index_indexing_slowlog` for indexing)
- Verbose logging - helps debug stack issues by increasing component logs (still `event.dataset: elasticsearch.server`, just more entries)
- [Deprecation logging](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/logging-configuration/elasticsearch-deprecation-logs) - surfaces use of deprecated features ahead of upgrades, collected by default (`event.dataset: elasticsearch.deprecation`)
- GC logging - helps diagnose JVM garbage collection pauses, collected by default (`event.dataset: elasticsearch.gc`)
After you’ve enabled log delivery on your deployment, you can [add the Elasticsearch user settings](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/deploy/cloud-enterprise/edit-stack-settings) to enable these features.
### For Kibana
Standard Kibana logs use `event.dataset: kibana.log`.
- [Audit logging](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/security/logging-configuration/enabling-audit-logs) - logs security-related events on your deployment (`event.dataset: kibana.audit`)
After you’ve enabled log delivery on your deployment, you can [add the Kibana user settings](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/deploy/cloud-enterprise/edit-stack-settings) to enable this feature.
### Other components
Enabling log collection also supports collecting and indexing the following types of logs from other components in your deployments:
**APM**
- `apm*.log*`
**Fleet and Elastic Agent**
- `fleet-server-json.log-*`
- `elastic-agent-json.log-*`
The `*` indicates that we also index the archived files of each type of log.
APM, Fleet Server, and general Elastic Agent activity all share `event.dataset: agent.log`, since APM runs as an Elastic Agent-managed component rather than a standalone service. To filter to one of these specifically, use `service.name` instead (for example, `apm-server` or `fleet-server`). On these logs, `service.type` and `service.name` can each hold multiple values on a single entry, for example `service.type: [agent, fleet-server]`, reflecting that one instance can run more than one component.
Check the respective product documentation for more information about the logging capabilities of each product.
## Restrictions and limitations
- The monitoring deployment and production deployment must be on the same major version, cloud provider, and region.
- To avoid compatibility issues, ensure your monitoring cluster and production cluster run on the same Elastic Stack version. Monitoring clusters that use 9.x do work with production clusters that use the latest release of 8.x, but this setup should only occur when upgrading clusters to the same version.
- Monitoring across regions is not supported. If you need to move your existing monitoring to the same region, you can do a reindex or create a new deployment and select the snapshot from the old deployment.
- The logs shipped to a monitoring cluster use an ILM managed data stream (`elastic-cloud-logs-`). If you need to delete indices due to space, do not delete the current `is_write_enabled: true` index.
- When sending metrics to a dedicated monitoring deployment, the graph for IO Operations Rate(/s) is blank. This is due to the fact that this graph actually contains metrics from of all of the virtualized resources from the provider.
## Next steps
After your monitoring data is flowing, learn more about what you can do with it:
- [Visualize monitoring data](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/monitoring-data/visualizing-monitoring-data): view Elasticsearch, Logstash, Kibana, and Beats health and performance metrics in the **Stack Monitoring** application in Kibana, and investigate shipped logs alongside those metrics in Discover.
- [Configure stack monitoring alerts](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/8159/deploy-manage/monitor/monitoring-data/configure-stack-monitoring-alerts): set up notifications for cluster health, resource utilization, and error conditions through connectors such as Slack, email, or PagerDuty.