---
title: Kibana task management
description: Kibana Task Manager is used by features such as Alerting, Actions, and Reporting to run mission critical work as persistent background tasks. These background...
url: https://www.elastic.co/elastic/docs-builder/docs/3873/deploy-manage/distributed-architecture/kibana-tasks-management
products:
  - Kibana
applies_to:
  - Elastic Stack: Generally available
---

# Kibana task management
Kibana Task Manager is used by features such as Alerting, Actions, and Reporting to run mission critical work as persistent background tasks. These background tasks distribute work across multiple Kibana instances. This has three major benefits:
- **Persistence**: All task state and scheduling is stored in Elasticsearch, so if you restart Kibana, tasks will pick up where they left off.
- **Scaling**: Multiple Kibana instances can read from and update the same task queue in Elasticsearch, allowing the work load to be distributed across instances. If a Kibana instance no longer has capacity to run tasks, you can increase capacity by adding additional Kibana instances. For more information on scaling, see [Kibana task manager scaling considerations](/elastic/docs-builder/docs/3873/deploy-manage/production-guidance/kibana-task-manager-scaling-considerations#task-manager-scaling-guidance).
- **Load Balancing**: Task Manager is equipped with a reactive self-healing mechanism, which allows it to reduce the amount of work it executes in reaction to an increased load related error rate in Elasticsearch. Additionally, when Task Manager experiences an increase in recurring tasks, it attempts to space out the work to better balance the load.

<important>
  Task definitions for alerts and actions are stored in the index called `.kibana_task_manager`.You must have at least one replica of this index for production deployments.If you lose this index, all scheduled alerts and actions are lost.
</important>


## How background tasks are managed

Kibana background tasks are managed as follows:
- An Elasticsearch task index is polled for overdue tasks at 500-millisecond intervals. You can change this interval using the [`xpack.task_manager.poll_interval`](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3873/reference/kibana/configuration-reference/task-manager-settings#task-manager-settings) setting.
- Tasks are claimed by updating them in the Elasticsearch index, using optimistic concurrency control to prevent conflicts. Each Kibana instance can run a maximum of 10 concurrent tasks, so a maximum of 10 tasks are claimed each interval.
- Elasticsearch and Kibana instances use the system clock to determine the current time. To ensure schedules are triggered when expected, synchronize the clocks of all nodes in the cluster using a time service such as [Network Time Protocol](http://www.ntp.org/).
- Tasks are run on the Kibana server. In large deployments with high background task and user traffic workloads, it is recommended to run tasks on dedicated Kibana nodes to prevent resource contention. Refer to [Dedicated background task nodes](#task-manager-dedicated-task-nodes).
- Task Manager ensures that tasks:
  - Are only executed once
- Are retried when they fail (if configured to do so)
- Are rescheduled to run again at a future point in time (if configured to do so)
  <important>
  It is possible for tasks to run late or at an inconsistent schedule.
  </important>

This is usually a symptom of the specific usage or scaling strategy of the cluster in question.
To address these issues, tweak the Kibana Task Manager settings or the cluster scaling strategy to better suit the unique use case.
For details on the settings that can influence the performance and throughput of Task Manager, see [Task Manager Settings](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3873/reference/kibana/configuration-reference/task-manager-settings).
For detailed troubleshooting guidance, see [Troubleshooting](https://www.elastic.co/elastic/docs-builder/docs/3873/troubleshoot/kibana/task-manager).

## Dedicated background task nodes

Because tasks run on the Kibana server, heavy background workloads can compete with user traffic for resources. To avoid this, it is recommended to run background tasks on dedicated Kibana nodes:
- On self-managed and Elastic Cloud on Kubernetes deployments, set `node.roles: ["background_tasks"]` in [`kibana.yml`](https://www.elastic.co/elastic/docs-builder/docs/3873/deploy-manage/stack-settings) to dedicate a Kibana instance to background tasks.
- On Elastic Cloud Hosted and Elastic Cloud Enterprise deployments, when Kibana is allocated more than 8 GB of RAM, it is automatically split into multiple instances with specialized roles: UI nodes, which serve user requests, and background task nodes, which run Task Manager tasks.

To identify a node's type on Elastic Cloud Hosted and Elastic Cloud Enterprise deployments, check the node's instance name in the `kibana_status.json` file included in the [Kibana diagnostic bundle](https://www.elastic.co/elastic/docs-builder/docs/3873/troubleshoot/kibana/capturing-diagnostics). A UI node's instance name contains `- UI`.
On Elastic Cloud Hosted and Elastic Cloud Enterprise deployments, API requests, including health and monitoring APIs such as `GET api/task_manager/_health`, are always served by UI nodes, so their responses don't reflect task execution state. On self-managed and Elastic Cloud on Kubernetes deployments, the same applies to any request served by a node without the `background_tasks` role. To retrieve complete Task Manager health data, refer to [Retrieve health data when Kibana uses separate task nodes](/elastic/docs-builder/docs/3873/troubleshoot/kibana/task-manager#task-manager-health-multi-node).