---
title: Configure cluster settings for ES|QL Data Federation
description: Reference for ES|QL Data Federation cluster settings. Tune object limits, control request concurrency, and adjust file discovery and caching.
url: https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-cluster-settings
products:
  - Elasticsearch
applies_to:
  - Elastic Cloud Serverless: Unavailable
  - Elastic Stack: Experimental in 9.5
---

# Configure cluster settings for ES|QL Data Federation
The data sources feature adds the following cluster settings. For general guidance on how to apply cluster settings across deployment types, refer to [configure Elasticsearch](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/deploy-manage/stack-settings).
The object-count limits and authentication gates are operator-managed and take effect without a restart. The remaining settings require a node restart.

## Object limits

These settings cap the number of data sources and datasets a cluster can hold.

| Setting                       | Default | Description                                                       |
|-------------------------------|---------|-------------------------------------------------------------------|
| `esql.data_sources.max_count` | 100     | Maximum number of data sources that can be defined. Range 0–1000. |
| `esql.datasets.max_count`     | 1000    | Maximum number of datasets that can be defined. Range 0–10,000.   |


## Request concurrency and throttling

These settings control how many concurrent requests each node sends to external storage and how long it retries throttled requests.

| Setting                                     | Default                                                | Description                                                                                                                                        |
|---------------------------------------------|--------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| `esql.external.max_concurrent_requests`     | `allocated processors * 3`, minimum 16 and maximum 100 | Maximum concurrent cloud API requests per storage scheme, per node. `0` removes the limit. Range 0–500.                                            |
| `esql.external.throttle_max_retry_duration` | 30                                                     | Maximum total time, in seconds, spent retrying throttled cloud API requests before failing the query. `0` removes the budget. Range 0–300 seconds. |
| `esql.external.max_concurrent_segmentators` | `0`                                                    | Maximum number of file segmentation tasks that run concurrently. `0` derives the value automatically. Range 0–4096.                                |


## Glob and file-discovery limits

These settings bound file discovery for glob patterns. Exceeding the file cap aborts the query; exceeding the brace-expansion cap falls back to listing the storage.

| Setting                              | Default | Description                                                                                                                           |
|--------------------------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------|
| `esql.external.max_discovered_files` | 10,000  | Hard cap on files collected by glob expansion before the query aborts. Protects against degenerate globs. Range 1–1,000,000. Dynamic. |
| `esql.external.max_glob_expansion`   | 100     | Cap on concrete paths generated by brace expansion before falling back to listing. Range 1–10,000. Dynamic.                           |


## Authentication

These settings control which authentication modes data sources can use.

| Setting                                      | Default | Description                                                                                                                                                                                                                                                                                                                                                                                                                             |
|----------------------------------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `esql.datasource.managed_identity.enabled`   | false   | Enables `auth: "managed_identity"` (the node's own cloud identity through the instance metadata service (IMDS)). Operator-only. Intended for single-cloud, single-tenant deployments. Never enable in serverless or multi-tenant clusters. Refer to the managed identity row in [authentication models](/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-sources#authentication) for guidance.      |
| `esql.datasource.federated_identity.enabled` | false   | Enables `auth: "federated_identity"` (OIDC-to-STS token exchange). Operator-only. Available on Elastic Cloud Hosted and Serverless; not available on self-managed, Elastic Cloud Enterprise, or Elastic Cloud on Kubernetes. For setup details, refer to [connect with federated identity](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-federated-identity). |


## Caching

These settings control the external-source cache, which stores inferred schemas and file listings.

| Setting                         | Default      | Description                                                                                                                                                                                                 |
|---------------------------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `esql.source.cache.enabled`     | true         | Enables the external-source cache (inferred schemas and file listings). [Dynamic](https://docs-v3-preview.elastic.dev/elastic/docs-content/tree/main/deploy-manage/stack-settings#dynamic-cluster-setting). |
| `esql.source.cache.size`        | 0.4% of heap | Memory budget for the cache. Applied at node startup only.                                                                                                                                                  |
| `esql.source.cache.schema.ttl`  | 5m           | How long an inferred schema is cached. Applied at node startup only.                                                                                                                                        |
| `esql.source.cache.listing.ttl` | 30s          | How long a file-listing result is cached. Applied at node startup only.                                                                                                                                     |