---
title: Configure cluster settings for ES|QL Data Federation
description: Reference for ES|QL Data Federation cluster settings. Tune object limits, control request concurrency, and adjust file discovery and caching.
url: https://www.elastic.co/elastic/docs-builder/docs/3895/reference/query-languages/esql/esql-data-federation-cluster-settings
products:
- Elasticsearch
applies_to:
- Elastic Cloud Serverless: Unavailable
- Elastic Stack: Experimental since 9.5
---
# Configure cluster settings for ES|QL Data Federation
The data sources feature adds the following cluster settings. For general guidance on how to apply cluster settings across deployment types, refer to [configure Elasticsearch](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3895/deploy-manage/stack-settings).
The object-count limits and authentication gates are operator-managed and take effect without a restart. Settings marked Dynamic can also be updated at runtime. The remaining settings require a node restart.
Some of these settings were renamed. Where a row lists two names, the badge on each shows which versions accept it. Unless a note says otherwise, the older name is no longer registered, and a node that still sets it in `elasticsearch.yml` fails to start.
## Object limits
These settings cap the number of data sources and datasets a cluster can hold.
| Setting | Default | Description |
|-------------------------------|---------|-------------------------------------------------------------------|
| `esql.data_sources.max_count` | 100 | Maximum number of data sources that can be defined. Range 0–1000. |
| `esql.datasets.max_count` | 1000 | Maximum number of datasets that can be defined. Range 0–10,000. |
## Request concurrency and throttling
These settings control how many concurrent requests each node sends to external storage and how long it retries throttled requests.
| Setting | Default | Description |
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| `esql.external.max_concurrent_requests` | `allocated processors * 3`, minimum 16 and maximum 100 | Maximum concurrent cloud API requests per storage scheme, per node. `0` removes the limit. Range 0–500. |
| `esql.external.throttle_max_retry_duration` | 30 | Maximum total time, in seconds, spent retrying throttled cloud API requests before failing the query. `0` removes the budget. Range 0–300 seconds. |
| `esql.external.max_concurrent_segmenters` Elastic Stack: Planned`esql.external.max_concurrent_segmentators` Elastic Stack: Experimental in 9.5 | `0` | Maximum number of file segmentation tasks that run concurrently. `0` derives the value automatically. Range 0–4096. |
## Glob and file-discovery limits
These settings bound file discovery for glob patterns. Exceeding the file cap aborts the query; exceeding the brace-expansion cap falls back to listing the storage.
| Setting | Default | Description |
|--------------------------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------|
| `esql.external.max_discovered_files` | 10,000 | Hard cap on files collected by glob expansion before the query aborts. Protects against degenerate globs. Range 1–1,000,000. Dynamic. |
| `esql.external.max_glob_expansion` | 100 | Cap on concrete paths generated by brace expansion before falling back to listing. Range 1–10,000. Dynamic. |
## Authentication
These settings control which authentication modes data sources can use.
| Setting | Default | Description |
|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `esql.external.managed_identity.enabled` Elastic Stack: Planned`esql.datasource.managed_identity.enabled` Elastic Stack: Planned for deprecation, Elastic Stack: Experimental in 9.5 | false | Enables `auth: "managed_identity"` (the node's own cloud identity through the instance metadata service (IMDS)). Operator-only. Intended for single-cloud, single-tenant deployments. Never enable in serverless or multi-tenant clusters. Refer to the managed identity row in [authentication models](/elastic/docs-builder/docs/3895/reference/query-languages/esql/esql-data-federation-sources#authentication) for guidance. |
| `esql.external.federated_identity.enabled` Elastic Stack: Planned`esql.datasource.federated_identity.enabled` Elastic Stack: Planned for deprecation, Elastic Stack: Experimental in 9.5 | false | Enables `auth: "federated_identity"` (OIDC-to-STS token exchange). Operator-only. Available on Elastic Cloud Hosted and Serverless; not available on self-managed, Elastic Cloud Enterprise, or Elastic Cloud on Kubernetes. For setup details, refer to [connect with federated identity](https://www.elastic.co/elastic/docs-builder/docs/3895/reference/query-languages/esql/esql-data-federation-federated-identity). |
## Caching
These settings control the external-source cache, which stores inferred schemas and file listings.
| Setting | Default | Description |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `esql.external.cache.enabled` Elastic Stack: Planned`esql.source.cache.enabled` Elastic Stack: Planned for deprecation, Elastic Stack: Experimental in 9.5 | true | Enables the external-source cache (inferred schemas and file listings). [Dynamic](https://docs-v3-preview.elastic.dev/elastic/docs-builder/docs/3895/deploy-manage/stack-settings#dynamic-cluster-setting). |
| `esql.external.cache.size` Elastic Stack: Planned`esql.source.cache.size` Elastic Stack: Planned for deprecation, Elastic Stack: Experimental in 9.5 | 0.4% of heap | Memory budget for the cache. Applied at node startup only. |
| `esql.source.cache.schema.ttl` | — | Deprecated and ignored. Inferred schemas are invalidated by file identity and bounded by the cache memory budget, not by a TTL. |
| `esql.external.cache.listing.ttl` Elastic Stack: Planned`esql.source.cache.listing.ttl` Elastic Stack: Planned for deprecation, Elastic Stack: Experimental in 9.5 | 30s | How long a file-listing result is cached. Applied at node startup only. |
The `esql.source.cache.*` keys are accepted as deprecated fallbacks and emit a deprecation warning. You can set them in `elasticsearch.yml`, but you can't update them through the cluster settings API. Use the `esql.external.cache.*` keys for new configuration.