---
title: Create and manage datasets with the Elasticsearch API
description: Create, get, list, and delete ES|QL Data Federation datasets with the /_query/dataset API.
url: https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-manage-datasets-api
products:
  - Elasticsearch
applies_to:
  - Elastic Cloud Serverless: Unavailable
  - Elastic Stack: Experimental since 9.5
---

# Create and manage datasets with the Elasticsearch API
<warning>
  This feature is experimental. It is not intended for production use and there are no guarantees around performance, scale, or stability in this release.
</warning>

Datasets are managed under the `/_query/dataset` endpoint. All dataset operations require the index `manage` privilege on the dataset name, or a fine-grained dataset privilege. Refer to [manage credentials and privileges](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-security) for details.

| Operation                                       | Endpoint                        | API reference                                                                                                               |
|-------------------------------------------------|---------------------------------|-----------------------------------------------------------------------------------------------------------------------------|
| [Create or update](#create-or-update-a-dataset) | `PUT /_query/dataset/{name}`    | [Create or update an ES|QL dataset](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-put-dataset) |
| [Get](#get-a-dataset)                           | `GET /_query/dataset/{name}`    | [Get ES|QL datasets](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-get-dataset)                |
| [List all](#list-all-datasets)                  | `GET /_query/dataset`           | [Get ES|QL datasets](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-get-dataset)                |
| [Delete](#delete-a-dataset)                     | `DELETE /_query/dataset/{name}` | [Delete ES|QL datasets](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-delete-dataset)          |


## Create or update a dataset

[`PUT /_query/dataset/{name}`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-put-dataset) creates a new dataset or replaces an existing one entirely.
A dataset cannot have the same name as an existing index, data stream, alias, or view, because dataset names share the same namespace. Dataset names must be lowercase and cannot begin with `-`, `_`, or `+`.
The optional `description` can be at most 1,000 characters long.
<applies-to>Elastic Stack: Planned</applies-to> For restrictions on S3 bucket names, aliases, and Amazon Resource Names (ARNs), refer to [Amazon S3 resource restrictions](/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-patterns#s3-resource-requirements).
<tab-set>
  <tab-item title="Console">
    ```json

    {
      "data_source": "prod_s3_logs",
      "resource": "s3://logs-bucket/access/**/*.parquet",
      "description": "Production access logs",
      "settings": {
        "partition_detection": "hive"
      }
    }
    ```
  </tab-item>

  <tab-item title="curl">
    ```bash
    curl -X PUT "${ELASTICSEARCH_URL}/_query/dataset/access_logs" \
      -H "Authorization: ApiKey ${API_KEY}" \
      -H "Content-Type: application/json" \
      -d '{
      "data_source": "prod_s3_logs",
      "resource": "s3://logs-bucket/access/**/*.parquet",
      "description": "Production access logs",
      "settings": {
        "partition_detection": "hive"
      }
    }'
    ```
  </tab-item>
</tab-set>


### Setting validation

<applies-to>
  - Elastic Stack: Planned
</applies-to>

Elasticsearch validates setting values when you register the dataset, rather than waiting until the first query. A value that isn't supported, such as a multi-character `delimiter`, an unknown `encoding`, or a `segment_size` below the minimum, returns a `400` error that identifies the setting.
<note>
  Datasets registered before this validation was introduced continue to work without being revalidated. Replacing one of these datasets triggers validation, so correct any unsupported values in the replacement request.
</note>


### Verify the inferred schema

After creating a dataset, verify the inferred schema by [checking its field mappings](/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-quickstart#check-field-mappings).

## Get a dataset

[`GET /_query/dataset/{name}`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-get-dataset) retrieves a dataset by name.
<tab-set>
  <tab-item title="Console">
    ```json
    ```
  </tab-item>

  <tab-item title="curl">
    ```bash
    curl -X GET "${ELASTICSEARCH_URL}/_query/dataset/access_logs" \
      -H "Authorization: ApiKey ${API_KEY}"
    ```
  </tab-item>
</tab-set>


## List all datasets

[`GET /_query/dataset`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-get-dataset) returns all registered datasets.
<tab-set>
  <tab-item title="Console">
    ```json
    ```
  </tab-item>

  <tab-item title="curl">
    ```bash
    curl -X GET "${ELASTICSEARCH_URL}/_query/dataset" \
      -H "Authorization: ApiKey ${API_KEY}"
    ```
  </tab-item>
</tab-set>


## Delete a dataset

[`DELETE /_query/dataset/{name}`](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-esql-delete-dataset) deletes a dataset by name.
<tab-set>
  <tab-item title="Console">
    ```json
    ```
  </tab-item>

  <tab-item title="curl">
    ```bash
    curl -X DELETE "${ELASTICSEARCH_URL}/_query/dataset/access_logs" \
      -H "Authorization: ApiKey ${API_KEY}"
    ```
  </tab-item>
</tab-set>


## Next steps

After creating a dataset, [query it with ES|QL](https://docs-v3-preview.elastic.dev/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-querying). To change how the dataset selects or interprets files, return to [define a dataset](/elastic/elasticsearch/tree/main/reference/query-languages/esql/esql-data-federation-datasets#define-a-dataset).