Create or update an ES|QL dataset

PUT /_query/dataset/{name}

Creates or replaces a dataset that references a data source in ES|QL data federation. Dataset names participate in the index namespace and must follow index or alias naming rules. Returns 404 if the referenced data source does not exist.

Required authorization

Index privileges: manage

ES|QL Data Federation

Path Parameters

name string Name required: The dataset name to create or update.

Query Parameters

Request

data_source string Name required: The name of the referenced data source. The data source must already exist.
resource string required: The URI that identifies the data to read, resolved against the referenced data source. It can include glob patterns. For example, a recursive pattern can match all Parquet files under the s3://logs-bucket/access prefix.
description string: A free-text description of the dataset.
mappings {} DatasetMapping: User-declared mapping on the dataset definition
settings map string to object: Format and parsing-specific settings that configure how the resource is read. Common keys include format, which explicitly selects a registered format, and partition_detection, which accepts auto, hive, template, or none. Additional keys depend on the format reader. Compression can be inferred from the resource URI.

Response

application/json

Examples

Create a dataset that reads partitioned Parquet access logs through the prod_s3_logs data source.

Request Console 1 / 7 languages

Console Kibana Dev Tools

PUT /_query/dataset/access_logs
{
  "data_source": "prod_s3_logs",
  "resource": "s3://logs-bucket/access/**/*.parquet",
  "description": "Production access logs",
  "settings": {
    "partition_detection": "hive"
  }
}

JavaScript Elasticsearch JavaScript Client

const response = await client.esql.putDataset({
  name: "access_logs",
  data_source: "prod_s3_logs",
  resource: "s3://logs-bucket/access/**/*.parquet",
  description: "Production access logs",
  settings: {
    partition_detection: "hive",
  },
});

Python Elasticsearch Python Client

resp = client.esql.put_dataset(
    name="access_logs",
    data_source="prod_s3_logs",
    resource="s3://logs-bucket/access/**/*.parquet",
    description="Production access logs",
    settings={
        "partition_detection": "hive"
    },
)

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_query/dataset/access_logs" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "data_source": "prod_s3_logs",
    "resource": "s3://logs-bucket/access/**/*.parquet",
    "description": "Production access logs",
    "settings": {
      "partition_detection": "hive"
    }
  }'

Java Elasticsearch Java Client

client.esql().putDataset(p -> p
    .dataSource("prod_s3_logs")
    .description("Production access logs")
    .name("access_logs")
    .resource("s3://logs-bucket/access/**/*.parquet")
    .settings("partition_detection", JsonData.fromJson("\"hive\""))
);

PHP Elasticsearch PHP Client

$resp = $client->esql()->putDataset([
    "name" => "access_logs",
    "body" => [
        "data_source" => "prod_s3_logs",
        "resource" => "s3://logs-bucket/access/**/*.parquet",
        "description" => "Production access logs",
        "settings" => [
            "partition_detection" => "hive",
        ],
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.esql.put_dataset(
  name: "access_logs",
  body: {
    "data_source": "prod_s3_logs",
    "resource": "s3://logs-bucket/access/**/*.parquet",
    "description": "Production access logs",
    "settings": {
      "partition_detection": "hive"
    }
  }
)

Response

{
  "acknowledged": true
}

Required authorization

Path Parameters

Query Parameters { master_timeout , timeout , }

Request { data_source , resource , description , mappings , settings , }

Response

Query Parameters

Request