Create or update an ES|QL dataset

PUT /_query/dataset/{name}

Auth

Creates or replaces a dataset that references a data source in ES|QL data federation. Dataset names participate in the index namespace and must follow index or alias naming rules. Returns 404 if the referenced data source does not exist.

ES|QL Data Federation

Required privileges

Index privileges manage

Path parameters

name string required: The dataset name to create or update.

Query parameters

master_timeout string · default: 30s: Period to wait for a connection to the master node.
timeout string · default: 30s: The time to wait for the request to be completed.

Request body

data_source string required: The name of the referenced data source. The data source must already exist.
resource string required: The URI that identifies the data to read, resolved against the referenced data source. It can include glob patterns. For example, a recursive pattern can match all Parquet files under the s3://logs-bucket/access prefix.
description string: A free-text description of the dataset.
mappings object: User-declared mapping on the dataset definition
settings map string to object: Format and parsing-specific settings that configure how the resource is read. Common keys include format, which explicitly selects a registered format, and partition_detection, which accepts auto, hive, template, or none. Additional keys depend on the format reader. Compression can be inferred from the resource URI.

Response

application/json

acknowledged boolean required: For a successful response, this value is always true. On failure, an exception is returned instead.

Examples

Request Console 1 / 8 languages

Create a dataset that reads partitioned Parquet access logs through the prod_s3_logs data source.

Console Kibana Dev Tools

PUT /_query/dataset/access_logs
{
  "data_source": "prod_s3_logs",
  "resource": "s3://logs-bucket/access/**/*.parquet",
  "description": "Production access logs",
  "settings": {
    "partition_detection": "hive"
  }
}

JavaScript Elasticsearch JavaScript Client

const response = await client.esql.putDataset({
  name: "access_logs",
  data_source: "prod_s3_logs",
  resource: "s3://logs-bucket/access/**/*.parquet",
  description: "Production access logs",
  settings: {
    partition_detection: "hive",
  },
});

Python Elasticsearch Python Client

resp = client.esql.put_dataset(
    name="access_logs",
    data_source="prod_s3_logs",
    resource="s3://logs-bucket/access/**/*.parquet",
    description="Production access logs",
    settings={
        "partition_detection": "hive"
    },
)

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_query/dataset/access_logs" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "data_source": "prod_s3_logs",
    "resource": "s3://logs-bucket/access/**/*.parquet",
    "description": "Production access logs",
    "settings": {
      "partition_detection": "hive"
    }
  }'

Java Elasticsearch Java Client

client.esql().putDataset(p -> p
    .dataSource("prod_s3_logs")
    .description("Production access logs")
    .name("access_logs")
    .resource("s3://logs-bucket/access/**/*.parquet")
    .settings("partition_detection", JsonData.fromJson("\"hive\""))
);

C# Elasticsearch .NET Client

var response = await client.Esql
    .PutDatasetAsync(name: "access_logs", d1 => d1
        .DataSource("prod_s3_logs")
        .Description("Production access logs")
        .Resource("s3://logs-bucket/access/**/*.parquet")
        .Settings(new Dictionary<string, object>()
        {
            { "partition_detection", "hive" }
        })
    );

PHP Elasticsearch PHP Client

$resp = $client->esql()->putDataset([
    "name" => "access_logs",
    "body" => [
        "data_source" => "prod_s3_logs",
        "resource" => "s3://logs-bucket/access/**/*.parquet",
        "description" => "Production access logs",
        "settings" => [
            "partition_detection" => "hive",
        ],
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.esql.put_dataset(
  name: "access_logs",
  body: {
    "data_source": "prod_s3_logs",
    "resource": "s3://logs-bucket/access/**/*.parquet",
    "description": "Production access logs",
    "settings": {
      "partition_detection": "hive"
    }
  }
)

Response

The dataset definition was stored successfully.

{
  "acknowledged": true
}