--- type: api title: Preview a transform description: **All methods and paths for this operation:** url: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/operation/operation-transform-preview-transform resource: https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/operation/operation-transform-preview-transform products: - Elasticsearch --- # Preview a transform Availability: 7.2+ ## Paths - `POST` (also `GET`) `/_transform/{transform_id}/_preview` - `POST` (also `GET`) `/_transform/_preview` - [Api key auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#apikeyauth) - [Basic auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#basicauth) - [Bearer auth](/elastic/docs-builder/docs/4371/api/doc/elasticsearch/authentication#bearerauth) #### Path Parameters - `transform_id` (string Id) — optional, leave it out to use `/_transform/_preview` Identifier for the transform to preview. If you specify this path parameter, you cannot provide transform configuration details in the request body. ## Description Generates a preview of the results that you will get when you create a transform with the same configuration. It returns a maximum of 100 results. The calculations are based on all the current data in the source index. It also generates a list of mappings and settings for the destination index. These values are determined based on the field types of the source index and the transform aggregations. ## Required authorization * Index privileges: `read`,`view_index_metadata` * Cluster privileges: `manage_transform` ## Query String Parameters - `timeout` (string Duration · default: 30s) Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. ## Request Body `application/json` - `dest` ({} Destination) — optional The destination for the transform. - `index` (string IndexName) — optional The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the create index API prior to starting the transform. - `aliases` ([] DestinationAlias) — optional The aliases that the destination index for the transform should have. Aliases are manipulated using the stored credentials of the transform, which means the secondary credentials supplied at creation time (if both primary and secondary credentials are specified). The destination index is added to the aliases regardless of whether the destination index was created by the transform or pre-created by the user. - `alias` (string) — required The name of the alias. - `move_on_creation` (boolean · default: false) — optional Whether the destination index should be the only index in this alias. If `true`, all the other indices will be removed from this alias before adding the destination index to this alias. This does not delete the removed indices; it only removes them from the alias. - `pipeline` (string) — optional The unique identifier for an ingest pipeline. - `description` (string) — optional Free text description of the transform. - `frequency` (string Duration · default: 1m) — optional The interval between checks for changes in the source indices when the transform is running continuously. Also determines the retry interval in the event of transient failures while the transform is searching or indexing. The minimum value is 1s and the maximum is 1h. - `pivot` ({} Pivot) — optional The pivot method transforms the data by aggregating and grouping it. These objects define the group by fields and the aggregation to reduce the data. - `aggregations` (map string to {} AggregationContainer) — optional Defines how to aggregate the grouped data. The following aggregations are currently supported: average, bucket script, bucket selector, cardinality, filter, geo bounds, geo centroid, geo line, max, median absolute deviation, min, missing, percentiles, rare terms, scripted metric, stats, sum, terms, top metrics, value count, weighted average. See [AggregationContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/types/_types-aggregations-aggregationcontainer) - `group_by` (map string to PivotGroupByContainer) — optional Defines how to group the data. More than one grouping can be defined per pivot. The following groupings are currently supported: date histogram, geotile grid, histogram, terms. Map values ({} PivotGroupByContainer) - `date_histogram` ({} DateHistogramAggregation) — optional - `calendar_interval` (enum CalendarInterval) — optional Calendar-aware interval. Can be specified using the unit name, such as `month`, or as a single unit quantity, such as `1M`. Values: `second`, `1s`, `minute`, `1m`, `hour`, `1h`, `day`, `1d`, `week`, `1w`, `month`, `1M`, `quarter`, `1q`, `year`, `1y` - `extended_bounds` ({} ExtendedBoundsFieldDateMath) — optional Enables extending the bounds of the histogram beyond the data itself. - `max` (FieldDateMath) — optional Maximum value for the bound. - `min` (FieldDateMath) — optional Minimum value for the bound. - `hard_bounds` ({} ExtendedBoundsFieldDateMath) — optional Limits the histogram to specified bounds. - `max` (FieldDateMath) — optional Maximum value for the bound. - `min` (FieldDateMath) — optional Minimum value for the bound. - `field` (string Field) — optional The date field whose values are use to build a histogram. - `fixed_interval` (string Duration) — optional Fixed intervals: a fixed number of SI units and never deviate, regardless of where they fall on the calendar. - `format` (string) — optional The date format used to format `key_as_string` in the response. If no `format` is specified, the first date format specified in the field mapping is used. - `interval` (string Duration) — optional — deprecated - `min_doc_count` (number) — optional Only returns buckets that have `min_doc_count` number of documents. By default, all buckets between the first bucket that matches documents and the last one are returned. - `missing` (DateTime) — optional The value to apply to documents that do not have a value. By default, documents without a value are ignored. - `offset` (string Duration) — optional Changes the start value of each bucket by the specified positive (`+`) or negative offset (`-`) duration. - `order` (AggregateOrder) — optional The sort order of the returned buckets. - `params` (map string to object) — optional - `script` (string Script) — optional - `time_zone` (string) — optional Time zone used for bucketing and rounding. Defaults to Coordinated Universal Time (UTC). - `keyed` (boolean) — optional Set to `true` to associate a unique string key with each bucket and return the ranges as a hash rather than an array. - `geotile_grid` ({} GeoTileGridAggregation) — optional - `field` (string Field) — optional Field containing indexed `geo_point` or `geo_shape` values. If the field contains an array, `geotile_grid` aggregates all array values. - `precision` (number · default: 7) — optional Integer zoom of the key used to define cells/buckets in the results. Values outside of the range [0,29] will be rejected. - `shard_size` (number) — optional Allows for more accurate counting of the top cells returned in the final result the aggregation. Defaults to returning `max(10,(size x number-of-shards))` buckets from each shard. - `size` (number · default: 10000) — optional The maximum number of buckets to return. - `bounds` (GeoBounds) — optional A bounding box to filter the geo-points or geo-shapes in each bucket. - `histogram` ({} HistogramAggregation) — optional - `extended_bounds` ({} ExtendedBoundsdouble) — optional Enables extending the bounds of the histogram beyond the data itself. - `max` (number) — optional Maximum value for the bound. - `min` (number) — optional Minimum value for the bound. - `hard_bounds` ({} ExtendedBoundsdouble) — optional Limits the range of buckets in the histogram. It is particularly useful in the case of open data ranges that can result in a very large number of buckets. - `max` (number) — optional Maximum value for the bound. - `min` (number) — optional Minimum value for the bound. - `field` (string Field) — optional The name of the field to aggregate on. - `interval` (number) — optional The interval for the buckets. Must be a positive decimal. - `min_doc_count` (number) — optional Only returns buckets that have `min_doc_count` number of documents. By default, the response will fill gaps in the histogram with empty buckets. - `missing` (number) — optional The value to apply to documents that do not have a value. By default, documents without a value are ignored. - `offset` (number) — optional By default, the bucket keys start with 0 and then continue in even spaced steps of `interval`. The bucket boundaries can be shifted by using the `offset` option. - `order` (AggregateOrder) — optional The sort order of the returned buckets. By default, the returned buckets are sorted by their key ascending. - `script` (string Script) — optional - `format` (string) — optional - `keyed` (boolean · default: false) — optional If `true`, returns buckets as a hash instead of an array, keyed by the bucket keys. - `terms` ({} TermsAggregation) — optional - `collect_mode` (enum TermsAggregationCollectMode) — optional Determines how child aggregations should be calculated: breadth-first or depth-first. Supported values include: - `depth_first`: Expands all branches of the aggregation tree in one depth-first pass, before any pruning occurs. - `breadth_first`: Caches the set of documents that fall into the uppermost buckets for subsequent replay. Values: `depth_first`, `breadth_first` - `exclude` (TermsExclude) — optional Values to exclude. Accepts regular expressions and partitions. - `execution_hint` (enum TermsAggregationExecutionHint) — optional Determines whether the aggregation will use field values directly or global ordinals. Values: `map`, `global_ordinals`, `global_ordinals_hash`, `global_ordinals_low_cardinality` - `field` (string Field) — optional The field from which to return terms. - `include` (TermsInclude) — optional Values to include. Accepts regular expressions and partitions. - `min_doc_count` (number · default: 1) — optional Only return values that are found in more than `min_doc_count` hits. - `missing` (boolean | number | string) — optional The value to apply to documents that do not have a value. By default, documents without a value are ignored. - `missing_order` (enum MissingOrder) — optional Values: `first`, `last`, `default` - `missing_bucket` (boolean) — optional - `value_type` (string) — optional Coerced unmapped fields into the specified type. - `order` (AggregateOrder) — optional Specifies the sort order of the buckets. Defaults to sorting by descending document count. - `script` (string Script) — optional - `shard_min_doc_count` (number) — optional Regulates the certainty a shard has if the term should actually be added to the candidate list or not with respect to the `min_doc_count`. Terms will only be considered if their local shard frequency within the set is higher than the `shard_min_doc_count`. - `shard_size` (number) — optional The number of candidate terms produced by each shard. By default, `shard_size` will be automatically estimated based on the number of shards and the `size` parameter. - `show_term_doc_count_error` (boolean) — optional Set to `true` to return the `doc_count_error_upper_bound`, which is an upper bound to the error on the `doc_count` returned by each shard. - `size` (number · default: 10) — optional The number of buckets returned out of the overall terms list. - `format` (string) — optional - `source` ({} Source) — optional The source of the data for the transform. - `index` (string Indices) — required The source indices for the transform. It can be a single index, an index pattern (for example, `"my-index-*""`), an array of indices (for example, `["my-index-000001", "my-index-000002"]`), or an array of index patterns (for example, `["my-index-*", "my-other-index-*"]`. For remote indices use the syntax `"remote_name:index_name"`. If any indices are in remote clusters then the master node and at least one transform node must have the `remote_cluster_client` node role. - `runtime_mappings` (RuntimeFields) — optional Definitions of search-time runtime fields that can be used by the transform. For search runtime fields all data nodes, including remote nodes, must be 7.12 or later. - `query` (object) — optional A query clause that retrieves a subset of data from the source index. - `settings` ({} Settings) — optional Defines optional transform settings. - `align_checkpoints` (boolean · default: true) — optional Specifies whether the transform checkpoint ranges should be optimized for performance. Such optimization can align checkpoint ranges with the date histogram interval when date histogram is specified as a group source in the transform config. As a result, less document updates in the destination index will be performed thus improving overall performance. - `dates_as_epoch_millis` (boolean · default: false) — optional Defines if dates in the ouput should be written as ISO formatted string or as millis since epoch. epoch_millis was the default for transforms created before version 7.11. For compatible output set this value to `true`. - `deduce_mappings` (boolean · default: true) — optional Specifies whether the transform should deduce the destination index mappings from the transform configuration. - `docs_per_second` (number) — optional Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling. - `max_page_search_size` (number) — optional Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is `10` and the maximum is `65,536`. The default value is `500` for `pivot` transforms and `5000` for `latest` transforms. - `use_point_in_time` (boolean · default: true) — optional Specifies whether the transform checkpoint will use the Point In Time API while searching over the source index. In general, Point In Time is an optimization that will reduce pressure on the source index by reducing the amount of refreshes and merges, but it can be expensive if a large number of Point In Times are opened and closed for a given index. The benefits and impact depend on the data being searched, the ingest rate into the source index, and the amount of other consumers searching the same source index. - `num_failure_retries` (number) — optional Defines the number of retries on a recoverable failure before the transform task is marked as `failed`. The minimum value is `0` and the maximum is `100`, where `-1` indicates that the transform retries indefinitely. If unset, the cluster-level setting `num_transform_failure_retries` is used. This setting cannot be specified when `unattended` is `true`, because unattended transforms always retry indefinitely. - `unattended` (boolean · default: false) — optional If `true`, the transform runs in unattended mode. In unattended mode, the transform retries indefinitely in case of an error which means the transform never fails. Setting the number of retries other than infinite fails in validation. - `sync` ({} SyncContainer) — optional Defines the properties transforms require to run continuously. - `time` ({} TimeSync) — optional Specifies that the transform uses a time field to synchronize the source and destination indices. - `delay` (string Duration · default: 60s) — optional The time delay between the current time and the latest input data time. - `field` (string Field) — required The date field that is used to identify new documents in the source. In general, it’s a good idea to use a field that contains the ingest timestamp. If you use a different field, you might need to set the delay such that it accounts for data transmission delays. - `retention_policy` ({} RetentionPolicyContainer) — optional Defines a retention policy for the transform. Data that meets the defined criteria is deleted from the destination index. - `time` ({} RetentionPolicy) — optional Specifies that the transform uses a time field to set the retention policy. - `field` (string Field) — required The date field that is used to calculate the age of the document. - `max_age` (string Duration) — required Specifies the maximum age of a document in the destination index. Documents that are older than the configured value are removed from the destination index. - `latest` ({} Latest) — optional The latest method transforms the data by finding the latest document for each unique key. - `sort` (string Field) — required Specifies the date field that is used to identify the latest documents. - `unique_key` ([] string Field) — required Specifies an array of one or more fields that are used to group the data. ## Response `application/json` - `generated_dest_index` ({} IndexState) - `aliases` (map string to Alias) Map values ({} Alias) - `filter` ({} QueryContainer) Query used to limit documents the alias can access. See [QueryContainer](https://www.elastic.co/elastic/docs-builder/docs/4371/api/doc/elasticsearch/types/_types-query_dsl-querycontainer) - `index_routing` (string) Value used to route indexing operations to a specific shard. If specified, this overwrites the `routing` value for indexing operations. - `is_hidden` (boolean · default: false) If `true`, the alias is hidden. All indices for the alias must have the same `is_hidden` value. - `is_write_index` (boolean · default: false) If `true`, the index is the write index for the alias. - `routing` (string) Value used to route indexing and search operations to a specific shard. - `search_routing` (string) Value used to route search operations to a specific shard. If specified, this overwrites the `routing` value for search operations. - `mappings` ({} TypeMapping) - `all_field` ({} AllField) - `analyzer` (string) - `enabled` (boolean) - `omit_norms` (boolean) - `search_analyzer` (string) - `similarity` (string) - `store` (boolean) - `store_term_vector_offsets` (boolean) - `store_term_vector_payloads` (boolean) - `store_term_vector_positions` (boolean) - `store_term_vectors` (boolean) - `date_detection` (boolean) - `dynamic` (enum DynamicMapping) Values: `strict`, `runtime`, `true`, `false` - `dynamic_date_formats` ([] string) - `dynamic_templates` ([] string to DynamicTemplate) - `_field_names` ({} FieldNamesField) - `enabled` (boolean) - `index_field` ({} IndexField) - `enabled` (boolean) - `_meta` (Metadata) - `numeric_detection` (boolean) - `properties` (map string to Property) - `_routing` ({} RoutingField) - `required` (boolean) - `_size` ({} SizeField) - `enabled` (boolean) - `_source` ({} SourceField) - `enabled` (boolean) - `excludes` ([] string) - `includes` ([] string) - `mode` (enum SourceFieldMode) Supported values include: - `disabled` - `stored` - `columnar_stored` - `synthetic`: Instead of storing source documents on disk exactly as you send them, Elasticsearch can reconstruct source content on the fly upon retrieval. Values: `disabled`, `stored`, `columnar_stored`, `synthetic` - `runtime` (map string to RuntimeField) Map values ({} RuntimeField) - `fields` (map string to CompositeSubField) For type `composite` Map values ({} CompositeSubField) - `type` (enum RuntimeFieldType) Values: `boolean`, `composite`, `date`, `double`, `geo_point`, `geo_shape`, `ip`, `keyword`, `long`, `lookup` - `fetch_fields` ([] RuntimeFieldFetchFields) For type `lookup` - `field` (string Field) - `format` (string) - `format` (string) A custom format for `date` type runtime fields. - `input_field` (string Field) For type `lookup` - `target_field` (string Field) For type `lookup` - `target_index` (string IndexName) For type `lookup` - `script` (string Script) Painless script executed at query time. - `on_script_error` (enum OnScriptError) Values: `fail`, `continue` - `type` (enum RuntimeFieldType) Field type, which can be: `boolean`, `composite`, `date`, `double`, `geo_point`, `ip`,`keyword`, `long`, or `lookup`. Values: `boolean`, `composite`, `date`, `double`, `geo_point`, `geo_shape`, `ip`, `keyword`, `long`, `lookup` - `enabled` (boolean) - `subobjects` (enum Subobjects) Values: `true`, `false` - `_data_stream_timestamp` ({} DataStreamTimestamp) - `enabled` (boolean) - `settings` (IndexSettings) - `defaults` (IndexSettings) Default settings, included when the request's `include_default` is `true`. - `data_stream` (string) - `lifecycle` ({} DataStreamLifecycle) Data stream lifecycle applicable if this is a data stream. - `data_retention` (string Duration) If defined, every document added to this data stream will be stored at least for this time frame. Any time after this duration the document could be deleted. When empty, every document in this data stream will be stored indefinitely. - `effective_retention` (string Duration) The least amount of time data should be kept by elasticsearch. - `retention_determined_by` (enum RetentionSource) Configuration source that can influence the retention of a data stream. Values: `data_stream_configuration`, `default_global_retention`, `max_global_retention`, `default_failures_retention` - `downsampling` ([] DownsamplingRound) The list of downsampling rounds to execute as part of this downsampling configuration - `after` (string Duration) The duration since rollover when this downsampling round should execute - `fixed_interval` (string) The downsample interval. - `downsampling_method` (enum SamplingMethod) The method used to downsample the data. There are two options `aggregate` and `last_value`. It requires `downsampling` to be defined. Defaults to `aggregate`. Values: `aggregate`, `last_value` - `enabled` (boolean · default: true) If defined, it turns data stream lifecycle on/off (`true`/`false`) for this data stream. A data stream lifecycle that's disabled (enabled: `false`) will have no effect on the data stream. - `frozen_after` (string Duration) The period after which data stream backing indices are automatically converted to partially mounted searchable snapshots. This field is valid only on main data stream lifecycles and cannot be set on failure-store lifecycles. - `preview` ([] object) ## Code Examples #### Console ```console POST _transform/_preview { "source": { "index": "kibana_sample_data_ecommerce" }, "pivot": { "group_by": { "customer_id": { "terms": { "field": "customer_id", "missing_bucket": true } } }, "aggregations": { "max_price": { "max": { "field": "taxful_total_price" } } } } } ``` #### JavaScript ```javascript const response = await client.transform.previewTransform({ source: { index: "kibana_sample_data_ecommerce", }, pivot: { group_by: { customer_id: { terms: { field: "customer_id", missing_bucket: true, }, }, }, aggregations: { max_price: { max: { field: "taxful_total_price", }, }, }, }, }); ``` #### Python ```python resp = client.transform.preview_transform( source={ "index": "kibana_sample_data_ecommerce" }, pivot={ "group_by": { "customer_id": { "terms": { "field": "customer_id", "missing_bucket": True } } }, "aggregations": { "max_price": { "max": { "field": "taxful_total_price" } } } }, ) ``` #### curl ```curl curl -X POST "$ELASTICSEARCH_URL/_transform/_preview" \ -H "Authorization: ApiKey $ELASTIC_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "source": { "index": "kibana_sample_data_ecommerce" }, "pivot": { "group_by": { "customer_id": { "terms": { "field": "customer_id", "missing_bucket": true } } }, "aggregations": { "max_price": { "max": { "field": "taxful_total_price" } } } } }' ``` #### Java ```java client.transform().previewTransform(p -> p .pivot(pi -> pi .aggregations("max_price", a -> a .max(m -> m .field("taxful_total_price") ) ) .groupBy("customer_id", g -> g .terms(t -> t .field("customer_id") .missingBucket(true) ) ) ) .source(s -> s .index("kibana_sample_data_ecommerce") ) ); ``` #### C# ```c# var response = await client.TransformManagement .PreviewTransformAsync(d1 => d1 .Pivot(d2 => d2 .Aggregations(d3 => d3 .Add("max_price", d4 => d4 .Max(d5 => d5 .Field(x => x.TaxfulTotalPrice) ) ) ) .GroupBy(d3 => d3 .Add("customer_id", d4 => d4 .Terms(d5 => d5 .Field(x => x.CustomerId) .MissingBucket(true) ) ) ) ) .Source(d2 => d2 .Indices(new[] { "kibana_sample_data_ecommerce" }) ) ); ``` #### PHP ```php $resp = $client->transform()->previewTransform([ "body" => [ "source" => [ "index" => "kibana_sample_data_ecommerce", ], "pivot" => [ "group_by" => [ "customer_id" => [ "terms" => [ "field" => "customer_id", "missing_bucket" => true, ], ], ], "aggregations" => [ "max_price" => [ "max" => [ "field" => "taxful_total_price", ], ], ], ], ], ]); ``` #### Ruby ```ruby response = client.transform.preview_transform( body: { "source": { "index": "kibana_sample_data_ecommerce" }, "pivot": { "group_by": { "customer_id": { "terms": { "field": "customer_id", "missing_bucket": true } } }, "aggregations": { "max_price": { "max": { "field": "taxful_total_price" } } } } } ) ``` ## Response Examples #### Preview transform response example1 An abbreviated response from `POST _transform/_preview` that contains a preview a transform that uses the pivot method. ```json { "preview": [ { "max_price": 171, "customer_id": "10" }, { "max_price": 233, "customer_id": "11" }, { "max_price": 200, "customer_id": "12" }, { "max_price": 301, "customer_id": "13" }, { "max_price": 176, "customer_id": "14" }, { "max_price": 2250, "customer_id": "15" }, { "max_price": 170, "customer_id": "16" }, { "max_price": 243, "customer_id": "17" }, { "max_price": 154, "customer_id": "18" }, { "max_price": 393, "customer_id": "19" }, { "max_price": 165, "customer_id": "20" }, { "max_price": 115, "customer_id": "21" }, { "max_price": 192, "customer_id": "22" }, { "max_price": 169, "customer_id": "23" }, { "max_price": 230, "customer_id": "24" }, { "max_price": 278, "customer_id": "25" }, { "max_price": 200, "customer_id": "26" }, { "max_price": 344, "customer_id": "27" }, { "max_price": 175, "customer_id": "28" }, { "max_price": 177, "customer_id": "29" }, { "max_price": 190, "customer_id": "30" }, { "max_price": 190, "customer_id": "31" }, { "max_price": 205, "customer_id": "32" }, { "max_price": 215, "customer_id": "33" }, { "max_price": 270, "customer_id": "34" }, { "max_price": 184, "customer_id": "36" }, { "max_price": 222, "customer_id": "37" }, { "max_price": 370, "customer_id": "38" }, { "max_price": 240, "customer_id": "39" }, { "max_price": 230, "customer_id": "4" }, { "max_price": 229, "customer_id": "41" }, { "max_price": 190, "customer_id": "42" }, { "max_price": 150, "customer_id": "43" }, { "max_price": 175, "customer_id": "44" }, { "max_price": 190, "customer_id": "45" }, { "max_price": 150, "customer_id": "46" }, { "max_price": 310, "customer_id": "48" }, { "max_price": 223, "customer_id": "49" }, { "max_price": 283, "customer_id": "5" }, { "max_price": 185, "customer_id": "50" }, { "max_price": 190, "customer_id": "51" }, { "max_price": 333, "customer_id": "52" }, { "max_price": 165, "customer_id": "6" }, { "max_price": 144, "customer_id": "7" }, { "max_price": 198, "customer_id": "8" }, { "max_price": 210, "customer_id": "9" } ], "generated_dest_index": { "mappings": { "_meta": { "_transform": { "transform": "transform-preview", "version": { "created": "10.0.0" }, "creation_date_in_millis": 1712948905889 }, "created_by": "transform" }, "properties": { "max_price": { "type": "half_float" }, "customer_id": { "type": "keyword" } } }, "settings": { "index": { "number_of_shards": "1", "auto_expand_replicas": "0-1" } }, "aliases": {} } } ```