source
The source from which to derive the index or snapshot age. Can be one of name, creation_date, or field_stats.
When using the age filtertype, source requires
direction, unit, unit_count,
and additionally, the optional setting, epoch.
Using name as the source tells Curator to look for a timestring within the index or snapshot name, and convert that into an epoch timestamp (epoch implies UTC).
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 3
Timestrings are parsed from strftime patterns, like %Y.%m.%d, into regular expressions. For example, %Y is 4 digits, so the regular expression for that looks like \d{{4}}, and %m is 2 digits, so the regular expression is \d{{2}}.
What this means is that a simple timestring to match year and month, %Y.%m will result in a regular expression like this: ^.*\d{{4}}\.\d{{2}}.*$. This pattern will match any 4 digits, followed by a period ., followed by 2 digits, occurring anywhere in the index name. This means it will match monthly indices, like index-2016.12, as well as daily indices, like index-2017.04.01, which may not be the intended behavior.
To compensate for this, when selecting indices matching a subset of another pattern, use a second filter with exclude set to True
- filtertype: pattern
kind: timestring
value: '%Y.%m'
- filtertype: pattern
kind: timestring
value: '%Y.%m.%d'
exclude: True
This will prevent the %Y.%m pattern from matching the %Y.%m part of the daily indices.
This applies whether using timestring as a mere pattern match, or as part of date calculations.
creation_date extracts the epoch time of index or snapshot creation.
- filtertype: age
source: creation_date
direction: older
unit: days
unit_count: 3
source can only be field_stats when filtering indices.
In Curator 5.3 and older, source field_stats uses the Field Stats API to calculate either the min_value or the max_value of the field as the stats_result, and then use that value for age comparisons. In 5.4 and above, even though it is still called field_stats, it uses an aggregation to calculate the same values, as the field_stats API is no longer used in Elasticsearch 6.x and up.
field must be of type date in Elasticsearch.
- filtertype: age
source: field_stats
direction: older
unit: days
unit_count: 3
field: '@timestamp'
stats_result: min_value