AWS billing metricset

You can monitor your estimated AWS charges by using Amazon CloudWatch and Cost Explorer.

This aws billing metricset collects metrics both from Cloudwatch and cost explorer for monitoring purposes.

Some specific AWS permissions are required for IAM user to collect estimated billing metrics.

cloudwatch:GetMetricData
cloudwatch:ListMetrics
tag:getResources
sts:GetCallerIdentity
iam:ListAccountAliases
ce:GetCostAndUsage
organizations:ListAccounts

The aws billing metricset comes with a predefined dashboard. For example:

- module: aws
  period: 24h
  metricsets:
    - billing
  credential_profile_name: elastic-beats
  cost_explorer_config:
    group_by_dimension_keys:
      - "AZ"
      - "INSTANCE_TYPE"
      - "SERVICE"
    group_by_tag_keys:
      - "aws:createdBy"

When querying AWS Cost Explorer API, you can group AWS costs using up to two different groups, either dimensions, tag keys, or both. Right now we support group by type dimension and type tag with separate config parameters:

• group_by_dimension_keys: A list of keys used in Cost Explorer to group by dimensions. Valid values are AZ, INSTANCE_TYPE, LINKED_ACCOUNT, OPERATION, PURCHASE_TYPE, REGION, SERVICE, USAGE_TYPE, USAGE_TYPE_GROUP, RECORD_TYPE, OPERATING_SYSTEM, TENANCY, SCOPE, PLATFORM, SUBSCRIPTION_ID, LEGAL_ENTITY_NAME, DEPLOYMENT_OPTION, DATABASE_ENGINE, CACHE_ENGINE, INSTANCE_TYPE_FAMILY, BILLING_ENTITY and RESERVATION_ID.
• group_by_tag_keys: A list of keys used in Cost Explorer to group by tags.

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "aws": {
        "billing": {
            "Currency": "USD",
            "EstimatedCharges": 39.26,
            "ServiceName": "AmazonEKS"
        }
    },
    "cloud": {
        "account": {
            "id": "428152502467",
            "name": "elastic-beats"
        },
        "provider": "aws"
    },
    "event": {
        "dataset": "aws.billing",
        "duration": 115000,
        "module": "aws"
    },
    "metricset": {
        "name": "billing",
        "period": 10000
    },
    "service": {
        "type": "aws"
    }
}