Spike in AWS Error Messages

Elastic Stack Serverless Security

A machine learning job detected a significant spike in the rate of a particular error in the CloudTrail messages. Spikes in error messages may accompany attempts at privilege escalation, lateral movement, or discovery.

Rule type: machine_learning

Rule indices: None

Severity: low

Risk score: 21

Runs every: 15m

Searches indices from: now-60m (https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#date-math[Date Math format], see also Additional look-back time)

Maximum alerts per execution: 100

References:

• /reference/security/prebuilt-jobs.md

Tags:

• Elastic
• Cloud
• AWS
• ML

Version: 7

Rule authors:

• Elastic

Rule license: Elastic License v2

The AWS Fleet integration, Filebeat module, or similarly structured data is required to be compatible with this rule.

CloudTrail logging provides visibility on actions taken within an AWS environment. By monitoring these events and understanding
what is considered normal behavior within an organization, suspicious or malicious activity can be spotted when deviations
are observed. This example rule triggers from a large spike in the number of CloudTrail log messages that contain a
particular error message. The error message in question was associated with the response to an AWS API command or method call,
this has the potential to uncover unknown threats or activity.

• Examine the history of the error. Has it manifested before? If the error, which is visible in the aws.cloudtrail.error_message field, only manifested recently, it might be related to recent changes in an automation module or script.
• Examine the request parameters. These may provide indications as to the nature of the task being performed when the error occurred. Is the error related to unsuccessful attempts to enumerate or access objects, data, or secrets? If so, this can sometimes be a byproduct of discovery, privilege escalation or lateral movement attempts.
• Consider the user as identified by the user.name field. Is this activity part of an expected workflow for the user context? Examine the user identity in the aws.cloudtrail.user_identity.arn field and the access key ID in the aws.cloudtrail.user_identity.access_key_id field, which can help identify the precise user context. The user agent details in the user_agent.original field may also indicate what kind of a client made the request.
• Consider the source IP address and geolocation for the calling user who issued the command. Do they look normal for the calling user? If the source is an EC2 IP address, is it associated with an EC2 instance in one of your accounts, or could it be sourcing from an EC2 instance that's not under your control? If it is an authorized EC2 instance, is the activity associated with normal behavior for the instance role or roles? Are there any other alerts or signs of suspicious activity involving this instance?

• This rule has the possibility to produce false positives based on unexpected activity occurring such as bugs or recent
  changes to automation modules or scripting.
• Adoption of new services or implementing new functionality to scripts may generate false positives

• Unusual AWS Command for a User
• Rare AWS Error Code

• If activity is observed as suspicious or malicious, immediate response should be looked into rotating and deleting AWS IAM access keys
• Validate if any unauthorized new users were created, remove these accounts and request password resets for other IAM users
• Look into enabling multi-factor authentication for users
• Follow security best practices outlined by AWS