integration-docs
Loading

Google Threat Intelligence

Serverless Security Stack 9.0.0

Version 0.8.0 Beta (View all)
Subscription level
What's this?
Basic
Developed by
What's this?
Partner
Ingestion method(s) API
The Google Threat Intelligence integration v0.8.0 is in beta

To use beta integrations, go to the Integrations page in Kibana, scroll down, and toggle on the Display beta integrations option.

Google Threat Intelligence is a security solution that helps organizations detect, analyze, and mitigate threats. It leverages Google's global telemetry, advanced analytics, and vast infrastructure to provide actionable insights. Key features include threat detection, malware and phishing analysis, and real-time threat alerts.

Google Threat Intelligence integration offers support for two APIs:

  1. Threat List API to deliver hourly data chunks. The Threat Lists feature allows customers to consume Indicators of Compromise (IOCs) categorized by various threat types.
  2. IOC Stream API to deliver various types of Indicators of Compromise (IOCs) originating from multiple sources. Depending on the source of the notification, different context-specific attributes are added to enrich the IOCs.

The Threat List API provides the following types of threat feeds:

  • Cryptominers
  • Daily Top Trending
  • First Stage Delivery Vectors
  • Infostealers
  • Internet of Things (IoT)
  • Linux
  • Malicious Network Infrastructure
  • Malware
  • Mobile
  • OS X
  • Phishing
  • Ransomware
  • Threat Actor
  • Vulnerability Weaponization

Customers can access a subset of the available threat lists based on their Google Threat Intelligence (GTI) tier:

  • GTI Standard: Ransomware, Malicious Network Infrastructure
  • GTI Enterprise: Ransomware, Malicious Network Infrastructure, Malware, Threat Actor, Daily Top Trending
  • GTI Enterprise+: Access to all available threat lists

Data collection is available for all threat feeds and IOC Stream, each with a separate data stream. By default, Ransomware and Malicious Network Infrastructure is enabled. Users can enable additional data streams based on their GTI subscription tier. If a user enables data collection for a data stream they do not have access to, it will result in an error log on the Discover page.

Agentless integrations allow you to collect data without having to manage Elastic Agent in your cloud. They make manual agent deployment unnecessary, so you can focus on your data instead of the agent that collects it. For more information, refer to Agentless integrations and the Agentless integrations FAQ.

Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. This functionality is in beta and is subject to change. Beta features are not subject to the support SLA of official GA features.

Elastic Agent must be installed. For more details, check the Elastic Agent installation instructions.

  • VirusTotal URL will work as the base URL for this integration: https://www.virustotal.com
  • An API key will be used to authenticate your request.
  • Time Selection of Initial Interval and Interval:
    • Users need to specify the initial interval and interval in an hourly format, such as 2h, 3h, etc. Note: Please make sure both initial interval and interval are in hours and the initial interval is greater than 2 hours.
  1. In Kibana, go to Management > Integrations.
  2. In the "Search for integrations" search bar, type Google Threat Intelligence.
  3. Click on the Google Threat Intelligence integration from the search results.
  4. Click on the Add Google Threat Intelligence button to add the integration.
  5. While adding the integration, to collect logs via REST API, provide the following details:
    • Enable the type of data stream you have access to.
    • Access Token
    • Initial Interval
    • Interval
    • (Optional) Query to add custom query filtering on relationship, GTI score, and positives. (not applicable to IOC Stream)
  6. Click on Save and Continue to save the integration. Note: Please make only the threat feed types you have the privilege to access are enabled.

To keep the collected data up to date, Transforms are used.

Users can view the transforms by navigating to Management > Stack Management > Transforms.

Here, users can see continuously running transforms and also view the latest transformed GTI data in the Discover section.

Currently, 10 transforms are available across all GTI integration data streams.

The following list contains the transforms and their associated pipelines:

Transform Name Description
IP Transform (ID: logs-ti_google_threat_intelligence.ip_ioc, Pipeline: ti_google_threat_intelligence-latest_ip_ioc-transform-pipeline) Keeps IP entity type data up to date.
URL Transform (ID: logs-ti_google_threat_intelligence.url_ioc, Pipeline: ti_google_threat_intelligence-latest_url_ioc-transform-pipeline) Keeps URL entity type data up to date.
Domain Transform (ID: logs-ti_google_threat_intelligence.domain_ioc, Pipeline: ti_google_threat_intelligence-latest_domain_ioc-transform-pipeline) Keeps Domain entity type data up to date.
File Transform (ID: logs-ti_google_threat_intelligence.file_ioc, Pipeline: ti_google_threat_intelligence-latest_file_ioc-transform-pipeline) Keeps File entity type data up to date.
IP IOC Stream Transform (ID: logs-ti_google_threat_intelligence.ip_ioc_st, Pipeline: ti_google_threat_intelligence-latest_ip_ioc_st-transform-pipeline) Keeps IP entity type data up to date for IOC Stream.
URL IOC Stream Transform (ID: logs-ti_google_threat_intelligence.url_ioc_st, Pipeline: ti_google_threat_intelligence-latest_url_ioc_st-transform-pipeline) Keeps URL entity type data up to date for IOC Stream.
Domain IOC Stream Transform (ID: logs-ti_google_threat_intelligence.domain_ioc_st, Pipeline: ti_google_threat_intelligence-latest_domain_ioc_st-transform-pipeline) Keeps Domain entity type data up to date for IOC Stream.
File IOC Stream Transform (ID: logs-ti_google_threat_intelligence.file_ioc_st, Pipeline: ti_google_threat_intelligence-latest_file_ioc_st-transform-pipeline) Keeps File entity type data up to date for IOC Stream.
Detected IOC Transform (ID: logs-ti_google_threat_intelligence.rule, Pipeline: ti_google_threat_intelligence-correlation_detection_rule-pipeline) Filters and extracts necessary information from Detected IOCs from threat feed.
Detected IOC from IOC stream Transform (ID: logs-ti_google_threat_intelligence.rule_ioc_st, Pipeline: ti_google_threat_intelligence-correlation_detection_rule_ioc_st-pipeline) Filters and extracts necessary information from Detected IOCs from IOC stream.

All the GTI transforms are automatically started and the dashboards Threat Feed Overview and IOC Stream Overview are populated.

The labels.is_transform_source field indicates log origin:

  • False for transformed index
  • True for source index

For example:

  • The query event.module: ti_google_threat_intelligence and labels.is_transform_source: true indicates that the logs originate from the source index.
  • The query event.module: ti_google_threat_intelligence and labels.is_transform_source: false indicates that the logs originate from the transformed index.

A retention policy is used to remove data older than the default retention period. For more details, refer to the Retention Policy Documentation.

In this integration, all data streams have a retention period of 30 days.

Detection Rules match the user's Elastic environment data with GTI data, generating an alert if a match is found. To access detection rules:

  1. Navigate to Security > Rules > Detection Rules and click on Add Elastic Rules.
  2. Search for Google Threat Intelligence to find prebuilt Elastic detection rules.
  3. Four detection rules are available for IP, URL, File, and Domain. Users can install one or more rules as needed.

To tailor a rule based on Elastic environment:

  1. Click the three dots on the right side of any detection rule.
  2. Select Duplicate Rule.
  3. Modify the duplicated rule to tailor it to your Elastic environment:
    • Index Pattern: Add the index pattern relevant to your data. Keeping this specific ensures optimal performance.
    • Custom Query: Further refine rule conditions.
    • Indicator Mapping: Map relevant fields from your Elastic environment to GTI fields. Do not modify the indicator index field.
    • Schedule Rules:
      • Set Runs Every - Defines how frequently the rule runs.
      • Additional Lookback Time - Specifies how far back to check for matches.

Once saved, successfully executed rules will generate alerts. Users can view these alerts in the Alerts section.

The following are the names of the eight sample rules:

Sample Rule Name Description
Google Threat Intelligence URL IOC Correlation Detects and alerts on matches between URL IOCs collected by GTI data with user's selected Elastic environment data.
Google Threat Intelligence Domain IOC Correlation Detects and alerts on matches between Domain IOCs collected by GTI data with user's selected Elastic environment data.
Google Threat Intelligence File IOC Correlation Detects and alerts on matches between File IOCs collected by GTI data with user's selected Elastic environment data.
Google Threat Intelligence IP Address IOC Correlation Detects and alerts on matches between IP Address IOCs collected by GTI data with user's selected Elastic environment data.
Google Threat Intelligence URL IOC Stream Correlation Detects and alerts on matches between URL IOCs collected by GTI IOC Stream data with user's selected Elastic environment data.
Google Threat Intelligence Domain IOC Stream Correlation Detects and alerts on matches between Domain IOCs collected by GTI IOC Stream data with user's selected Elastic environment data.
Google Threat Intelligence File IOC Stream Correlation Detects and alerts on matches between File IOCs collected by GTI IOC Stream data with user's selected Elastic environment data.
Google Threat Intelligence IP Address IOC Stream Correlation Detects and alerts on matches between IP Address IOCs collected by GTI IOC Stream data with user's selected Elastic environment data.
Note

The following two transforms are available to filter relevant data from alerts.

  • Detected IOC Transform (ID: logs-ti_google_threat_intelligence.rule)
  • Detected IOC from IOC stream Transform (ID: logs-ti_google_threat_intelligence.rule_ioc_st)

These transforms are automatically started to populate Threat Intelligence, Adversary Intelligence and IOC Stream Threat Intelligence dashboards. The data_stream.dataset: ti_google_threat_intelligence.enriched_ioc and data_stream.dataset: ti_google_threat_intelligence.enriched_ioc_stream field represents logs for enriched threat intelligence data, which can be analyzed in the Discover section.

  1. If an event contains multiple matching mappings (e.g., two file hash fields within the same event match GTI data), only one alert per detection rule will be generated for that event.
  2. If an IOC from the user's Elasticsearch index is enriched with GTI information, and the GTI information is updated later, the changes are not reflected in the dashboards because Elastic detection rules only run on live data.
  1. If you see an error like Package 2025031310 is not available until 2025-03-13 at 11:00 UTC because of privacy policy., ensure that your initial interval and interval are set in hours and the initial interval is greater than two hours.
  2. If events are not appearing in the transformed index, check if transforms are running without errors. If you encounter issues, refer to Troubleshooting transforms.
  3. If detection rules take longer to run, ensure you have specified index patterns and applied queries to make your source events more specific. Note: More events in index patterns mean more time needed for detection rules to run.
  4. Ensure that relevant fields are correctly mapped in the Indicator Mapping section. Verify that fields in the specified index pattern are properly mapped, and ensure entity-specific fields (e.g., IP fields to IP fields, keyword fields like file hash SHA256 to corresponding file hash SHA256 fields) are accurately configured.
  5. If any transform is not in a Healthy state, try resetting it:
    • Click the three dots next to the transform, then select Reset.
    • After resetting, restart the transform if they are not automatically started.

This is the Cryptominer dataset.

This is the First Stage Delivery Vectors dataset.

This is the Infostealers dataset.

This is the IOC Stream dataset.

This is the Internet of Things dataset.

This is the Linux dataset.

This is the Malicious Network Infrastructure dataset.

This is the Malware dataset.

This is the Mobile dataset.

This is the OS X dataset.

This is the Phishing dataset.

This is the Ransomware dataset.

This is the Threat Actor dataset.

This is the Daily Top trending dataset.

This is the Vulnerability Weaponization dataset.

This integration includes one or more Kibana dashboards that visualizes the data collected by the integration. The screenshots below illustrate how the ingested data is displayed.