BBOT integration

The Bighuge BLS OSINT Tool (BBOT) integration is intended for BBOT installations, an Attack Surface Management (ASM) Open Source Inteligence (OSINT) Tool.

Once the BBOT scan is complete, the integration will ingest the results into Elastic.

This integration requires the external use of BBOT. You will have to download and run the tool apart from this integration. Once your scan is complete, this integration can ingest the results into Elastic using the BBOT json output module or the http output module aka. HTTP POST/webhooks.

As of version 1.1.2 of this integration both BBOT 1.x and 2.x are supported. However all examples provided by the module documentation now assume use of BBOT 2.x.

Please note that BBOT 1.x formats are currently being supported on a best effort basis only and support will eventually be removed.

Please upgrade to BBOT 2.x as soon as possible to ensure continuity of features and functionality.

In order to support the conflicting BBOT NDJSON event formats the following occurs during ingest of BBOT 2.x SCAN events:

1. The bbot.data.scan field, if received as an object, is renamed to bbot.data.scan_config
2. bbot.data.scan is set using the ID and name of the scan in order to replicate the field type and content generated by BBOT 1.x

This tool is used to enhance your external knowledge of your environment. This is done through the integration of many tools into BBOT providing a overview of your attack surface. Here is how it works.

Important Note

You will have to provide the following parameter in your BBOT scan for your output.json to be formatted correctly.

-c modules.json.siem_friendly=true

Or if using the HTTP output modules,

-c modules.http.siem_friendly=true

We also strongly recommend ensuring you filter, at minimum, HTTP_RESPONSE and RAW_TEXT events from reaching the SIEM as these events can contain extreme field data which may not be consumable.

Ensure the configuration option is left at the default or is similar to the following:

omit_event_types:
- HTTP_RESPONSE
- RAW_TEXT

Example BBOT Scan

If using the integration to collect log files from disk, you can simply use the json output module.

bbot -t example.com -p subdomain-enum -c modules.json.siem_friendly=true -om json

If using the integration to receive events via HTTP endpoint listener, you can use the the http output module.

bbot -t example.com -p subdomain-enum --config modules.http.url=http://your.elastic.agent:8080/bbot/asm_intel modules.http.username=bbot modules.http.password=P@55w0rd modules.http.siem_friendly=true -om http

You will have to configure the path for the output file within the integration settings. A common and popular path that could work here is:

The output modules can also be configured as part of the bbot.yml or preset files.

config:
  modules:
    http:
      url: 'https://your.elastic.agent:8443'
      method: POST
      username: 'bbot'
      password: 'P@55w0rd'
      siem_friendly: true
    json:
      siem_friendly: true

Example BBOT Path

/home/<user>/.bbot/scans/*/output.json

BBOT Scanning Documentation.

This integration collects the following logs:

• asm_intel Made up of the findings found in the BBOT Scans.

Elastic Agent must be installed. For more details and installation instructions, please refer to the Elastic Agent Installation Guide.

There are several options for installing and managing Elastic Agent:

With this approach, you install Elastic Agent and use Fleet in Kibana to define, configure, and manage your agents in a central location. We recommend using Fleet management because it makes the management and upgrade of your agents considerably easier.

With this approach, you install Elastic Agent and manually configure the agent locally on the system where it’s installed. You are responsible for managing and upgrading the agents. This approach is reserved for advanced users only.

You can run Elastic Agent inside a container, either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry, and we provide deployment manifests for running on Kubernetes.

Please note, there are minimum requirements for running Elastic Agent. For more information, refer to the Elastic Agent Minimum Requirements.

1. In Kibana navigate to Management > Integrations.
2. In "Search for integrations" top bar, search for BBOT.
3. Select the "BBOT" integration from the search results.
4. Select "Add BBOT" to add the integration.
5. Add all the required integration configuration parameters including the Path to ndjson output file.
6. Save the integration.

{
    "@timestamp": "2024-02-29T01:41:47.779Z",
    "agent": {
        "ephemeral_id": "0c16a7c4-c1a9-424a-b859-29af68197f5c",
        "id": "91b85fe6-d0c4-4bfc-863d-e10a462e0968",
        "name": "elastic-agent-38224",
        "type": "filebeat",
        "version": "8.15.3"
    },
    "bbot": {
        "data": {
            "dns_name": "example.com"
        },
        "id": "DNS_NAME:f57ba0828becd7bf94faa616db081ed06f31bd3d",
        "module": "TARGET",
        "module_sequence": "TARGET",
        "resolved_hosts": [
            "1.128.0.0"
        ],
        "scan": "SCAN:725368977d3a680e579707504e59428a7e3acc9d",
        "scope_distance": 0,
        "source": "SCAN:725368977d3a680e579707504e59428a7e3acc9d",
        "tags": [
            "resolved",
            "a-record",
            "target",
            "in-scope",
            "subdomain"
        ],
        "timestamp": "1.709170907779394E9",
        "type": "DNS_NAME"
    },
    "data_stream": {
        "dataset": "bbot.asm_intel",
        "namespace": "28270",
        "type": "logs"
    },
    "ecs": {
        "version": "8.11.0"
    },
    "elastic_agent": {
        "id": "91b85fe6-d0c4-4bfc-863d-e10a462e0968",
        "snapshot": false,
        "version": "8.15.3"
    },
    "event": {
        "agent_id_status": "verified",
        "category": [
            "network"
        ],
        "dataset": "bbot.asm_intel",
        "ingested": "2024-11-14T08:17:02Z",
        "kind": "asset",
        "original": "{\"data\":{\"DNS_NAME\":\"example.com\"},\"id\":\"DNS_NAME:f57ba0828becd7bf94faa616db081ed06f31bd3d\",\"module\":\"TARGET\",\"module_sequence\":\"TARGET\",\"resolved_hosts\":[\"1.128.0.0\"],\"scan\":\"SCAN:725368977d3a680e579707504e59428a7e3acc9d\",\"scope_distance\":0,\"source\":\"SCAN:725368977d3a680e579707504e59428a7e3acc9d\",\"tags\":[\"resolved\",\"a-record\",\"target\",\"in-scope\",\"subdomain\"],\"timestamp\":1709170907.779394,\"type\":\"DNS_NAME\"}",
        "type": [
            "info"
        ]
    },
    "host": {
        "name": "example.com"
    },
    "input": {
        "type": "http_endpoint"
    },
    "related": {
        "hosts": [
            "1.128.0.0"
        ]
    },
    "tags": [
        "preserve_original_event",
        "preserve_duplicate_custom_fields",
        "forwarded",
        "bbot"
    ],
    "url": {
        "domain": [
            "example.com"
        ]
    }
}