Full Report
Recently, I decided to take a look at Steampipe again. I like SQL and the structure it provides, and after playing around a bit I figured: “Wouldn’t it be cool to write a plugin for the immensely popular projectdiscovery tools?”. That is exactly what I did and you can find the source code for it here: https://github.com/sensepost/steampipe-plugin-projectdiscovery. overview For the purposes of footprinting, everything you can do with steampipe you can do with a bash script. You technically don’t need SQL. However, with bash you always need to bust out some text wrangling with tools like sed and awk. That in itself isn’t bad, but the data is inherently unstructured and error-prone as a result. Instead, if we could have our data in a database, we could do arbitrary lookups, join and more!
Analysis Summary
# Tool/Technique: Steampipe Plugin for ProjectDiscovery Tools
## Overview
This entry summarizes the development and use of a custom Steampipe plugin designed to integrate the functionality of several popular ProjectDiscovery tools (such as `subfinder`, `httpx`, `naabu`, etc.) into a structured, SQL-queryable environment via Steampipe. The primary purpose is to move beyond unstructured text processing common in traditional scripting (e.g., using `sed` and `awk` on bash script outputs) towards structured data analysis, enabling complex lookups, joins, and easier data manipulation for reconnaissance and footprinting activities.
## Technical Details
- Type: Tool (Plugin/Integration)
- Platform: Environments leveraging Steampipe (generally cross-platform, but the underlying ProjectDiscovery tools often target Linux/macOS/Windows).
- Capabilities: Allows users to query the outputs and capabilities of ProjectDiscovery tools using standard SQL syntax.
- First Seen: Context suggests the plugin creation was recent around the article's publication date (July 03, 2023).
## MITRE ATT&CK Mapping
The tool itself is an offensive/security research tool used primarily during the initial phases of engagement.
- **TA0043 - Reconnaissance**
- **T1593 - Gather Victim Identity Information** (Indirectly, by enumerating domains/IPs)
- **T1595 - Active Scanning** (Leveraging tools like `naabu` via SQL queries)
- **T1598 - Obtain Virtualization** (Leveraging tools like `cdncheck` via SQL queries)
## Functionality
### Core Capabilities
The plugin implements tables corresponding to several ProjectDiscovery utilities:
* `asnmap`
* `cdncheck`
* `chaos` (Requires API Key)
* `cloudlist`
* `dnsx`
* `httpx`
* `katana`
* `naabu`
* `subfinder`
* `tlsx`
The core benefit is structuring the resulting data from these tools (which are typically command-line utilities spitting out text) into database tables, allowing for relational operations.
### Advanced Features
* **Data Joining:** The key advanced feature is the ability to join results across different ProjectDiscovery tools or other Steampipe plugins (e.g., joining `crt.sh` data with `net` plugin data, and subsequently analyzing IPs with the custom `projectdiscovery_cdncheck` table).
* **Complex Filtering and Aggregation:** Users can perform complex SQL queries, such as filtering subdomains based on certificate transparency logs, resolving them to IPs, and then querying CDN information, all within a single query.
* **Structured Output:** Eliminates the need for manual text processing using tools like `sed` and `awk`.
## Indicators of Compromise
This entry describes a legitimate research/auditing tool integration, not malware. Therefore, standard malware IOCs (Hashes, C2s, Registry Keys) are not applicable.
* **Behavioral Indicators:** Execution of complex SQL queries through the Steampipe interface that trigger reconnaissance activities corresponding to the incorporated ProjectDiscovery toolsets (e.g., mass DNS queries, port scanning attempts based on IP lists derived from certificates).
## Associated Threat Actors
This tool is primarily associated with security researchers, penetration testers, and bug bounty hunters utilizing the ProjectDiscovery ecosystem for efficient reconnaissance. No specific malicious threat actor group is cited as regularly using this specific Steampipe plugin integration.
## Detection Methods
Detection focuses on monitoring the execution of the Steampipe application or unauthorized calls to the underlying ProjectDiscovery functions if the plugin were to be implemented in an environment intended for security research/auditing.
* **Behavioral Detection:** Monitoring for the execution of complex, multi-step SQL queries against the Steampipe instance that strongly suggest automated infrastructure enumeration.
* **Signature-based detection:** Detection of the specific Steampipe plugin installation directory or related file paths.
## Mitigation Strategies
Mitigation concerns controlling the scope and execution environment of Steampipe instances if they are not intended for use within the network.
* **Prevention Measures:** Restricting the installation or execution of Steampipe environments, especially those containing third-party plugins sourced from external Repositories like GitHub, in sensitive production environments.
* **Hardening Recommendations:** Reviewing and scoping access to plugins (e.g., ensuring the `chaos` plugin cannot function without a valid, authorized API key).
## Related Tools/Techniques
* **Steampipe:** The primary framework used to run the SQL interface.
* **ProjectDiscovery Tools:** `subfinder`, `httpx`, `naabu`, `katana`, etc.
* **Alternative Data Wrangling:** Traditional bash scripting using `sed`, `awk`, and standard command-line tool chaining.
* **Other Steampipe Plugins:** `crt.sh` plugin, `net` plugin (used in examples for joining data).