Full Report
But what if we need to wrangle Windows Event Logs for more than one system? In part 2, we’ll wrangle EVTX logs at scale by incorporating Hayabusa and SOF-ELK into my rapid endpoint investigation workflow (“REIW”)! The post Wrangling Windows Event Logs with Hayabusa & SOF-ELK (Part 2) appeared first on Black Hills Information Security, Inc..
Analysis Summary
# Tool/Technique: Hayabusa & SOF-ELK Integration
## Overview
This summary details a workflow combining the Windows Event Log analyzer **Hayabusa** with the security analytics stack **SOF-ELK** to facilitate the high-volume, rapid investigation of collected endpoint triage data (specifically EVTX logs) at scale. Hayabusa processes individual endpoint logs, and the resulting data is then consolidated and ingested into SOF-ELK for centralized searching, sorting, and filtering across many endpoints. This is presented as the second part of an expanded Rapid Endpoint Investigation Workflow (REIW).
## Technical Details
- Type: Tool / Workflow Integration
- Platform: Windows (source logs), Linux/Elastic Stack (SOF-ELK target)
- Capabilities: EVTX log parsing, reduction/refining of event data, centralized log aggregation and analysis.
- First Seen: Not specified (Hayabusa and SOF-ELK are established tools, but this specific workflow combination is the focus).
## MITRE ATT&CK Mapping
This workflow is heavily focused on Defensive Capabilities (Detection, Investigation, Response), but the underlying components map to defensive evasion and analysis techniques performed by defenders:
- **TA0008 - Collection** (Defense focus: Analyzing collected data)
- T1003 - OS Credential Dumping (Relevant if analyzing logs related to credential access)
- T1059.001 - Command and Scripting Interpreter: PowerShell (Relevant for investigating executed commands logged)
- **TA0011 - Command and Control** (Defense focus: Analyzing network connections logged in EVTX)
- T1071.001 - Application Layer Protocol: Web Protocols (If investigating HTTP/S C2)
- **TA0012 - Detection** (Defense Focus: Using the tools themselves)
- T1560.001 - Archive Collected Data: Archive via Utility (Hayabusa analyzes data that was likely archived)
## Functionality
### Core Capabilities
- **Hayabusa Processing:** Parsing and reducing raw Windows Event Logs (`.evtx`) from endpoint triage collections (REIW).
- **Data Consolidation:** Scripts are used to collect dispersed Hayabusa CSV output files from multiple endpoints and rename/copy them into a single directory structure (`hayabusa-events-offline`).
- **Scalable Ingestion:** Transferring the consolidated CSV files from the endpoint analysis environment to the SOF-ELK ingestion directory (`/logstash/hayabusa/`) using `scp`.
### Advanced Features
- **REIW Integration:** The workflow explicitly plugs into Patterson Cake's Rapid Endpoint Investigation Workflow (REIW), moving beyond single-endpoint spreadsheet analysis to centralized analysis.
- **CSV vs. JSONL:** The workflow specifically utilizes Hayabusa's CSV output format for integration with REIW and subsequent ingestion into SOF-ELK, contrasting with the JSONL output used in Part 1.
- **Elasticsearch Indexing:** SOF-ELK automatically parses the incoming CSVs into the `evtxlogs` index, enabling large-scale searching against historical event data.
## Indicators of Compromise
*Note: The article focuses on the methodology; specific IOs related to an attack were not provided, only IOs related to the process structure.*
- File Hashes: N/A
- File Names: `Hostname-Date-Timestamp-web-and-exe-evtx.xlsx`, `Hostname-Date-Timestamp` folders, `hayabusa-events-offline\*.csv`.
- Registry Keys: N/A
- Network Indicators: SOF-ELK VM IP address (used for `scp` transfer), `elk_user` (SOF-ELK default user), password `forensics`. (Defanged: `[email protected]`)
- Behavioral Indicators: Ingestion monitored via `sof-elk_clear.py -i list`, looking for population of the `evtxlogs` index.
## Associated Threat Actors
- Associated with the builder of the workflow: Patterson Cake / Black Hills Information Security. (No specific malicious APT groups are mentioned as users of this defensive workflow).
## Detection Methods
These are tools used for detection and analysis:
- **Hayabusa:** Signature-based and rule-based analysis of EVTX data.
- **SOF-ELK:** Provides centralized logging and Kibana visualization for searching and pattern detection across ingested data.
- **YARA rules:** Not explicitly mentioned in the context of Hayabusa/SOF-ELK rule sets, only general IR tool usage.
## Mitigation Strategies
The entire process is a mitigation/detection strategy:
- **Rapid Triage:** Speed in collection, parsing, reduction, and analysis ("CPR") to expedite triage.
- **Centralized Analysis:** Ingesting data into SOF-ELK allows analysts to search at scale, quickly ruling out or confirming activity across many systems.
- **Consistent Staging:** Using dedicated volumes (C: for tools, D: for case data) ensures reproducible and predictable workflows.
## Related Tools/Techniques
- **Hayabusa:** Standalone EVTX parser.
- **SOF-ELK:** SIEM/Log Analysis Stack (Elastic Stack distribution).
- **REIW (Rapid Endpoint Investigations Workflow):** The overarching framework that uses Hayabusa and Excel outputs before scaling up to SOF-ELK.
- **KAPE (Kroll Artifact Parser and Extractor):** Referenced as part of the REIW zip expansion and output processing.