Full Report
Kent Ickler // You’ve heard us before talk about Bro, an IDS for network monitoring and analysis. We’ve had several installs of Bro over time here at BHIS. It’s about […] The post How to Monitor Network Traffic with Virtualized Bro 2.51 on Ubuntu 16.04.2 on ESXi 6.5 appeared first on Black Hills Information Security, Inc..
Analysis Summary
# Best Practices: Deploying Network Intrusion Detection Systems (IDS) via Virtualization (Bro/Zeek Example)
## Overview
These practices detail the methodology for setting up a dedicated IDS—specifically referencing a Bro (now Zeek) installation—within a virtualized environment (VMware ESXi). The primary goal is to leverage existing virtual infrastructure (like vSwitch mirroring) to feed network traffic to multiple IDS applications running as guest VMs efficiently.
## Key Recommendations
### Immediate Actions
1. **Verify Hardware Capacity:** Ensure the host hardware is adequate to handle the virtualization overhead *and* network processing load, especially noting that budget NICs may not suffice.
2. **Ensure Two Physical NICs:** Confirm the ESXi host has at least two dedicated Network Interface Cards (NICs): one for ESXi Management and one designated as the Mirror/Tap interface.
3. **Secure Initial Access:** Configure ESXi management access via VLANs on the management NIC, ensuring connectivity to both the ESXi server GUI and the future IDS VM.
4. **Set up Promiscuous Mirror Interface on Host:** Dedicate the chosen physical NIC for traffic mirroring (e.g., NIC1) and configure the corresponding vSwitch/Port Group to accept traffic in **Promiscuous Mode**.
### Short-term Improvements (1-3 months)
1. **Configure VM Network Interfaces:** Create the IDS VM specifying two network adapters: one connected to the standard network (for management) and the second connected to the dedicated **Mirror Port Group**.
2. **Configure VM Network Stack (Static Management):** Set a static IP address for the VM's management interface within the guest OS (e.g., Ubuntu).
3. **Enable Mirror NIC Promiscuity in Guest OS:** In the guest OS configuration, explicitly enable promiscuous mode (`promisc on`) and bring the mirror interface up (`up`) upon boot (e.g., via `/etc/rc.local`).
4. **Install and Compile Core Dependencies:** Install the necessary prerequisites for the IDS software (e.g., `cmake`, `make`, `libpcap-dev`, `zlib1g-dev`, etc.) on the guest OS.
5. **Install and Compile IDS Software (e.g., Bro/Zeek):** Compile the IDS software, customizing the installation prefix (e.g., `/opt/bro`) and including optional performance modules like PF_RING if utilized.
### Long-term Strategy (3+ months)
1. **Implement Runtime Path Permanence:** Make the path to the IDS binaries permanent by adding the custom installation bin directory (`/opt/bro/bin`) to the user's profile (`~/.profile`).
2. **Configure IDS Startup Scripts:** Establish an automated mechanism to start the IDS control service upon host reboot (e.g., adding `/opt/bro/bin/broctl start` to `/etc/rc.local`).
3. **Schedule Regular Maintenance:** Set up scheduled tasks (using `cron`) to run daily or periodic maintenance checks for the IDS control utility (e.g., `/opt/bro/bin/broctl cron` every 5 minutes).
4. **Establish Monitoring and Validation Routine:** Implement a routine check (e.g., `tail -f /path/to/conn.log`) to periodically validate that the IDS is actively receiving and processing mirrored traffic after reboots or service restarts.
## Implementation Guidance
### For Small Organizations
- **Consolidate Monitoring:** Utilize the virtualization capability to run multiple IDS/monitoring tools (like Bro and Snort) off a single network tap point by duplicating the mirrored traffic stream via vSwitch settings.
- **Focus on Core Setup:** Prioritize setting up the mirroring and basic configuration. Skip complex optional components (like PF_RING) initially unless resource constraints are severe or specific performance is required.
- **Manual Path Setup:** Rely on manual profile updates (`~/.profile`) for path configuration during the initial build phase.
### For Medium Organizations
- **Utilize Designated Hardware:** Ensure the ESXi host is a purpose-built server, not a repurposed desktop, to handle sustained network monitoring loads.
- **Implement Configuration Strategy:** Use the provided configuration file structure (e.g., `node.cfg`) to clearly separate management/proxy roles from the dedicated worker roles listening on the mirror interface.
- **Monitor Resource Usage:** Closely monitor CPU and memory usage on the ESXi host, as running multiple resource-intensive security tools in VMs can impact host stability.
### For Large Enterprises
- **Adopt Production-Grade Tapping:** Investigate dedicated hardware taps or advanced switch configurations for mirroring critical WAN/Internet traffic segments.
- **Formalize Kernel Update Procedures:** Develop strict procedures for testing and recompiling performance modules (like PF_RING) whenever the underlying host OS kernel is updated, as kernel updates will break modules compiled directly against older kernels.
- **Implement Centralized Management:** If deploying multiple IDS instances, establish a robust, centralized management system to coordinate deployment, status checks (`broctl deploy`), and log collection, rather than relying only on local configuration files.
## Configuration Examples
| Component | Configuration Detail | Value/Setting |
| :--- | :--- | :--- |
| **ESXi vSwitch** | Security Setting | Promiscuous mode: **Accept** |
| **ESXi Port Group** | Security Setting | Promiscuous mode: **Accept** |
| **Guest Interface (Mirror)**| Promiscuous Mode Command (Boot) | `ip link set <interface> promisc on` |
| **Bro Config (`node.cfg`)** | Worker Interface | Change `interface` to mirror NIC (e.g., `ens160`) |
| **Bro Config (`node.cfg`)** | Load Balancing Method | `lb_method=pf_ring` (if PF_RING used) |
| **Bro Config (`node.cfg`)** | Processing Cores | `lb_procs=5` (Example setting) |
| **PATH Permanence** | File location | `nano ~/.profile` |
| **Autostart Script** | File location | `nano /etc/rc.local` |
| **Autostart Command** | Command to add | `/opt/bro/bin/broctl start` |
## Compliance Alignment
While the article focuses on deployment, the resulting IDS setup aligns generally with the following principles:
* **NIST SP 800-53 (AC/AU families):** Provides the foundation for continuous monitoring and audit trail generation necessary for security controls validation.
* **ISO 27001 (A.12.4 Operational Procedures):** Supports required practices for monitoring and logging of network activity.
* **CIS Controls (Control 12: Network Infrastructure Management):** Deployment of network traffic analysis tools contributes to securing the infrastructure layer.
## Common Pitfalls to Avoid
* **Ignoring Promiscuous Mode:** Forgetting to set promiscuous mode on both the ESXi vSwitch/Port Group AND the network interface within the Guest OS will result in zero or incomplete mirrored traffic being received by the IDS.
* **Using Budget NICs:** Using lower-quality NICs that cannot handle the high throughput of mirrored traffic will lead to packet drops before reaching the IDS application, rendering the analysis incomplete.
* **Kernel Version Mismatch:** Failing to anticipate that kernel updates will break custom-compiled modules like PF_RING, necessitating mandatory recompilation after every kernel patch.
* **Default Install Location:** Installing Bro/Zeek to the default location (`/usr/local/bro`) when the configuration examples or maintenance scripts rely on an alternate path (e.g., `/opt/bro`).
## Resources
* **IDS Software Documentation:** Official Bro/Zeek installation documentation (link provided in the source article).
* **VMware Configuration:** ESXi Web GUI for managing Networking -> Physical NICs, Virtual Switches, and Port Groups.
* **Linux Configuration Utilities:** `ip addr`, `tcpdump`, `nano`, `crontab -e`.