Full Report
Insights, philosophy, and candor to make you rethink the way you look at data science
Analysis Summary
# Research: 🎙️SECURITY.COM The Podcast: Diving Deep Into Vulnerability Risk Modeling with Empirical Security
## Metadata
- Authors: Dan Mellinger (Host), Ed Bellis, Michael Roytman, Jay Jacobs (Guests from Empirical Security)
- Institution: Broadcom / Empirical Security
- Publication: SECURITY.COM The Podcast
- Date: September 5, 2025 (Projected based on podcast date structure)
## Abstract
This podcast episode features an in-depth discussion with the founders of Empirical Security regarding the evolution and philosophy behind data-driven vulnerability risk modeling, particularly focusing on empirical, risk-based vulnerability management (RBVM) frameworks. Key topics include the development and use of predictive scoring models like EPSS, the concept of focusing remediation efforts on the "tail of risk," and the necessity of integrating security data across traditionally siloed team functions. The conversation also touches upon advanced topics such as leveraging Large Language Models (LLMs) (e.g., JayPT) in security analysis.
## Research Objective
The primary goal of the discussion, as framed by the guests from Empirical Security, is to foster a deeper, more empirical understanding of vulnerability management, moving beyond simple severity ratings (like CVSS) to prioritize actual exploitable risk, and to critically examine how data science and AI are being applied or misused in the current security landscape.
## Methodology
### Approach
The methodology is a dialogic, expert-interview format designed to convey experiential knowledge, philosophical stances, and technical approaches used by Empirical Security in developing and implementing risk-based vulnerability management solutions. It centers on case studies and real-world insights derived from analyzing vast datasets of vulnerability and exploit information.
### Dataset/Environment
The discussion heavily references the datasets used to derive empirical security metrics, such as historical vulnerability data, exploit telemetry, threat intelligence feeds, and internal organizational context (implied by the need to 'break siloes'). Specific mentions relate to the data underpinning the Exploit Prediction Scoring System (EPSS).
### Tools & Technologies
- **EPSS (Exploit Prediction Scoring System):** Explicitly discussed as a foundational, data-driven metric for predicting which vulnerabilities are likely to be exploited in the wild.
- **Data Science Pipelines:** The underlying infrastructure required to synthesize disparate security data sources into actionable risk scores.
- **Large Language Models (LLMs) / JayPT:** Mentioned as an experimental application for enhancing security analysis or operational workflows.
## Key Findings
### Primary Results
1. **Evolution to Data-Driven Vulnerability Management:** The discussion highlights the necessary shift from static, severity-based vulnerability scoring (like CVSS) to dynamic, empirical models that predict *actual* exploitation likelihood.
2. **Focusing on the "Tail of Risk":** A core philosophy demonstrated is the strategic importance of concentrating remediation efforts on the small subset of vulnerabilities that carry the highest predicted risk (the extreme tail of the risk distribution), rather than diffuse remediation across all high-severity flaws.
3. **Necessity of Breaking Silos:** Effective risk modeling and management require the merging of operational data, threat intelligence, and system context, demanding collaboration across traditionally separate team functions (e.g., vulnerability scanning teams, patching teams, threat intelligence teams).
4. **Differentiation in the Age of AI:** Empirical approaches differentiate themselves from competitors by grounding their models in verifiable, quantitative outcomes rather than relying solely on broad, generalized AI descriptions or hype.
### Supporting Evidence
Evidence is presented through the success of their advocated methodologies (like EPSS adoption) and their claimed ability to accurately prioritize assets and vulnerabilities that subsequently face exploitation attempts. (Specific metrics would require listening to the audio content.)
### Novel Contributions
- **Philosophical Rigor in RBVM:** A contribution lies in articulating a disciplined, skeptical approach to risk quantification, especially regarding the integration of burgeoning technologies like LLMs into security decision frameworks.
- **Methodological Insight into Risk Tail Management:** Practical articulation of *how* organizations can operationalize insights from predictive models to manage the highest-leverage risks effectively.
## Technical Details
The technical discussion revolves around the mechanics of quantitative risk prediction. The concept of "living on the tail of risk" implies that the underlying models (like EPSS) must have sufficient statistical power in the extreme right tail of the predicted distribution to reliably separate the few truly dangerous vulnerabilities from the many potential ones. The discussion implicitly critiques models lacking this empirical grounding when applied to prioritization, suggesting many current tools rely too heavily on qualitative assessments disguised as data science.
## Practical Implications
### For Security Practitioners
Practitioners must aggressively seek tools and processes that integrate real-world exploit data (e.g., EPSS) into their daily triage workflows to avoid wasting effort on vulnerabilities that pose negligible risk.
### For Defenders
Defenders should advocate for breaking down internal data silos. A vulnerability's true risk is context-dependent, requiring intelligence on asset criticality, existing compensating controls, and external threat activity—data often held in separate departments.
### For Researchers
The research implies a need for continued rigor in developing and validating open, empirical models, pushing back against proprietary "black-box" risk scores that lack transparent underlying data and methodologies.
## Limitations
As a podcast discussion rather than a formal paper, specific quantitative limitations (e.g., model drift, specific statistical confidence intervals) are likely discussed qualitatively rather than rigorously documented. The inherent challenge of relying on external data (like exploit logs) to predict individual internal risk is an implicit limitation.
## Comparison to Prior Work
This work directly builds upon and seeks to refine the foundational work in predictive vulnerability scoring, most notably codified by EPSS. It represents a progression from purely theoretical risk frameworks to applied, empirical risk engineering within a commercial product context.
## Real-world Applications
- **Vulnerability Prioritization:** Direct application in security operations centers (SOCs) and vulnerability management teams to automate triage and focus patching resources.
- **Resource Allocation:** Justifying budget and team effort allocation based on statistically modeled risk exposure rather than anecdotal evidence or compliance checklists.
## Future Work
The mention of LLM experiments (JayPT) suggests future work centers on integrating advanced generative AI techniques to augment security analysis, provided these applications meet the empirical rigor demanded by the Empirical Security philosophy.
## References
- [Empirical Security blog](https://www.empiricalsecurity.com/) (General reference for their work)
- **Related Research (Implied):** Papers and documentation related to the development and validation of the Exploit Prediction Scoring System (EPSS).