Full Report
Originally published at Arachne Digital.A Familiar Hype CycleArtificial-intelligence agents embedded in security information and event management (SIEM) platforms promise to automate investigation and triage. Some are claiming that AI will replace human analysts. Yet the effectiveness of any analytic model, machine-learning or rule-based, remains constrained by the quality of the telemetry it receives. If essential events or log fields are missing, even the most sophisticated model will incorrectly classify or overlook malicious activity.Threat-informed defence offers a rigorous, repeatable framework for determining exactly which logs, and which log fields, are required to detect the tactics, techniques, and procedures (TTPs) of cyber threat actors (CTAs) that actively target your industry and geography. Continuous cyber threat intelligence (CTI) keeps those requirements aligned with the evolving threat landscape.AI Agents Cannot Compensate for Missing TelemetryModern SIEMs can ingest billions of events per day, and AI agents excel at correlating and prioritising this volume. However, even the most advanced AI analytics cannot overcome fundamental telemetry gaps. An effective detection pipeline still depends on three conditions:The required event types must be collected.The specific fields needed for analysis must be present.The data must arrive clean, consistently formatted, and in near-real time.When any of these prerequisites fail, like critical command-line parameters are excluded to save storage, or endpoint logs arrive hours late, true positives become false negatives, alerts lack sufficient context, and investigations stall.On the flip side, attempting to “ingest everything” is neither sustainable nor cost-effective. Only a disciplined, intelligence-driven log-onboarding strategy ensures that AI is working with evidence strong enough to justify automated decisions.Threat-Informed Defence: A Four-Step MethodIdentify Relevant Adversaries: Use curated CTI to determine which actors are actively targeting organisations with your industry profile and geographic footprint.Enumerate Their TTPs: Map each adversary’s behaviours to MITRE ATT&CK techniques. This creates a formalised threat model grounded in evidence.Link Techniques to Detection Data Sources: ATT&CK provides data-source–to-technique mappings (Data Source IDs). Translate these into specific log sources. For example, DS0017: Command Execution maps to Windows Event ID 4688 plus parent-process correlation in EDR telemetry.Validate Log Coverage and Field Completeness: Build a matrix indicating whether each required source and field is present (green), partially present (yellow), or absent (red). The matrix becomes both a roadmap for engineering work and an audit artefact for regulators and executives.Once established, this process should be repeated on a defined cadence or when major technology changes occur.Case Study: Financial Institutions in South AmericaKey findings from Arachne Digital CTI for financial institutions across South America, taken 21 June 2025 included:Primary adversaries include: FIN7, TA549, Battery Elf, Gold Lagoon, and APT44High-frequency techniques include:T1059.001 PowerShellT1105 Ingress Tool TransferT1555.003 Credentials from Web BrowsersT1190 Exploit Public-Facing ApplicationT1005 Data from Local SystemUsing these techniques, the data-source requirements include:ATT&CK Technique: T1059.001 PowerShellEssential data sources include: Command and Process logsCritical data components include: Command Execution and Process CreationATT&CK Technique: T1105 Ingress Tool TransferEssential data sources include: File and Network Traffic logsCritical data components include: File Creation and Network Traffic ContentATT&CK Technique: T1555.003 Browser Credential TheftEssential data sources include: File and Process logsCritical data components include: File Access and OS API ExecutionATT&CK Technique: T1190 Exploit Public-Facing ApplicationEssential data sources include: Application Log and Network Traffic logsCritical data components include: Application Log Content and Network Traffic ContentATT&CK Technique: T1005 Data from Local SystemEssential data sources include: Process and Script logsCritical data components include: Process Creation and Script ExecutionOften, required fields are absent or inconsistently collected, primarily due to default configurations that suppressed “verbose” logging categories. If you are anyone working with a SOC, from a CISO right down to a tier one analyst, can you say that you know all the relevant CTAs to your organisation, their current TTPs, and that all the required logs are ingested into your SIEM with all the required fields? And do you have a way to ensure you stay up to date as the CTAs and TTPs shift?If you can’t, an AI agent won’t solve your fundamental issue.To prepare for deploying AI agents, maintain a configuration-management baseline that specifies the event ID, logging channel, and policy setting for each ATT&CK data component. Automate compliance checks via PowerShell, Ansible, or your preferred configuration-management tool.Cost-Efficiency: Log More Where It Matters, Less Where It Doesn’tStrengthening telemetry does not have to equal runaway storage bills. The same ATT&CK-aligned matrix that highlights missing data sources also exposes over-collected ones, logs and fields that contribute little or nothing to detections relevant to your threat model.For each log source:Tag the ATT&CK techniques it enables and assign a rough business value: high (critical detection gap), medium (useful enrichment), or low (no mapped techniques).Pull ingestion metrics from your SIEM or data-lake billing dashboard to calculate daily gigabytes and monthly cost.Create a simple 3×3 heat map (value on one axis, cost on the other). Anything “low value / high cost” is a candidate for optimisation.Based on your findings you can make a judgement to:Retain but Tier: Move low-value logs to chilled or object storage with longer query latency but a fraction of the price.Sample or Filter: Keep only events that include fields tied to medium or high ATT&CK value. For example, you could look at dropping firewall allows, but retaining denies.Shorten Retention: Regulatory requirements rarely mandate 365-day hot storage for every log type. Right-size retention based on compliance need plus investigative usefulness.Dollars freed by pruning low-value telemetry can bankroll onboarding of high-value sources, extended EDR fields, detailed SaaS audit logs, or container runtime events, without increasing the overall budget line.Also, track the before-and-after cost curve alongside detection coverage metrics. This evidence helps justify future security spend to finance and the board.Threat-informed defence is not just a security win; it’s a budget optimisation tool that ensures every gigabyte you keep is pulling its weight.Continuous Intelligence Keeps the Matrix CurrentThreat landscapes are dynamic:New or re-emerging groups (e.g., FIN6, applicable to our case study above) may adopt techniques that demand additional telemetry.Shifts in tooling (PowerShell downgraded, WMI upgraded) alter the priority of data sources.Emerging vulnerabilities introduce detection requirements for previously irrelevant platforms.Arachne Digital’s feeds deliver sector-specific intelligence as machine-readable JSON, including ATT&CK mappings, and first-/last-seen dates. Integrating this feed with your log-coverage matrix allows automatic creation of engineering tickets whenever a new technique enters the scope of relevant threats, or when there are possible cost savings to be made.By contrast, deploying AI on incomplete data often increases workload, as analysts chase poorly prioritised or context deficient alerts.Implementation RoadmapAcquire an Industry-Specific Intelligence BaselineFree introductory reports and API trials are available from Arachne Digital.Construct or Update the ATT&CK Log-Coverage MatrixInclude source, event ID, and critical fields. Mark gaps clearly.Remediate GapsPrioritise high-impact techniques and low-effort fixes.Align storage budgets with security value.Automate Continuous ValidationCombine configuration-management tools with CTI updates to keep the matrix evergreen.Deploy or Enhance AI AnalyticsOnce telemetry quality is verified, AI agents can work to their full potential.How Arachne Digital Accelerates the ProcessThread & Tracery: Automatically map threat-report text to ATT&CK techniques, providing machine-readable context suitable for log engineering workflows.Sector-Focused Intelligence Feeds: Deliver only the adversary activity relevant to your environment, reducing analysis overhead.Human Curated Accuracy: Experienced analysts validate each mapping, ensuring false data does not contaminate automated pipelines.Customers who adopt this threat-informed-defence methodology typically realise measurable gains within one quarter, including a reduction in false positives as redundant or missing telemetry is corrected, and faster incident triage due to richer context in each alert. Threat-informed-defence will also set your SOC up for success come audit time, through a maintained ATT&CK-aligned evidence trail.Are You Ready?AI agents offer genuine value in security operations, but they cannot transcend fundamental telemetry limitations. Threat-informed defence, anchored by current, high-fidelity CTI, remains the most efficient path to ensuring that the “right logs with the right fields” reach your SIEM. Only when that foundation is secure can AI reliably assume analytic tasks and allow your human teams to focus on higher order tasks.If you would like to review a complimentary, sector-specific ATT&CK coverage report, or to explore how Arachne Digital can integrate continuous intelligence directly into your log engineering workflows, contact us at [email protected] Logs for Smarter SOCs: Threat-Informed Telemetry That Powers AI Agents and Cuts Costs was originally published in MeetCyber on Medium, where people are continuing the conversation by highlighting and responding to this story.
Analysis Summary
The article content could not be fully retrieved due to a security challenge. Based on the provided context (the description of the article), here are the security recommendations structured as requested, focusing on **Threat-Informed Telemetry and SIEM Optimization**:
# Best Practices: Threat-Informed Telemetry for Effective SIEM and AI Utilization
## Overview
These practices focus on moving away from indiscriminate log collection ("ingest everything") toward a **Threat-Informed Defence** strategy. This approach ensures that the security monitoring infrastructure (including AI agents in SIEM platforms) receives the *exact* necessary telemetry—specific logs and fields—required to detect the Tactics, Techniques, and Procedures (TTPs) of relevant cyber threat actors (CTAs), leading to optimized costs and higher detection fidelity.
## Key Recommendations
### Immediate Actions
1. **Identify Relevant Adversaries:** Use curated Cyber Threat Intelligence (CTI) specific to your industry and geography to formally identify the top targeted CTAs relevant to your organization.
2. **Initial TTP Mapping:** Map the identified high-priority adversaries' current activities directly to MITRE ATT&CK techniques.
3. **Inventory Critical Gaps:** For the top 3-5 most frequent TTPs identified, immediately validate whether the required event types and specific log fields are present in your SIEM/Log Aggregation pipeline. Mark deficiencies clearly (e.g., using a Red/Yellow/Green matrix).
### Short-term Improvements (1-3 months)
1. **Establish Log-Coverage Matrix:** Formalize the Threat-Informed Defence matrix, linking specific ATT&CK techniques to required log sources (Event IDs) and the *critical data components (fields)* needed for analysis.
2. **Prioritize Gap Remediation:** Prioritize remediation efforts based on the matrix: focus on closing gaps for high-impact techniques executed by priority adversaries. Prioritize low-effort fixes over complex engineering changes initially.
3. **Implement Configuration Baseline:** For all critical log sources, establish and document a configuration-management baseline detailing the specific policy settings, event IDs, and channel configurations required to ensure all necessary fields are collected (e.g., enabling verbose logging where necessary).
### Long-term Strategy (3+ months)
1. **Automate Compliance Validation:** Implement automated checks (using tools like PowerShell or Ansible) against the configuration baseline to continuously verify that required logging sources and fields remain correctly enabled across the environment.
2. **Optimize Telemetry Spend:** Use the log-coverage matrix alongside SIEM ingestion metrics to calculate the business/detection value versus the cost of each log source. Systematically prune, sample, or shorten retention periods for "low value / high cost" logs.
3. **Integrate Continuous CTI Updates:** Establish a repeatable process to continually update the Threat-Informed Defence matrix (e.g., quarterly or upon major threat intelligence releases) to ensure alignment with evolving CTAs and TTPs.
4. **Validate AI Effectiveness:** Once telemetry quality is verified and optimized, deploy or fine-tune AI agents, using the clean, context-rich data to maximize the value of automated investigation and triage.
## Implementation Guidance
### For Small Organizations
- Focus initial CTI efforts on free/public sector-specific reports combined with internal knowledge of recent incidents.
- Prioritize remediation for the immediate collection of command-line processes and basic network flow data, as these cover numerous initial access and execution techniques.
- Leverage existing configuration management tools for basic validation checks rather than investing in new validation platforms.
### For Medium Organizations
- Fully implement the four-step Threat-Informed Defence method to create a robust, evidence-grounded threat model.
- Use the cost-optimization step aggressively to justify budget shifts: redirect funds saved from deprecating non-essential logs toward onboarding high-value, necessary telemetry (e.g., detailed EDR fields or SaaS application audit logs).
- Begin automating the validation process via scriptable checks integrated into change control workflows.
### For Large Enterprises
- Integrate sector-specific CTI feeds directly into the log engineering and ticket creation workflow to maintain the matrix evergreen automatically.
- Conduct rigorous cost/value analysis to tier log retention policies (e.g., hot vs. chilled storage) based strictly on regulatory need and detection usefulness.
- Utilize configuration-as-code practices (Ansible, Terraform) to enforce the logging configuration baseline across complex, decentralized infrastructure.
## Configuration Examples
*Note: Specific technical configuration details were not exhaustively provided, but the requirement is clear:*
| ATT&CK Technique | Required Data Source Example | Critical Field Example | Remediation Goal |
| :--- | :--- | :--- | :--- |
| T1059.001 (PowerShell) | Windows Event ID 4688 (Process Creation) + EDR Telemetry | Parent Process Name, Full Command Line Arguments | Ensure verbose command-line logging is enabled and not suppressed. |
| T1105 (Ingress Tool Transfer) | File and Network Traffic logs | File Hash, Destination IP/URL, File Creation Event | Verify network logs capture full content/metadata for file transfers. |
| T1555.003 (Credential Theft) | File and Process logs | File Access path, OS API Calls related to credential harvesting | Ensure process monitoring captures necessary low-level API interactions. |
## Compliance Alignment
- **MITRE ATT&CK Framework:** Central organizing standard for mapping TTPs to required detection data sources.
- **Regulatory Requirements:** Right-sizing log retention policies should be based on balancing compliance mandates (e.g., 180-day vs. 365-day storage requirements) against investigative utility.
- **General Security Posture:** Improving telemetry fidelity directly improves the effectiveness of controls required by frameworks like **NIST CSF** (Identify and Detect functions) and **ISO 27001** (A.12/A.13 controls).
## Common Pitfalls to Avoid
1. **"Ingest Everything" Mentality:** This leads to unsustainable storage costs, data poisoning (noise), and prevents focus on high-value evidence.
2. **Ignoring Field Quality:** Assuming that because a log source is collected (e.g., Event ID 4688), all necessary fields (like full command-line arguments) are present. AI cannot compensate for missing context within collected events.
3. **Static Threat Modeling:** Fixing telemetry only once. Threat landscapes, TTPs, and organizational tooling constantly change, invalidating the log matrix without continuous updates.
4. **Over-Reliance on AI Alone:** Deploying AI agents before validating the underlying telemetry quality. Incomplete data will cause the AI to produce inaccurate results or increase analyst workload chasing false negatives/context deficient alerts.
## Resources
- **MITRE ATT&CK Framework:** For mapping adversary behaviors to required data sources.
- **Curated CTI Services (e.g., mention of Arachne Digital feeds):** Essential for receiving machine-readable, up-to-date adversary activity mapped to ATT&CK.
- **Configuration Management Tools (PowerShell, Ansible):** Necessary for automating the validation of log source policies against the established baseline.