Agentic AI,
AI-Driven Security Operations,
Artificial Intelligence & Machine Learning
Opaque Decision-Making, Lack of Guardrails, and Poor Auditability Pose Security Risks
The aspiration to replace overburdened Security Operations Center (SOC) analysts with fully autonomous AI agents remains a topic of ongoing exploration, yet it reveals more challenges than solutions at this stage.
See Also: On Demand | Global Incident Response Report 2025
Agentic AI refers to systems capable of independent reasoning, decision-making, and action execution across multiple tasks—moving beyond traditional automation towards functionality resembling that of autonomous co-workers. As cybersecurity leaders consider integrating these tools within their operations, they find the intended relief from workload more often results in merely redefining the burden rather than alleviating it.
Allie Mellen, a principal analyst at Forrester, emphasizes the current inadequacy of agentic AI across all sectors, particularly in security: “If an AI agent misjudges an alert, it risks missing a critical attack. Increased misjudgments create mental strain on human analysts as they are left to sift through what may essentially become a new form of false positives.”
The expectation that agentic AI could effectively minimize workload is often met with a reality check by industry professionals. Chad Cragle, Chief Information Security Officer at Deepwatch, notes that the essence of security measures remains unchanged, requiring security teams to verify each decision made by the AI agent. “It’s a transition rather than a reduction in workload,” he explained, highlighting how the introduction of AI in this context creates layers of complexity rather than eliminating the necessity for human discernment in decision-making processes.
This addition of complexity involves new risks associated with large language models, which many of these systems employ. According to Ophir Dror, co-founder of the security startup Lasso, these models excel in generating responses that can sound convincing even when they are based on faulty premises. They act on predicting what appears ‘correct,’ rather than performing actual fact checks.
He cites a recent study involving GitHub Copilot, revealing that nearly 20% of software libraries suggested by AI were fabricated by the model itself. In cybersecurity contexts, these ‘hallucinations’ may lead to failure in detecting threats or result in inaccurate indicators of compromise and erroneous remediation protocols. Continuous inaccuracies can lead to blind spots within detection frameworks.
The deployment of agentic AI systems without a comprehensive understanding of their design risks is also alarming. Dror warns that many enterprises introduce these agents without clear parameters surrounding their memory, autonomy, and accessibility. Instances have been noted where agents, unregulated and quietly integrated into systems or applications, become vulnerable to memory poisoning, allowing adversaries to inject misleading data that persists and influences the agent’s future actions.
Dror advocates for the most cautious deployment of these autonomous agents in SOCs, suggesting they function best with read-only access. “Assistants represent a less risky integration,” he contended, as the risks escalate with the agents’ potential write permissions in production environments.
The consensus among experts supports the notion that the most responsible application of AI agents in SOCs today is as triage tools, promoting relevant issues and presenting analysts with concise summaries or suggested actions.
Ultimately, the trust placed in agentic AI systems hinges significantly on their deployment strategies and the safeguards in place to oversee their operations. The non-deterministic characteristics inherent in large language models complicate efforts to maintain reproducibility and transparency, both crucial for auditing and testing within security operations.
There remains a significant gap in the security frameworks required to oversee these technologies. Dror noted that many systems are activated without essential protocols such as encryption, identity validation, or robust logging—safeguards that would be obligatory for any human-operated function.
Industry professionals caution against underestimating the potential liability and compliance issues that arise when AI systems operate autonomously. Misconceptions circulate, particularly the belief that agentic AI might fully replace human analysts, as Cragle pointed out. Additionally, misleading performance expectations can cloud assessments of an AI system’s efficacy; potentially, the training data or the approach taken may demand reevaluation before dismissing the AI itself.
