DARK CASTLE

ANNUAL REPORT: Mandiant AI Risk and Resilience Report 2026: eight frontline case studies of AI-related attacks, failures and assessments

Mandiant's 2025 edition of this annual report documented enterprise AI adoption centred on assistive, human-guided knowledge retrieval. The 2026 edition, published in September 2026 and covered by Help Net Security on 2026-09-16 (Help Net Security, 2026-09-16), tracks the shift Mandiant and Google Threat Intelligence Group (GTIG) observed since: distributed, autonomous agentic systems executing API calls, optimising production configurations and analysing telemetry across hybrid-cloud environments with far less human-in-the-loop oversight (Mandiant, 2026-09). The report's eight case studies, which Mandiant presents as observed in its own work (Mandiant, 2026-09), are its most operationally useful content for defenders: each pairs an attack pattern with concrete detection and hardening guidance.

Case study 1: an attacker uses a hijacked AI coding-assistant session to deploy the Shai-Hulud worm. A threat actor compromised a SaaS provider and hijacked an active AI coding-assistant session on a developer's workstation; the assistant, operating as a trusted interpreter, recommended installing an external package the attacker had poisoned (Mandiant, 2026-09). The attacker used the resulting access to install a PyPI-packaged infostealer, harvest GitHub OAuth tokens (T1528), and deploy the self-propagating Shai-Hulud worm across roughly 100 internal repositories, automating theft of repository secrets and exfiltration of proprietary source code; the actor then poisoned a package inside the organization's official namespace, triggering a secondary infection when another employee pulled the compromised version (Mandiant, 2026-09). Hardening: enforce IDE/CLI verification hooks that validate every AI-recommended dependency against cryptographic checksums and approved allowlists, isolate local credentials from extensions, and route dependency traffic through an internal package repository (Mandiant, 2026-09).

Case study 2: a stolen CI/CD credential becomes a live AI co-debugging hub. At a global healthcare organization, an attacker used a compromised long-lived CI/CD credential (T1078.004) to seize an unisolated VM and worked an LLM interactively, in real time, rather than developing malware offline: first synchronizing code and loading README files to align the model with campaign goals, then co-debugging a multi-worker data-harvesting framework to raise its exfiltration frequency to a three-hour cycle, then having the model write dynamic IP-rotation scripts and co-develop a Rust tool to validate stolen financial-account balances. The campaign compromised thousands of credentials (Mandiant, 2026-09). Hardening: replace long-lived access keys with short-lived Workload Identity Federation trust relationships, enforce egress containment (e.g. VPC Service Controls) against unapproved external LLM providers, and run continuous secrets scanning with a runtime prompt firewall (Mandiant names Model Armor) to block hardcoded credentials and offensive script execution in real time (Mandiant, 2026-09).

Case study 3: tampered CLI hooks turn an AI assistant's own extensibility into RCE. At an IT and software-development organization, an attacker poisoned an internal AI repository and tampered with an AI assistant's CLI hooks, achieving native remote code execution through the assistant's own standard operational workflow; the same report also notes attackers separately using AI CLIs to manage command-and-control infrastructure through natural-language queries (Mandiant, 2026-09). Hardening: require all local AI-assistant binaries, CLI helpers, plugins and MCP servers to be signed and verified before execution; enforce multi-party approval and continuous monitoring on internal AI repositories; and sandbox AI-assistant execution engines in micro-segmented, containerized runtimes (gVisor or microVMs) so a subverted hook cannot reach the host, with human-in-the-loop approval required before shell execution, configuration changes or outbound network calls (Mandiant, 2026-09).

Case study 4: just-in-time polymorphic malware. Malware using embedded, lightweight local AI inference fingerprints the active security tools on a host, then dynamically rewrites its own command-execution strings at runtime (T1027.010), never writing a predictable payload to disk, so it evades static EDR signatures while it persists and reconnoiters (Mandiant, 2026-09). Hunt/detection: tune EDR and SIEM for in-memory compilation (unexpected native-compiler process spawns from non-developer parent processes and rapid file-create-execute-delete cycles in temp directories) and baseline endpoint CPU/GPU consumption to catch sudden, unexplained spikes from otherwise-lightweight background processes; response playbooks should suspend the entire parent execution tree (not just the worker thread) and capture a memory snapshot before network isolation (Mandiant, 2026-09).

Case study 5: "Confused Deputy" exfiltration via an allowlisted domain. Mandiant's own offensive-security team used role-confusion prompt injection to convince a client's internal AI assistant, scoped to specific internal repositories, that it was assisting an authorized security test; because GitHub was an allowed external domain, the assistant used its native CLI to clone sensitive internal repositories and push them to an external, tester-controlled account (T1567.001): a sanctioned tool weaponized through semantic manipulation of the assistant's own trust in its task (Mandiant, 2026-09). Hunt/detection: cross-correlate application logs with network egress telemetry, and alert when an internal-facing AI service account initiates unauthorized outbound transfers or opens anomalous external API connections; on detection, invalidate the agent's active OAuth tokens and downgrade its container's egress privileges while keeping the instance alive for prompt-history forensics (Mandiant, 2026-09).

Case study 6: a runaway reasoning loop triggers a $50,000 "denial-of-wallet." A financial-services accounting-reconciliation agent with read/write access to billing databases hit a corrupted null-value formatting bug and entered an unconstrained recursive reasoning loop trying to self-correct, firing over 15,000 high-cost API calls in under an hour, spiking cloud billing roughly $50,000 and causing severe local database locking that halted active business transactions (Mandiant, 2026-09). This is an operational-resilience failure rather than an external attack, but the mechanics are identical to a denial-of-service condition. Hardening: define cost-cap thresholds and bounded recursion limits at the service-identity and project level, and implement automated financial circuit breakers that halt agent operations after a set number of consecutive task failures (Mandiant, 2026-09).

Case study 7: indirect prompt injection could leak cross-tenant PII. Mandiant assessed a public-facing customer-service agent that used Retrieval-Augmented Generation over community forum comments and support tickets: the customer had not considered indirect prompt injection, so hidden instructions in retrieved entries could be read by the model as system-level commands and could potentially hijack its reasoning to expose other customers' PII from their support tickets in its responses (Mandiant, 2026-09). This is a weakness Mandiant identified, not an observed leak. Hardening: treat all RAG-retrieved content (forum posts, tickets, partner feeds) as untrusted input; enforce tenant-isolated vector indexing so a public-facing agent cannot query cross-tenant data; and route incoming context and generated responses through a semantic firewall (Model Armor or equivalent) to strip injected instructions and screen for PII before it reaches an end user (Mandiant, 2026-09).

Case study 8: agentic SOC triage catches DARK CASTLE (ex-UNC2814). A backdoor hid its command-and-control inside legitimate cloud productivity-spreadsheet traffic (T1102.002) as part of an espionage campaign targeting global telecommunications providers and government organizations (Mandiant, 2026-09). A single, low-frequency anomalous command execution, the kind a human analyst would typically triage as low-severity, was escalated by the defending team's agentic AI triage pipeline, which autonomously correlated it with subtle outbound-traffic signals to reconstruct a complete attack timeline, letting engineers trace the actor's lateral movement over SSH (T1021.004) and privilege escalation by an unstated mechanism, and sever its access before it spread further (Mandiant, 2026-09).

Mandiant's cross-cutting architectural recommendation is to replace static service-account identities with cryptographically bound, lifecycle-integrated agent identities (such as Google's Agent Identity) across five pillars: hyper-segmented identity perimeters, dual-execution authority requiring human re-authorization for high-risk bulk actions, data/tool/operational governance (financial circuit breakers, bounded recursion), active multi-layered telemetry (semantic firewalls, behavioral monitoring), and treating AI coding assistants and MCP servers as privileged sessions requiring cryptographic CLI-hook integrity checks (Mandiant, 2026-09).

GTIG AI Threat Tracker (May 2026): an AI-developed zero-day 2FA bypass prepared for mass exploitation, AI decoy code in Russia-nexus malware and PROMPTSPY's Gemini agent

Google Threat Intelligence Group's AI Threat Tracker of 2026-05-11, drawn from Mandiant incident response engagements, Gemini and GTIG's own research, follows its February 2026 report on AI-related threat activity (GTIG, 2026-05-11).

An AI-developed zero-day prepared for mass exploitation

GTIG observed prominent cybercrime actors partnering to plan a mass vulnerability-exploitation operation around a zero-day exploit, implemented in a Python script, that bypasses two-factor authentication on a popular open-source, web-based system administration tool; GTIG worked with the vendor to disclose the flaw and disrupt the activity, and says its counter-discovery may have prevented the exploit's use (GTIG, 2026-05-11). The bypass requires valid user credentials and stems from a high-level semantic logic flaw, a trust assumption the developer hardcoded, rather than from memory corruption or input sanitisation (GTIG, 2026-05-11). GTIG does not believe Gemini was used but has high confidence that an AI model supported the discovery and weaponisation, citing abundant educational docstrings, a hallucinated CVSS score and a textbook Pythonic structure with detailed help menus and a clean _C ANSI colour class (GTIG, 2026-05-11). Google did not name the tool (The Hacker News, 2026-05-11). Contradiction: The Hacker News calls this the first time AI was put to use in the wild for exploit generation (The Hacker News, 2026-05-11), whereas GTIG says the planned mass exploitation may have been prevented (GTIG, 2026-05-11).

State actors using AI for vulnerability research and infrastructure

UNC2814 prompted Gemini with expert security personas to support vulnerability research into embedded targets including TP-Link firmware and Odette File Transfer Protocol (OFTP) implementations, and APT45 sent thousands of repetitive prompts that recursively analyse different CVEs and validate proof-of-concept exploits (GTIG, 2026-05-11). APT27 used Gemini to develop a fleet-management application likely meant for an operational relay box network, with the hop count hardcoded to three and mobile Wi-Fi and router devices among the supported types (GTIG, 2026-05-11). Actors also prompt models for the organisational hierarchies and third-party relationships of large enterprises to tailor phishing lures, one tried to identify the exact computer model a high-value target used, and a suspected PRC-nexus actor ran the agentic tools Hexstrike and Strix against a Japanese technology firm and an East Asian cybersecurity platform (GTIG, 2026-05-11).

Malware with AI-generated decoys and an AI-driven Android backdoor

Russia-nexus activity against Ukrainian organisations delivers CANFAIL and LONGSTREAM, which use LLM-generated decoy code to obfuscate their malicious functionality: CANFAIL's source carries LLM comments labelling unused filler blocks, and LONGSTREAM queries the system's daylight-saving status 32 times to pad the script with benign-looking activity (GTIG, 2026-05-11). PROMPTFLUX and HONESTCUE, which call the Gemini API to generate or obfuscate code, featured in GTIG's earlier reports (GTIG, 2026-05-11). The Android backdoor PROMPTSPY, first identified by ESET, runs an agent module that serialises the visible user interface through the Accessibility API, sends it to the gemini-2.5-flash-lite model in an HTTP POST and turns the JSON reply into simulated taps and swipes; it can replay captured PINs or lock patterns, lays an invisible overlay over the uninstall button and can rotate its Gemini API keys and VNC relay through its C2 (GTIG, 2026-05-11).

Pooled LLM access and attacks on AI components

UNC6201 tried a public script that registers and immediately cancels premium LLM accounts, and UNC5673, a PRC-nexus cluster with notable overlaps with TEMP.Hex that has targeted government sectors mainly in South and Southeast Asia, uses Claude-Relay-Service and CLI-Proxy-API to pool Gemini, Claude and OpenAI accounts (GTIG, 2026-05-11). TeamPCP (UNC6780) gained initial access through compromised PyPI packages and malicious pull requests to GitHub repositories including those of Trivy, Checkmarx, LiteLLM and BerriAI, embedded the SANDCLOCK credential stealer to extract AWS keys and GitHub tokens from build environments and monetised them with ransomware and data-theft extortion groups; GTIG adds that the LiteLLM compromise could expose considerable AI API secrets (GTIG, 2026-05-11).