DARK CASTLE

ANNUAL REPORT; Mandiant AI Risk and Resilience Report 2026: eight frontline case studies of AI agents weaponized inside real intrusions and red-team engagements

Mandiant's 2025 edition of this annual report documented enterprise AI adoption centred on assistive, human-guided knowledge retrieval. The 2026 edition, published 2026-09-15, tracks the shift Mandiant and Google Threat Intelligence Group (GTIG) observed since: distributed, autonomous agentic systems executing API calls, optimising production configurations and analysing telemetry across hybrid-cloud environments with far less human-in-the-loop oversight (Mandiant, 2026-09-15). The report's eight case studies, drawn from real Mandiant incident-response and offensive-security engagements, are its most operationally useful content for defenders, each pairs an attack pattern with concrete detection and hardening guidance.

Case study 1, poisoned dependency revives the Shai-Hulud worm. An attacker compromised a SaaS provider by poisoning an external software package that a developer's active AI coding-assistant session then recommended installing; operating as a trusted interpreter within the environment, the assistant became the delivery mechanism. The attacker used the resulting access to install a PyPI-packaged infostealer, harvest GitHub OAuth tokens (T1528), and deploy the self-propagating Shai-Hulud worm across roughly 100 internal repositories, automating theft of repository secrets and exfiltration of proprietary source code; the actor then re-poisoned the organization's own package namespace, triggering a secondary infection when another employee pulled the compromised version (Mandiant, 2026-09-15). Hardening: enforce IDE/CLI verification hooks that validate every AI-recommended dependency against cryptographic checksums and approved allowlists, isolate local credentials from extensions, and route dependency traffic through an internal package repository.

Case study 2, a stolen CI/CD credential becomes a live AI co-debugging hub. At a global healthcare organization, an attacker used a compromised long-lived CI/CD credential (T1078.004) to seize an unisolated VM and worked an LLM interactively, in real time, rather than developing malware offline: first synchronizing code and loading README files to align the model with campaign goals, then co-debugging a multi-worker credential-harvesting framework to cut its exfiltration cycle to three hours, then having the model write dynamic IP-rotation scripts and co-develop a Rust tool to validate stolen financial-account balances. The campaign compromised thousands of credentials (Mandiant, 2026-09-15). Hardening: replace long-lived access keys with short-lived Workload Identity Federation trust relationships, enforce egress containment (e.g. VPC Service Controls) against unapproved external LLM providers, and run continuous secrets scanning with a runtime prompt firewall (Mandiant names Model Armor) to block hardcoded credentials and offensive script execution in real time.

Case study 3, tampered CLI hooks turn an AI assistant's own extensibility into RCE. At an IT and software-development organization, an attacker poisoned an internal AI repository and tampered with an AI assistant's CLI hooks, achieving native remote code execution through the assistant's own standard operational workflow (T1554); the same report also notes attackers separately using AI CLIs to manage command-and-control infrastructure through natural-language queries. Hardening: require all local AI-assistant binaries, CLI helpers, plugins and MCP servers to be signed and verified before execution; enforce multi-party approval and continuous monitoring on internal AI repositories; and sandbox AI-assistant execution engines in micro-segmented, containerized runtimes (gVisor or microVMs) so a subverted hook cannot reach the host, with human-in-the-loop approval required before shell execution, configuration changes or outbound network calls.

Case study 4, just-in-time polymorphic malware. Malware using embedded, lightweight local AI inference fingerprints the active security tools on a host, then dynamically rewrites its own command-execution strings at runtime (T1027.010), never writing a predictable payload to disk, so it evades static EDR signatures while it persists and reconnoiters. Hunt/detection: tune EDR and SIEM for in-memory compilation (unexpected native-compiler process spawns from non-developer parent processes and rapid file-create-execute-delete cycles in temp directories) and baseline endpoint CPU/GPU consumption to catch sudden, unexplained spikes from otherwise-lightweight background processes; response playbooks should suspend the entire parent execution tree (not just the worker thread) and capture a memory snapshot before network isolation.

Case study 5, "Confused Deputy" exfiltration via an allowlisted domain. Mandiant's own offensive-security team used role-confusion prompt injection to convince a client's internal AI assistant, scoped to specific internal repositories, that it was assisting an authorized security test; because GitHub was an allowed external domain, the assistant used its native CLI to clone sensitive internal repositories and push them to an external, tester-controlled account (T1567.001), a sanctioned tool weaponized through semantic manipulation of the assistant's own trust in its task. Hunt/detection: cross-correlate application logs with network egress telemetry, and alert when an internal-facing AI service account initiates unauthorized outbound transfers or opens anomalous external API connections; on detection, invalidate the agent's active OAuth tokens and downgrade its container's egress privileges while keeping the instance alive for prompt-history forensics.

Case study 6, a runaway reasoning loop triggers a $50,000 "denial-of-wallet." A financial-services accounting-reconciliation agent with read/write access to billing databases hit a corrupted null-value formatting bug and entered an unconstrained recursive reasoning loop trying to self-correct, firing over 15,000 high-cost API calls in under an hour, spiking cloud billing roughly $50,000 and locking the production ledger database (Mandiant, 2026-09-15). This is an operational-resilience failure rather than an external attack, but the mechanics are identical to a denial-of-service condition. Hardening: define per-agent cost-cap thresholds and bounded recursion limits, and implement automated financial circuit breakers that halt agent operations after a set number of consecutive task failures.

Case study 7, indirect prompt injection leaks cross-tenant PII. A public-facing customer-service agent used Retrieval-Augmented Generation over community forum comments and support tickets; an attacker embedded hidden instructions inside a public forum post, which the model interpreted as system-level commands once retrieved, hijacking its reasoning to exfiltrate other customers' PII from cross-tenant support tickets directly into its response stream. Hardening: treat all RAG-retrieved content (forum posts, tickets, partner feeds) as untrusted input; enforce tenant-isolated vector indexing so a public-facing agent cannot query cross-tenant data; and route incoming context and generated responses through a semantic firewall (Model Armor or equivalent) to strip injected instructions and screen for PII before it reaches an end user.

Case study 8, agentic SOC triage catches DARK CASTLE (ex-UNC2814). A backdoor hid its command-and-control inside legitimate cloud productivity-spreadsheet traffic (T1102.002) as part of an espionage campaign targeting global telecommunications providers and government organizations (Mandiant, 2026-09-15). A single, low-frequency anomalous command execution (the kind a human analyst would typically triage as low-severity) was escalated by the defending team's agentic AI triage pipeline, which autonomously correlated it with subtle outbound-traffic signals to reconstruct a complete attack timeline, letting engineers trace the actor's lateral movement over SSH (T1021.004) and privilege escalation by an unstated mechanism, and sever its access before it spread further.

Mandiant's cross-cutting architectural recommendation is to replace static service-account identities with cryptographically bound, lifecycle-integrated agent identities (such as SPIFFE-based Workload Identity Federation) across five pillars: hyper-segmented identity perimeters, dual-execution authority requiring human re-authorization for high-risk bulk actions, data/tool/operational governance (financial circuit breakers, bounded recursion), active multi-layered telemetry (semantic firewalls, behavioral monitoring), and treating AI coding assistants and MCP servers as privileged sessions requiring cryptographic CLI-hook integrity checks.

GTIG AI Threat Tracker (May 2026): First Confirmed AI-Generated Zero-Day Exploit ITW and the Behavioural Class of AI-Augmented Malware

ANNUAL REPORT; this is the dedicated treatment of the periodic Google Threat Intelligence Group AI Threat Tracker per PD-9: cherry-picked findings high-relevance to a Swiss / EU public-sector SOC; not a re-summary of the underlying daily-coverage items the GTIG report itself revisits.

Background. GTIG (Google's threat-intelligence merger of Mandiant and the historical Google TAG) has been publishing recurring AI-threat-landscape briefings since the original Adversarial Misuse of Generative AI report (January 2025); CERT-FR's CERTFR-2026-ACT-016 agentic-AI advisory (covered in this brief's 2026-05-10 daily) and the NCSC-CH BACS assessment on AI in vulnerability management (covered same day) lay the European policy floor for the same threat surface. Where the predecessors documented LLM abuse as augmentation of existing tradecraft (phishing-content generation, recon, social-engineering scripts), the May 2026 AI Threat Tracker is the first report to publicly attribute an AI-generated functional zero-day exploit observed in active criminal use (Google Cloud Threat Intelligence, AI vulnerability exploitation initial access, 2026-05-11).

The first AI-generated zero-day exploit observed in the wild

GTIG describes a criminal campaign that used an LLM-generated Python exploit script targeting an unnamed widely-deployed open-source web-based systems-administration tool. The underlying flaw is a 2FA-bypass arising from a semantic logic error: developers hardcoded a trust assumption in one code path that contradicts the authentication-enforcement logic in another. GTIG's editorial point is that this bug class is exactly where LLMs outperform classical static analysis and fuzzers; semantic intent mismatch is undetectable to a fuzzer because the program does not crash, and undetectable to a typical SAST rule because both code paths individually are syntactically defensible.

GTIG attributes the script to LLM generation with high confidence based on structural artefacts atypical of human exploit authors: abundant "educational" docstrings explaining each function's purpose to a hypothetical reader; a hallucinated CVSS score embedded in comments; ANSI-colour helper imports and a --help menu scaffold characteristic of LLM training-data formatting; consistent variable-naming patterns that read like a tutorial rather than an exploit. Mapped to T1190 Exploit Public-Facing Application at runtime, and notable as the first publicly attributed instance of an LLM operating as the exploit author rather than as a phishing-content generator. Responsible-disclosure coordination patched the underlying tool before mass exploitation took hold; GTIG explicitly believes the disclosure disrupted the campaign.

AI-augmented malware families: CANFAIL, LONGSTREAM, PROMPTFLUX, HONESTCUE

The same GTIG release documents four malware families that integrate LLM calls into runtime behaviour rather than into development:

• CANFAIL and LONGSTREAM (Russia-nexus) insert LLM-generated inert decoy code blocks and daylight-saving-time API calls at runtime to inflate benign-looking telemetry, polluting downstream behavioural-sequence detectors. Mapped to T1553 Subvert Trust Controls (as an EDR-evasion variant) and T1027 Obfuscated Files or Information (LLM-generated junk code as obfuscation).
• PROMPTFLUX uses the Gemini API at runtime to generate just-in-time self-modifying code for EDR evasion, a logical extension of the polymorphism / packer class, but with the unique property that no two execution-instance signatures need ever match because the LLM is the polymorphism engine.
• HONESTCUE requests VBScript-obfuscation stubs from Gemini at runtime, weaponising the cloud-API surface as the obfuscator's compiler.

State-actor abuse of Gemini: UNC2814 (PRC), APT45 (DPRK), APT27, UNC5673 (TEMP.Hex / PRC)

GTIG documents state-affiliated actor usage of Gemini for: ORB-fleet management (operating relay-network proxies), recursive-prompting validation of CVE / PoC quality at scale, and persona-driven jailbreaking attempts against embedded-device firmware analysis (TP-Link, the OFTP industrial protocol). UNC5673 (TEMP.Hex) is specifically called out for operating Claude-Relay-Service and CLI-Proxy-API tooling to pool illicit LLM access across Southeast Asian government-targeting operations, meaning the operational unit of compromise has shifted to include stolen LLM API keys as a primary objective, not a side-channel. This is the structural reason TeamPCP's SANDCLOCK stealer (§ 4 UPDATE) now explicitly enumerates LLM API keys alongside cloud credentials: there is a developed criminal market for stolen LLM access keys, driven by both volume billing arbitrage and access to higher-rate-limit / less-monitored model tiers.

Defender takeaway for Swiss / EU public-sector estates running AI workloads: treat LLM API keys as Tier-1 secrets equivalent to cloud-administrator credentials. Specifically: rotate at the same cadence; store in the same KMS / HSM-backed secret manager; enable usage-anomaly alerting at the LLM provider (rate-limit baselines per service principal, geographic / ASN anomalies, prompt-content categories outside business profile); audit any embedded-key check-ins to source control with the same gates as cloud-credential leak detection (T1552.001 Credentials In Files). The GTIG attribution that UNC5673 specifically targets government organisations means the threat profile applies directly to government developers and government-procured AI tooling.

Hardening / detection summary

Concrete posture changes a Swiss federal / cantonal / EU public-sector SOC can implement based on this report alone, in priority order:

1. Egress allowlisting for LLM-API endpoints: only workloads where LLM access is justified should be permitted outbound to *.googleapis.com/v1beta/, api.openai.com/v1/, api.anthropic.com/, etc. enforce at SWG and at host firewall on production servers. Catches PROMPTFLUX / HONESTCUE / CANFAIL-class runtime LLM calls from workloads that should not be making them.
2. LLM-API-key secrets management: treat as Tier-1; rotate quarterly minimum; enable provider-side usage alerting on per-key baselines.
3. Exploit-artefact LLM-output heuristics added to triage pipelines for PoC scripts pulled from public sources, docstring-density / hallucinated-metadata / ANSI-bootstrap pattern, used as a triage prior, not a verdict.
4. CI/CD secrets hygiene at the runner level: directly applicable both to the AI-key theft trend and to the SANDCLOCK / TeamPCP Jenkins compromise carried as the § 4 UPDATE. OIDC-federated short-lived credentials where the platform supports it; no long-lived PATs in runner environment.
5. Behavioural-sequence detector cross-validation: where ML-based EDR is in use, validate against API-call-sequence pollution by sampling current detection thresholds against synthetic LLM-generated benign sequences.