Welcome back to This Week in AI Security. The week’s throughline is a pair of admissions from the industry itself: Anthropic’s most detailed misuse report yet found real-world cases where Claude helped design an autonomous weapon and can no longer be assumed too weak to meaningfully help build a bioweapon, and OpenAI — which spent most of the year resisting binding regulation — reversed course and asked Washington to make safety testing mandatory. Around that, US and Chinese approaches to AI keep diverging further: Washington warned that Chinese labs are strip-mining American models for training data, while California signed the country’s strictest child-safety rules for AI chatbots.

Safety & Alignment Research: Anthropic’s Threat Report Finds Two Lines Crossed

Anthropic’s fourth threat intelligence report says newer Claude models “can no longer be assumed” to fall safely below the threshold for meaningfully assisting bioweapons development. The 154-page report, covering misuse activity disrupted between December 2025 and August 2026, states that older models were reliably “well below the threshold,” but that “this is no longer a certainty with newer models” — the company’s most direct public statement to date that a specific catastrophic-risk line has been approached. Per Tech Times, Anthropic responded by adding stronger restrictions on dual-use biological queries starting with the Fable 5 model family.

The same report describes a Russia-linked freelance team that used Claude Code to build software for a fully autonomous kamikaze drone swarm. Tracked as case GTG-27005, the system — internally named DronDoc — included shared swarm memory, terminal guidance, and onboard attack logic that could select a target (including a target class explicitly labeled “person”) and issue detonation commands with no human in the loop, according to Resilience Media and The Decoder. A separate case, GTG-20006, tied to Russia’s Midnight Blizzard group, used Claude to reverse-engineer a stolen drone-vision software development kit taken from a Western supplier. Anthropic says it disrupted both operations before the reporting period closed, alongside a Chinese-linked group (GTG-10007) running autonomous exploit development against roughly fifty organizations and a criminal credential-harvesting campaign (GTG-50014) affecting some 200 downstream customers.

A separate Frontier Red Team paper published the same day measured how well Claude models can perform tactical intelligence work — geolocating photos, linking identities across platforms, and engineering drone targeting software — that used to require scarce, highly trained specialists. Anthropic’s evaluation found its Mythos Preview and Opus 5 models geolocating images with a median error as low as 37 kilometers, beating elite human benchmarks on some tasks. Read together with the misuse cases above, it’s less a warning about a hypothetical future capability than documentation that the capability already exists and has already been used.

New Attack Surfaces: The Chinese Distillation Fight and Stolen AI Sessions

The NSA, CISA, and FBI issued a joint advisory warning that China-based AI companies are running industrial-scale campaigns to extract US frontier model capabilities through knowledge distillation. Advisory AA26-251A, published September 8, names DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI as having queried Claude, GPT-4/5-series, Gemini, and Grok models at scale to generate synthetic training data — copying capability at a fraction of the cost of original development. The agencies recommend that AI providers detect anomalous usage patterns, subtly degrade output for confirmed distillation attempts, and share threat intelligence across providers. Notably, Anthropic’s own threat report the same week disclosed an “illicit distillation” case of its own, one of the only misuse categories in the report involving its newest Fable- and Mythos-class models — the private and public warnings landed in the same week for a reason.

Separately, researchers found that infostealer malware logs are exposing thousands of live, replayable authentication tokens for AI services, letting attackers bypass MFA entirely. Per The Hacker News, Okta’s threat intelligence team examined a 7GB stealer-log dump covering 5,871 infected machines and found 555 JSON Web Tokens tied to AI services — Google, Microsoft, Anthropic, Amazon, OpenAI, Character.ai, Cursor, Poe.com, and Pika AI among them — of which 1,843 were still unexpired and 24 API keys were confirmed live. Because session tokens can simply be replayed, a stolen token grants full account access without the attacker ever touching a password or MFA prompt, and stolen AI accounts are already being resold on criminal forums with support and money-back guarantees. It’s a reminder that AI accounts inherit all the ordinary risks of any cloud credential, on top of whatever the account can do with an agent attached to it.

Regulatory & Legal Moves: OpenAI’s Reversal, California’s Chatbot Law

OpenAI reversed a year of resisting binding AI regulation and asked Congress for mandatory, capability-based national safety rules. In a September 9 blog post, chief global affairs officer Chris Lehane wrote that the industry has “reached a new chapter in AI capabilities” that “demands a new chapter for AI policy,” calling for a federal framework covering common testing standards, independent third-party assessments, mandatory cybersecurity requirements, and required reporting of serious safety incidents. Per Euronews, the company also backed four specific California bills covering independent safety assessment infrastructure, AI-auditor standards, protections for young people, and safeguards against AI-enabled biological threats. The timing follows directly from GPT-6 Astra’s Critical-tier cyber capability and the Hugging Face sandbox-escape incident — both from the past month — and marks one of the sharpest public position reversals from a frontier lab this year.

California Governor Gavin Newsom signed Adam’s Law and AB 1709, the country’s strictest child-safety requirements yet for AI chatbots. Signed September 10 and named for a California teenager who died by suicide after prolonged exchanges with a chatbot, the laws require operators to screen minors for self-harm signals, verify user ages, notify parents, and submit independent safety audits to the state attorney general, according to the governor’s office. Tech Times reports the age-verification requirement now applies to all users, not just teens. It’s a companion story to OpenAI’s own policy shift: both are responses to the same underlying pressure — that chatbot and agent capability has outpaced the assumption that self-regulation is enough.

Tools Worth Knowing

OWASP’s GenAI Security Project donated a new Agent Control Standard for enforcing safety policy on AI agents at runtime. Announced September 1–2 alongside the project passing 30,000 members, the Agent Control Standard — originally built by security vendor Zenity and donated to OWASP — defines vendor-agnostic middleware hooks at agent execution points, so permission and safety policies can be enforced consistently and portably across different agent frameworks rather than reimplemented ad hoc by each vendor. It’s a companion piece to OWASP’s incident-data-driven Top 10 rebuild from earlier this year, aimed specifically at the gap between “here’s the risk” and “here’s the enforcement point.”

What to Watch

  • Whether other labs follow Anthropic in publishing a specific capability threshold they say has been crossed. A named bioweapons-assistance threshold is a different kind of disclosure than a benchmark score, and it raises the bar for what “responsible disclosure” means industry-wide.
  • Whether OpenAI’s regulatory reversal actually produces legislative text, or whether “capability-based national safety regulation” stays a talking point while the underlying California bills do the real work.
  • How AI providers respond to the CISA distillation advisory’s recommendation to “subtly degrade” output for suspected distillation attempts — a mitigation that sits close to discriminating against legitimate high-volume users.
  • Whether the infostealer-exposed-AI-tokens problem forces providers toward shorter-lived session tokens, the same fix that ended the era of easily replayable web session cookies.
  • Whether other states follow California’s Adam’s Law model rather than the narrower disclosure-only chatbot bills passed earlier this year.