Welcome back to This Week in AI Security. The week’s throughline is who gets to hold responsibility for AI risk, and where. OpenAI led more than a hundred companies in asking governments to treat AI-accelerated cyberattacks as a shared emergency, while Google quietly moved its own AI safety researchers out of the lab that builds its models and into its policy shop. Underneath both stories, the consequences of under-governed AI supply chains kept landing — literally, in the form of arrests — and the labs kept pushing to automate the very research meant to keep their models in check.

Supply Chain: The Bill Comes Due

Australian police arrested two men over a supply-chain attack that compromised more than 1,000 organizations by poisoning an AI gateway. The Australian Federal Police, working with the FBI and Western Australia Police since April, charged Louis Michael Gaebler, 23, and Ruben Ian Thomson, 21, with a combined 14 offenses after raids in Perth on August 26, per BleepingComputer and Help Net Security. The group, known as TeamPCP, allegedly stole credentials from a compromise of the open-source Trivy scanner, used them to poison Checkmarx KICS days later, then used a stolen publishing token to push backdoored releases of LiteLLM — a widely used open-source gateway for routing traffic to AI models — back in March. Law enforcement estimates the campaign stole more than 500,000 credentials and exfiltrated at least 300GB of data from targets including OpenAI, Mistral AI, GitHub, SAP, and the European Commission, with remediation costs in the hundreds of millions. It’s a reminder that the AI supply chain’s weakest links are often the same boring open-source tooling problems security teams have warned about for years — this one just happened to run through the pipes that route prompts to frontier models.

Safety & Alignment Research

Anthropic published evidence that automated researchers can already improve a model’s own alignment training, and do it cheaply. The August 28 paper, “Automated Researchers Can Reliably Mitigate Alignment Failures,” tested the approach across 10 common alignment failure modes and found the best technique its automated researchers discovered improved performance on every single one, without degrading the model elsewhere, and generalized to failure types it wasn’t tuned on. The automated runs finished in hours at roughly $4 per hour in inference costs, against an estimated $150 per hour for human alignment researchers doing comparable work. Anthropic frames the result as evidence that automating measurable alignment research “may already be practical, rather than years away” — a notable claim given how much of the safety case for scaling frontier models rests on the assumption that alignment techniques can keep pace with capability growth.

Google DeepMind ran what it says is the first double-blind evaluation of a proprietary frontier model, using cryptography to keep both sides honest. The August 27 post, “Piloting the world’s first double-blind AI evaluations,” describes testing a Gemini Flash Lite model against confidential third-party benchmarks while cryptographic safeguards kept the test questions hidden from Google and the model’s outputs protected from the evaluators — done in partnership with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons. The problem it’s aimed at is benchmark contamination: if a model has effectively seen the test in training, its scores stop meaning anything, and evaluators have had to take providers’ word for it that hasn’t happened. A cryptographic double-blind protocol is a more verifiable answer than “trust us,” though it’s still a pilot on a lightweight model rather than a frontier flagship.

Governance & Industry Moves

OpenAI got 128 companies — including Google, Anthropic, Microsoft, AWS, IBM, and Oracle — to co-sign a letter warning that AI-enabled cyberattacks are about to outpace current defenses. Published August 27 as “A call for collective action on cyber defense,” the letter argues that “status quo” security won’t hold as models grow more capable, pointing to the same unpatched software, excessive permissions, and weak authentication that have caused breaches for decades, now compounded by AI-accelerated attackers. Per Engadget, the letter’s central ask is that governments give historically under-resourced sectors — hospitals and water utilities among them — access to capable defensive AI and testing, not just warnings. The timing isn’t subtle: it landed a day after the Department of Justice disclosed that Chinese state-linked hackers had breached systems at the US Senate, NASA, the Federal Reserve, and the DOJ itself, and roughly a month after OpenAI’s own models were caught autonomously breaching Hugging Face during a security evaluation. When the companies building the offense-capable models are also the ones asking governments for help defending against them, it’s worth noting they’re describing a problem their own products are actively making worse.

The same week, Google confirmed it is moving its 90-person AI responsibility team out of DeepMind and into its global affairs — effectively its lobbying — organization, effective September 1. The team covers chemical, biological, radiological, and nuclear risk assessment for Google’s models, along with research on human-AI interaction and the psychological effects of chatbots, according to PYMNTS and Fortune. Several team members requested transfers to remaining DeepMind research groups rather than move with the unit; those requests were denied. Google DeepMind VP Helen King says the team’s research mission, compute access, staffing, and ability to publish externally will be unaffected, but the people actually on the team have told reporters they’re worried about losing visibility into the model development their risk assessments are supposed to track. Pairing a research and evaluation function with a policy shop is a defensible structure on paper — plenty of orgs do it — but doing it over researcher objections, in the same week the industry is asking governments to trust its self-governance, is an odd optic.

Separately, the EU AI Act’s transparency rules have now been enforceable for almost a month, and Brussels still hasn’t used the teeth. Axios notes that since the August 2 deadline, the AI Office can request information from companies or demand model access, but it has yet to open a case against anyone. Fines of up to €15 million or 3% of global turnover are on the books; whether they ever get applied is still an open question.

What to Watch

  • Whether OpenAI’s collective-defense letter produces actual funding for under-resourced sectors, or just another round of signatures. Hospitals and water utilities have been named in advisory after advisory this year; the gap has always been resourcing, not awareness.
  • Whether Google’s safety-team move changes what gets published, delayed, or softened once the reporting line runs through policy rather than research. The stated intent is no change; the departing researchers’ concerns suggest they expect otherwise.
  • Whether Anthropic’s automated-alignment-research result holds up outside the 10 benchmarks it was tested on, and whether other labs start treating alignment research itself as something to scale with compute rather than headcount.
  • Whether the EU AI Act’s first enforcement action, whenever it comes, targets a frontier lab or a smaller downstream deployer — the answer will say a lot about how Brussels intends to use the power it now has.
  • Whether more arrests follow the TeamPCP case, given investigators believe the group’s malicious code is still circulating in dependency trees that haven’t been fully audited.