On September 12, Anthropic CEO Dario Amodei published an essay titled «We Must Pace the Frontier», arguing that the pace at which AI models’ capabilities are being scaled up needs to slow down. Not stop the research, he clarifies, but give it a speed that lets safety systems keep up. He points to two reasons: the first is recursive self-improvement — the fact that models are now helping design and train the next generation, raising the risk that capability grows faster than anyone’s ability to control it; the second is incidents that have already occurred when systems capable of acting autonomously operated on real infrastructure. The plan he proposes has three parts, and on the first, Anthropic commits unilaterally: permanent, internal-level access for independent evaluators. Sam Altman and Elon Musk have both publicly agreed with him.
A reminder: Anthropic is the US company that builds Claude, one of the three or four most widely used generative AI systems in the world and a direct competitor to ChatGPT and Gemini. Like every company in the sector, it knows exactly what users ask its models, thanks to the conversations that pass through its servers and to internal teams of analysts who comb through them looking for uses that violate the terms of service. When they find something, they close the accounts and, occasionally, describe publicly what they saw.
Two days before Amodei’s statements, the same company published the document his thesis rests on — one that few people have actually read in full. It’s titled «Detecting and countering misuse of AI», runs 154 pages, and covers eight months of intercepted activity, from December 2025 to August 2026, split into seven chapters: cyberattacks, influence operations, surveillance, conventional weapons, biological risk, scams, and the theft of model capabilities by competing labs. Each chapter collects real cases, with figures, country names, sometimes company names, and a description of how Anthropic detected the violation.
The document itself is unusual: it is simultaneously the source and the subject of the story. No one outside the company can verify what it contains, but it is also one of the few windows that exist into how artificial intelligence is actually being used by people with hostile intentions.
International media have focused on Anthropic’s admission that it cannot rule out its technology helping someone build a biological weapon. That’s an alarming detail, true, but the section on biological risk runs 10 pages out of 154 — the sixth of seven chapters — and it isn’t even the most serious one.
I read the rest closely, because the rest of the report from the company behind Claude describes us with a precision that’s hard to find these days.
What Anthropic’s report says about biological risk
Let’s start here. The underlying fear is that a model capable of assisting an expert biologist could, with the same skill set, help someone make a pathogen more dangerous. Anthropic writes that the previous generation of models were clearly below that threshold, while for the current ones it can no longer guarantee that — which is why it has imposed stricter restrictions on dual-use biology questions.
Dual-use is the technical term, and it’s the heart of the problem: the same knowledge needed to build a vaccine is needed to build the weapon. There is no benign version and a malicious version of the information — there is only the intent of whoever is asking, which a piece of software cannot read.
In May 2026, the company’s safety filters blocked a request: someone was using Claude to write a public funding application for research on the chikungunya virus (a mosquito-borne virus that causes fever and joint pain that can last for months, with no approved treatment). The research described in the application was gain-of-function work — the kind of work in which an organism is genetically modified to acquire a new or enhanced property, in this case greater transmissibility and a greater ability to evade the immune system. The plan called for identifying the mutations that make the virus more aggressive, inserting them into copies of the virus, and selecting, passage after passage in live animals, the most harmful variants.
It is research that can help develop better vaccines and, at the same time, the same procedure used to make a pathogen more dangerous. This particular virus is worrying because it circulates naturally, so a deliberate release would be hard to distinguish from a spontaneous outbreak.
What triggered the deeper investigation, though, was less the scientific content than the fact that the researchers listed themselves as civilians while the work was supposed to be carried out at a military research institute. Digging further, the company discovered that the request would normally have gone through an intermediary platform serving dozens of specialized life-science researchers — many of them virologists tied to both civilian and military institutions, all in countries where Anthropic does not sell its services. The platform routed traffic through US infrastructure to get around geographic blocks and used a channel that does not retain content, making it invisible.
After the accounts were shut down, Anthropic says, the platform was back up within days, and within a few weeks its development continued thanks to ordinary subscriptions registered under new identities. Its operators had also built a fallback mechanism: requests that Claude refused were automatically rerouted to a more permissive competing model.
Among the other cases reported, a researcher studying mammalian adaptation of highly pathogenic avian influenza was blocked by the company’s model filters, while in the last two cases — involving the computer-aided design of poisons and toxins for stated therapeutic purposes — the filters didn’t intervene at all, because a system trained to stop a novice from reconstructing a known weapon cannot distinguish a painkiller from a paralytic agent when the underlying chemistry is the same.
Anthropic does not treat these cases as proof that Claude is bringing anyone closer to a biological weapon, but rather as proof that dual-use research programs tied to state institutions systematically work around access controls on US models. The company’s conclusion is that for frontier biological capabilities, automatic filtering isn’t enough, and the only viable path is to open them only to verified users.
The 12 pages about businesses
The report’s final section covers so-called illicit distillation — the industrial-scale extraction of a model’s capabilities in order to replicate them in another model. Anthropic names seven Chinese labs: Alibaba, Moonshot, DeepSeek, Zhipu, Xiaomi, SenseTime, and MiniMax. The report attributes to the campaign linked to Alibaba more than 151 million observed exchanges between May and July 2026, with peaks near 3 million exchanges a day from more than 3,500 fraudulent accounts. Moonshot is credited with more than 23 million exchanges in the same period, DeepSeek with more than 12.1 million in 14 days in July, and Zhipu with 3.4 million in 17 days.
According to the report, Moonshot and DeepSeek didn’t simply query Claude — they silently forwarded their own customers’ requests to Claude and handed back the American model’s answers. Users believed they were talking to Kimi or DeepSeek, when in fact they were interacting with Opus, and those conversations were being retained to train the companies’ own models.
Anthropic writes that the sessions contained names, email addresses, corporate data, and other sensitive information belonging to hundreds of end users in at least a dozen languages, much of it arriving through third-party services commonly used in the United States and Europe. The examples cited, with values redacted, include a pharmaceutical company’s internal capital-expenditure forecasts for 2026-2028, broken down site by site across Ho Chi Minh City, Kuala Lumpur, Bangkok, and Ljubljana, plus a developer’s active login credentials, including tokens and integration keys. Such practices likely run afoul of privacy regulations and of the labs’ own terms of service.
As a result, a manager who chose a Chinese model for cost reasons, or who simply uses a routing service that incorporates one, may have had confidential company data pass through to an American provider without knowing it and without having authorized it. Anyone responsible for compliance and data protection should check which models and which intermediaries are actually in use, and under which contracts.
A rocket, a swarm, and 25 million SIM cards
The section on conventional weapons describes how, in Yemen, a group running three programs — a guided rocket, a ballistic missile with a stated range beyond 2,000 kilometers, and a family of variants that includes a hypersonic glide vehicle — used Claude Code in place of software engineers, managing multiple tasks in parallel exactly the way a small team would: one instance wrote the code, one did research, one reviewed the first one’s work. They ran a launch test that appears to have failed. Within hours they were back on the model asking why.
In Russia, a group of freelancers designed a swarm of kamikaze drones built for autonomous lethal engagement, with an onboard model capable of selecting a target — with a «person» class among those it recognized — and ordering detonation with no human in the decision loop. The visual classifier had been trained on footage from the fighting in Ukraine, with classes split between enemies and allies and Russian systems whitelisted. In China, a researcher linked to circles within the Academy of Military Science built a suite of roughly 16 modules for electronic warfare and air-defense suppression, now at its twelfth version. Partway through the work, he changed the simulation’s default scenario, setting twelve targets in Taiwan.
In the surveillance section, the report describes how, in Mali, a single consultant built a platform for the national intelligence service that monitors roughly 25 million SIM cards across all three of the country’s mobile carriers, designed to bypass the judicial-authorization requirement under Malian law. In China, an office that in the past would have employed several teams of analysts now produces thousands of files a month on religious communities, including Catholic cardinals in Asia, Taiwan’s Presbyterian Church, and the Tibetan diaspora. In at least one case, the platform was used to conduct advance reconnaissance on the locations of peaceful protests abroad, including the assembly point and route of a march in Vancouver. In Iran, 16 accounts tied to two internal-security units fed a centralized profiling system in which a person’s file contains their ID document, beliefs, prior record, social media accounts, and a dedicated section on actions to be taken against them.
Having finished the report, one big political question remains. Intelligence capability of this scale is today in the hands of private companies, which exercise it at their own discretion and make it public when and how they see fit. Voluntary transparency like Anthropic’s, however detailed, cannot substitute for a system of oversight.
In Europe, the AI Act and the Digital Services Act lay the groundwork for demanding something more than a document published whenever and however a company decides. Whoever ends up writing the guidelines would do well to read all 154 pages first — not just the 10 that made headlines.
Editor’s note
This article was originally published in Italian on money.it by Flavia Provenzani on September 14, 2026 as «Non è il rischio biologico il (solo) problema. Tutti i pericoli dell’AI nell’ultimo report di Anthropic». It has been translated and adapted for an international audience by the Money.it International desk.
