On September 12, Anthropic CEO Dario Amodei published an essay titled «We Must Pace the Frontier», arguing that the pace at which AI model capabilities are being scaled up needs to slow down. Not stop the research, he clarifies, but give it a speed that lets safety systems keep up. He points to two reasons: the first is recursive self-improvement, meaning that models now help design and train the next generation, raising the possibility that capability could grow faster than anyone’s ability to control it; the second is incidents that have already happened when systems capable of acting autonomously operated on real infrastructure. The plan he proposes has three parts, and on the first one Anthropic commits unilaterally: permanent, internal-level access for independent evaluators. Sam Altman and Elon Musk both publicly agreed with him.
Anthropic is the US company that develops Claude, one of the three or four most widely used generative AI systems in the world, a direct competitor to ChatGPT and Gemini. Like every company in the sector, it knows exactly what users ask its models, thanks to the conversations that pass through its servers and the internal teams of analysts who comb through them looking for uses that violate its terms of service. When they find something, they close the accounts and, occasionally, describe publicly what they saw.
Two days before Amodei’s statements, the company published the document his argument rests on — one that few people have read in full. It’s titled «Detecting and countering misuse of AI», runs 154 pages, and covers eight months of intercepted activity, from December 2025 to August 2026, divided into seven chapters: cyberattacks, influence operations, surveillance, conventional weapons, biological risk, scams, and the theft of model capabilities by competing labs. Each chapter collects real cases, with figures, country names, sometimes company names, and a description of how Anthropic detected the violation.
The document itself is unusual, being simultaneously the source and the subject of the story. No one outside the company can verify its contents, but it is also one of the few windows that exist into how artificial intelligence is actually being used by people with hostile intentions.
International media focused on Anthropic’s admission that it cannot rule out its technology helping someone build a biological weapon. That’s an alarming detail, to be sure, but the section on biological risk runs 10 pages out of 154, is the sixth of seven chapters, and isn’t even the most serious one.
I read the rest of it carefully, because the rest of the report from the company behind Claude describes us with a level of precision that’s hard to find anywhere else today.
What Anthropic’s document says about biological risk
Let’s start here. The underlying fear is that a model capable of assisting an expert biologist can, with the same expertise, help someone make a pathogen more dangerous. Anthropic writes that the previous generation of models were clearly below that threshold, while for current ones it can no longer make that guarantee — which is why it has imposed stricter restrictions on dual-use biology questions.
«Dual use» is the technical term, and it’s also the heart of the problem, because the same knowledge needed to build a vaccine can be used to build a weapon. There is no benign version and a malicious version of the information; there is only the intent of whoever is asking, and software can’t read intent.
In May 2026, the company’s safety filters blocked a request: someone was using Claude to write a public funding application for research on the chikungunya virus. Chikungunya is mosquito-borne, causes fever and joint pain that can last for months, and has no approved treatment. The research described in the application was gain-of-function work — that is, the kind of research in which an organism is genetically modified to acquire a new or enhanced property (in this case, greater transmissibility and a greater ability to evade the immune system). The plan called for identifying the mutations that make the virus more aggressive, inserting them into copies of the virus, and selecting, passage by passage in live animals, the most harmful variants in a lab setting.
It’s research that could help develop better vaccines and, at the same time, is exactly the process by which a pathogen is made more dangerous. This particular virus is especially concerning because it circulates naturally, making a deliberate release hard to distinguish from a spontaneous outbreak.
What triggered the deeper investigation, though, wasn’t the scientific content so much as the fact that the researchers listed were civilians, even though the work was meant to be carried out at a military research institute. Digging further, the company discovered that the request would normally have gone through an intermediary platform serving dozens of specialized life-science researchers, many of them virologists tied to both civilian and military institutions, all in countries where Anthropic does not sell its services. The platform routed its traffic through US infrastructure to get around geographic blocks and used a channel that doesn’t retain content, making the traffic invisible.
After the accounts were shut down, Anthropic reports, the platform was back up within days, and within a few weeks its development continued thanks to ordinary subscriptions registered under new identities. Its operators had also built a fallback mechanism: requests that Claude refused were automatically rerouted to a more permissive competitor’s model.
Among the other cases reported, a researcher studying mammalian adaptation of highly pathogenic avian influenza was blocked by the company’s filters, while in the last two cases described — the computer-aided design of poisons and toxins with stated therapeutic goals — the filters didn’t intervene at all, because a system trained to stop a novice from reconstructing a known weapon can’t distinguish a painkiller from a paralytic agent when the underlying chemistry is the same.
Anthropic doesn’t treat these cases as proof that Claude is bringing someone closer to a bioweapon, but rather as evidence that dual-use research programs tied to state institutions are systematically working around access controls on US models. The company’s conclusion is that for frontier-level biological capabilities, automated filtering isn’t enough, and that the only workable path is to restrict access to verified users only.
The 12 pages about corporations
The report’s final section covers what it calls illicit distillation — the industrial-scale extraction of a model’s capabilities in order to replicate them in another model. Anthropic names seven Chinese labs: Alibaba, Moonshot, DeepSeek, Zhipu, Xiaomi, SenseTime, and MiniMax. The campaign attributed to Alibaba is credited with more than 151 million observed exchanges between May and July 2026, with peaks close to 3 million exchanges a day from more than 3,500 fraudulent accounts. Moonshot accounts for more than 23 million exchanges in the same period, DeepSeek for more than 12.1 million over 14 days in July, and Zhipu for 3.4 million over 17 days.
According to the report, Moonshot and DeepSeek didn’t just query Claude — they silently forwarded their own customers’ requests to Claude and returned the American model’s answers as their own. Users believed they were talking to Kimi or to DeepSeek, when in reality they were interacting with Claude Opus, and those conversations were being kept to train the companies’ own models.
Anthropic writes that the sessions contained names, email addresses, corporate data, and other sensitive information belonging to hundreds of end users in at least a dozen languages, much of it coming from third-party services commonly used in the United States and Europe. The examples cited, with values redacted, include a pharmaceutical company’s internal capital-expenditure forecasts for 2026–2028, broken down site by site across Ho Chi Minh City, Kuala Lumpur, Bangkok, and Ljubljana, as well as a developer’s active login credentials, including tokens and integration keys. Such practices are likely incompatible with privacy regulations and with the labs’ own terms of service.
As a result, a manager who chose a Chinese model for cost reasons, or who simply uses a routing service that incorporates one, could have had confidential company data pass through to an American provider without knowing it and without having authorized it. Anyone responsible for compliance and data protection would do well to check exactly which models and which intermediaries are actually in use, and under which contracts.
A rocket, a drone swarm, and 25 million SIM cards
The section on conventional weapons describes how, in Yemen, a group running three programs — a guided rocket, a ballistic missile with a stated range beyond 2,000 kilometers, and a family of variants that includes a hypersonic glide vehicle — used Claude Code in place of software engineers, managing multiple tasks in parallel exactly the way a small team would: one instance wrote code, one did research, one reviewed the first one’s work. They ran a launch test that appears to have failed. Within a few hours, they were back on the model asking why.
In Russia, a group of freelancers designed a swarm of kamikaze drones intended for autonomous lethal engagement, with an onboard model capable of selecting targets — including a “person” class among the categories it recognized — and ordering detonation with no human in the decision loop. The visual classifier had been trained on Ukrainian combat footage, with classes split between “enemy” and “friendly” and Russian systems whitelisted. In China, a researcher linked to circles within the Academy of Military Science built a roughly 16-module suite for electronic warfare and suppression of air defenses, which had reached its twelfth version. Partway through the work, he changed the simulation’s default scenario, setting twelve targets in Taiwan.
The surveillance section reports that, in Mali, a single contractor built a platform for the national intelligence service that monitors roughly 25 million SIM cards across all three of the country’s mobile carriers, designed to bypass the judicial-authorization requirement under Malian law. In China, an office that in the past would have employed several teams of analysts now produces thousands of files a month on religious communities, including Catholic cardinals in Asia, the Presbyterian Church of Taiwan, and the Tibetan diaspora. In at least one case, advance reconnaissance was carried out on the locations of peaceful protests abroad, including the assembly point and march route for a demonstration in Vancouver. In Iran, 16 accounts linked to two internal security units fed a centralized profiling system in which an individual’s file includes identity documents, beliefs, prior record, social media accounts, and a dedicated section on what action to take against them.
Having finished the report, one large political question remains. Intelligence capability on this scale currently sits in the hands of private companies, which exercise it at their own discretion and make it public when and how they see fit. Voluntary transparency of the kind Anthropic is offering, however detailed, cannot substitute for a system of oversight.
In Europe, the AI Act and the Digital Services Act are building the framework to demand something more than a report published on a company’s own terms and schedule. Whoever ends up writing the guidelines would do well to read all 154 pages first — not just the 10 that made headlines.
Editor’s note
This article was originally published in Italian on money.it by Flavia Provenzani on September 14, 2026 as «Non è il rischio biologico il (solo) problema. Tutti i pericoli dell’AI nell’ultimo report di Anthropic». It has been translated and adapted for an international audience by the Money.it International desk.
