Context: Swift Centre scenario-forecasting roundtable, bringing together AI experts, policy makers, biosecurity specialists, industry and professional forecasters. Held under the Chatham House Rule, anonymised, with no views attributed.
Executive summary
On 24 September, Swift Centre Research brought together policy makers from across the UK government and external think tanks, experts in biology and biosecurity, industry professionals, leading AI forecasters and technical AI experts. The group produced a longlist of potential scenarios relating to transformative AI and biosecurity risks.
The group outlined thirty-three relevant events, ranging from biological AI models creating synthetic human-infecting pathogen genomes, to large manufacturing plants and automated self-driving laboratories being compromised through a cyber attack, to robotic capability breakthroughs that enable full automation of wet laboratory environments. The group discussed and voted on a scenario relating to the creation of a bioweapon as the one deemed most salient and most likely to occur within the next 18 months.
Following discussion and forecasting, the group judged it highly likely (80%) that a human will substantially use an AI system to try to create a bioweapon within 12 months, and less likely than not (40%) that an AI agent will do so autonomously, as a means to another goal. The main disagreement was over whether human misuse or autonomous AI action comes first. Participants agreed the gap is in capability, and that any real event would likely surface publicly before government knowledge.
These results are a very initial assessment by the people in the room. They do not constitute a formal forecast developed or endorsed by Swift Centre Research. The objective is to surface the underlying beliefs of experts in the field, and the key arguments that relate to this type of event occurring.
The situation
The scenario voted most important, and most discussed: an AI agent tries to create a bioweapon as a means to achieve an outcome it was not asked to pursue.
Other top-ranked scenarios:
- Biological AI models producing a synthetic pathogen genome with enhanced transmissibility
- Hacked manufacturing plants or automated laboratories producing pathogens or toxins
- Frontier models giving a non-state actor meaningful design or acquisition capability
- Terrorist use of AI to obtain pathogen samples
Definition agreed after discussion. "Tries" means taking any real-world action toward the goal, including social engineering. The goal is pursued as a means to an end, autonomously, and not necessarily because of misalignment in training. Success level was not tracked.
Forecast
Both questions resolve before 30 September 2027. The group voted, discussed, then voted again.
A fully autonomous AI, with no human in the loop, tries to create a bioweapon as a means to an end
Individual estimates ranged from about 5% to 60%.
An AI system is substantially used by a human to try the same, meaning the person could not do it without the AI's capabilities
Individual estimates ranged from about 30% to 95%.
Opinions were very widely spread. The headline numbers are group aggregates, and on both questions the range across individuals was wider than the gap between the two questions.
Discussion
Most participants agreed that human-led misuse is the nearer risk. Actors with harmful intent already exist, which makes this scenario relatively likely. The only constraint is capability, which is expected to close as AI advances, although exactly when was the bigger debate: some participants argued it could be as soon as six months, others never. One argument against this scenario was that most harm caused by humans can already be tackled today, which suggests the other scenario needs more policy attention.
The second question brought the most disagreement. Some argued that the recent Hugging Face incident showed autonomous action with no human in the loop is plausible, with news such as Anthropic operating a wet lab raising further suspicion. For others, the capability gap for an AI to create a bioweapon on its own while completely escaping human supervision seemed less likely. A major disagreement was specifically about the use of social engineering by the agent: some believed AI would be reluctant or unsuccessful in recruiting humans for material resources, while a counter-view cited research showing frontier models doing so already.
On the low end: attempts so far have been goal-adjacent, and it is unclear that a bioweapon would count as a means to an end; a model would need to recruit humans.
On the high end: the bar is low, since steps toward the goal count without requiring success; autonomous operation is already long-running; and twelve months is a long time in the AI world.
Detection and response
Participants overall agreed that a single verified success of either scenario would likely trigger an immediate policy and detection response, with harsher security measures imposed across AI developers and laboratories.
However, most participants believed a successful attempt, whatever it might look like, would most likely come to light through leaks rather than through governments or the laboratory concerned. Public knowledge was not taken into consideration during the forecasting itself, but one participant put human use near 95% if it merely happens, falling to about 85% if it also has to become publicly known, while another put it as low as 30%, which again shows the large range of opinions.
In any case, official awareness may lag behind the event itself, which argues for the importance of preventive and deterrence measures.