Improving Forecasting in the UK Government

Summary

Despite significant investment, internal government forecasting platforms such as the Intelligence Community Prediction Market in the United States and Cosmic Bazaar in the UK have been wound down or not renewed. These systems were technically and administratively sophisticated, operating with thousands of participants making crowdsourced predictions about a wide range of global events, but were frictious, expensive, and failed to produce information valuable enough to justify their survival. Many forecasts were made, but ultimately they didn't matter.

The Swift Centre was funded by Open Philanthropy to conduct a series of workshops to understand how forecasting might work better in the British government specifically, and to develop models which might apply there and to other governments also.

The workshops were well-received and we learned important lessons; most significantly we developed a governance model for institutional forecasting which we believe explains many of the problems of legacy systems, and propose an innovative bottom-up solution which rebalances roles, cuts costs, and has been enthusiastically received by civil servants who have interacted with it.

Workshops delivered

6sessions across Q1 2025

Participants

107UK government officials

Forecasts generated

43questions generated and responded to

Typical roles

Senior policy strategists, economists, principal analysts, senior data scientists, delivery managers

Introduction

Though significant research has been conducted on improving our ability to accurately forecast the future, there has been less research on how to improve the application of forecasting methods to actually inform decision making in large, influential organisations.

Between 2011 and 2014, the United States' Intelligence Advanced Research Projects Activity (IARPA) ran a tournament to try and identify the most effective methods to forecast geopolitical events. Philip Tetlock and colleagues entered the tournament with their Good Judgement Project (GJP) program. GJP not only outperformed the other research programs, but did so to such a degree that IARPA ended the tournament two years earlier than planned in order to invest solely into GJP's research program. A key GJP finding was that identifying a minority of individuals (so-called "Superforecasters") who can forecast more accurately, grouping them, and providing cognitive-debiasing training, can lead to geopolitical forecasts more accurate than any other known method.¹

However, the attempt to institutionalise these methods in government was not sustained once the research ended. As one research paper which investigated US intelligence crowdsourcing efforts notes, the intelligence community made "negligible use" of the proven forecasting platforms in actual intelligence reports, despite efficacy having been demonstrated extensively.²

The UK Government tried a similar initiative with its Cosmic Bazaar programme, launched by the Professional Head of Intelligence Assessment (PHIA). The initiative acted as an internal forecasting tournament which sourced forecasts from a wide range of public officials, including civil servants, diplomats, police officers, and intelligence analysts across 41 government departments.³ However, the service was sparingly used by many civil servants, with even those working in roles involving horizon scanning often being unaware of its existence. While the ostensible goal of the project was to create forecasts, Cosmic Bazaar's own presentations began to emphasise secondary benefits such as "provid[ing] a meaningful corporate and L&D objective for [the] workforce"⁴ alongside producing actionable information.

Nevertheless, the evidence that forecasting techniques can improve our ability to predict the future and in turn improve high impact decisions makes matching theory to government practice a cause worth pursuing. In response to this gap, the Swift Centre was granted funding from Open Philanthropy to undertake a project to better understand the constraints, challenges, and opportunities for government institutions in embedding cutting-edge forecasting methods into their decision making processes.

The unbalanced "three centres of power"

The Swift Centre identified three centres of power in existing forecasting platforms and processes:

• Decision-makers. The ultimate customers of forecasts. They need probabilistic judgments to balance risks and decide on actions.
• Admins (process and platform operators). Operate the forecasting systems, and in most cases are the gatekeepers of which topics are chosen to forecast. They are motivated to maintain engagement from forecasters and keep the system up and running.
• Forecasters. Provide the probabilities in response to the forecasting questions. In the case of the ICPM and Cosmic Bazaar this was the only role forecasters had. Their motivations are intellectual engagement, competitive score keeping, and altruism.

Each group has its own failure mode which can misalign the system if given too much power.

Decision-makers need probabilistic judgments but may seek to confirm existing hypotheses, struggle to trust probabilistic forecasts, or ignore "unknown unknowns". A forecasting system in which decision-makers generate all forecast ideas is one in which it's harder to anticipate surprise.

Admins are indifferent to the subject being forecasted but suffer delays and frustrations if questions are ambiguous or cannot be resolved easily, especially if a tournament model is being used. If admins are too powerful the system will typically produce very rigorous but often irrelevant questions, very slowly and at considerable expense.⁵

Forecasters may feel social pressure to conform to institutional biases. In many systems, they identify significant risks in the topic area but other than a written comment will have no way to escalate them. They often enjoy forecasting "interesting" topics which engage them intellectually or against which they can test their mettle against the wider pool of forecasters. The colloquial term we began using for this risk is "Chess.com" forecasting: everyone is having a very engaging time competing with each other but the result doesn't matter outside the community.⁶

It's our belief based on conversations with participants that Cosmic Bazaar and other internal platforms have suffered from an imbalance of power favouring admins and forecasters.

In the first two years of its operation, Cosmic Bazaar recruited nearly 2,000 forecasters and solicited 22,800 estimates on 142 questions.⁷ Developing and then answering these questions required a full-time admin staff, large sums spent on software, and two thousand civil servants' time.

Swift's project, conducted over less than eight weeks, used half-day sessions with around 100 civil servants to deliver forecasts on 43 questions, each developed and estimated live in consultation with leaders during the workshop sessions we conducted. The enthusiastic response to this innovative approach and structure enabled faster, more decision-relevant forecasts, which in turn has led to multiple follow-on conversations and requests for assistance in using this approach within their institutions long-term.

The Swift Method

Through this project the Swift Centre tested our approach to rebalance, integrate and rescale the three power structures by:

• Giving decision-makers ultimate control over question selection but allowing all other groups to suggest ideas to them using upvoted crowdsourcing. This balances their judgment about what is relevant against the "unknown unknown" problem.
• Letting admins assist with question improvements but only once the topics and targets have been chosen by decision-makers. This avoids the tendency to choose "hygienic" but irrelevant questions.
• Allowing forecasters to anonymously crowdsource questions to present to decision-makers, protecting their psychological safety, but they can still be overridden.

To implement this process, the Swift Centre followed a unique four-step process through the workshops:

1. The group is prompted using various techniques including the PESTLE framework (political, economic, social, technological, legal and environmental factors) and asked to anonymously submit risks and opportunities their leadership should be thinking about. Leaders can contribute ideas here like everyone else. Everyone then votes on the risks they find most valuable, and the decision-makers choose a topic from the ranked list.
2. Only after the most important risks are identified are questions "cleaned" — dates and metrics added, imprecise language clarified — by the group. This avoids the common problem of selecting a topic because it would make for a "good question" rather than because it delivers actionable information.
3. The question is forecast individually and comments are solicited anonymously. On review of the aggregated forecast and comments, the group is again prompted, this time to identify potential conditional forecasts which may inform options that decision-makers may need to take or advise on.
4. Again the options are ranked by voting and the decision-maker chooses the topics. The relevant risks are cleaned as questions and forecast anonymously.

This balanced approach means that decision-makers input on priorities, forecasters use their skills to identify pertinent risks and opportunities, and admins assist with reducing ambiguity, all in a cohesive and coordinated way that creates much more useful and relevant forecasts. Our process took participants from initially identifying a risk to having questions generated and forecast within three hours. The typical baseline for large-scale systems is a matter of days at best, typically weeks.

In participants' words

Asked how their decision-making would change if they implemented the techniques from the workshop, officials who took part said:⁸

Survey methodology

Over the first quarter of 2025, the Swift Centre conducted a series of six workshops in collaboration with a range of UK government institutions. A total of 107 participants took part in these sessions, which were designed to explore and apply the latest forecasting methods to real-world policy challenges.

The workshops brought together representatives from a wide array of institutions, including the UK Cabinet Office, HM Treasury, the Department for Science, Innovation and Technology, the Government Office for Science, the Ministry of Housing, Communities and Local Government, Oxfordshire County Council, and the Home Office.

Participants included senior policy and strategy officials, economists, senior researchers, principal analysts, senior data scientists and delivery managers. Their areas of responsibility spanned a broad spectrum of policy domains, including technology, artificial intelligence, cybersecurity, national security, pandemics and health resilience, foresight, data analysis, disinformation policy, innovation policy, housing and infrastructure, climate change, local government and civil service reform.

Each workshop followed a consistent structure comprising three core components:

1. Introduction to forecasting. Participants received an overview of research and techniques in forecasting, including practical guidance on how these tools could be applied to support real-world decision-making.
2. Problem framing and question design. Attendees were guided through an interactive session on how to translate simple and complex problems into clear, forecastable questions. This included highlighting how conditional forecasts can support structured option analysis by evaluating potential outcomes under different scenarios.
3. Applied group forecasting. Participants worked together on a relevant example forecasting question — for instance, "Will the UK Government meet its target to build 1.5 million homes by 15 August 2029?" These questions were either agreed with the teams prior to the session or generated in the room.

Initially, individuals submitted their own forecasts, expressed as probabilities or quantity estimates, via the Swift Centre's online forecasting platform. These were followed by a facilitated discussion in which participants explained the reasoning behind their forecasts, engaged in deliberation with peers, and subsequently revised their estimates. The exercise concluded with the generation and forecasting of conditional versions of the original question — for example, "Will the UK Government meet its target to build 1.5 million homes, if [x] policy were implemented?" — enabling participants to explore the impact of specific interventions on future outcomes.

Results and findings

At the end of the Swift Centre workshops, participants were invited to anonymously respond to two central research questions: how would implementing the forecasting techniques from the workshop change decision-making and internal processes, and what constraints currently exist in high-impact government institutions that may create barriers to implementing forecasting as part of internal decision-making?

Benefits of forecasting in improving government institutions

Participant feedback at the end of the workshops revealed a broad consensus that these techniques can substantially improve decision-making in high-impact institutions such as government departments. Responses were all open text, allowing respondents to provide significant individual detail. Other attendees could upvote each other's responses to show which pieces of feedback had the most support in the room. While benefits were noted across a range of areas, we aggregated responses into six core themes. The percentages show the proportion of the total responses in the relevant theme.

How would implementing these forecasting techniques change your decision-making?

• Improved structure, rigour and transparency in decision-making31%
• Better analysis of outcomes and policy impacts27%
• Enhanced collaboration and use of diverse perspectives24%
• Limited benefit to existing processes7%
• Better framing, question design and forecasting technique6%
• Improved organisational culture and constructive challenge4%

Improved structure, rigour and transparency in decision-making (31%)

This was the most cited benefit, with many participants stating that the techniques encouraged clearer, more systematic reasoning. Several noted that the structured nature of forecasting forced them to articulate and justify assumptions, reducing reliance on intuition or authority. Participants appreciated how forecasting frameworks added discipline to their thinking: one described how they would now "explicitly practice forecasting and track my results properly", while another said it would "add more structure… like looking at the world view first, then localising". This theme also captured the role of base rates, outside views and iteration.

Better analysis of outcomes and policy impacts (27%)

Participants repeatedly emphasised forecasting's value in helping them anticipate potential downstream effects of policies. They saw it as a tool for surfacing "what ifs", clarifying trade-offs, and avoiding overconfidence. This benefit was particularly strong in strategy or evaluation-focused teams. One respondent highlighted that forecasting could "help evaluate if something needs to substantially change in a policy or project to make it have the intended impact or whether it should be aborted". Another said it was useful for thinking more systematically about technology surprise and horizon scanning.

Enhanced collaboration and use of diverse perspectives (24%)

Many participants praised the way the forecasting techniques supported collective reasoning and reduced the dominance of the loudest or most senior voices. The collaborative nature of forecasting workshops was seen as an asset, helping teams reveal implicit assumptions and surface disagreements constructively. One response, which received four upvotes from other participants, noted it encouraged more collaborative estimation of policy assumptions, while another reflected that it fostered "greater consideration of the wisdom of the crowd".

Limited benefit to existing processes (7%)

A small number of participants expressed scepticism about forecasting's added value, either because their teams already used similar approaches, or because they didn't see it integrating smoothly into their workflows. One team noted they were already forecasting regularly. Another expressed doubt that the approach would substantially change their decisions, whilst simultaneously recommending that it could be used by seniors who they advise.

Better framing, question design and forecasting technique (6%)

Participants saw value in learning how to better construct questions, assess assumptions, and apply structured methods like the Delphi technique. Several pieces of feedback referred to the benefit of clarifying ambiguous questions, breaking down problems, and improving the quality of inputs that go into decisions.

Improved organisational culture and constructive challenge (4%)

A small but important theme was the potential for forecasting to improve how teams engage with uncertainty and dissent. A few participants said it could help them more confidently challenge senior assumptions or engage in red teaming. One described how forecasting would enable "more rigorous structuring of decision-making assumptions", while another said it would support "balancing political priorities with emerging risks".

Challenges and constraints in implementing forecasting

Alongside widespread enthusiasm, participants also identified a clear set of constraints that could limit forecasting's effectiveness or hinder its sustained adoption.

Assume your team or department started forecasting with these techniques, and it failed to be useful. What would be the main reasons?

• Poor team composition, participation or follow-through37%
• Lack of expertise, training or infrastructure36%
• Insufficient political and senior buy-in15%
• Lack of time or money5%
• No noticeable improvement in decisions4%
• Not aligned to the problems trying to be solved3%

Poor team composition, participation or follow-through (37%)

This was the most cited barrier, encompassing issues related to who contributes, how forecasting is facilitated, and what happens after initial adoption. Several participants raised concerns about insufficient participation, particularly the risk of forecasts being produced by a small or non-diverse group. These groups may replicate blind spots, overlook key perspectives, or over-rely on the views of dominant individuals. Four participants explicitly highlighted issues with cross-governmental collaboration and transparency, including a lack of openness between departments.

Others pointed to institutional inertia, noting that even when workshops were valuable, there was a risk that forecasting would be treated as a one-off and lose energy once people returned to their day-to-day activities. Several respondents feared that the method would be used once, praised as useful, and then forgotten. Concerns were also raised about the lack of standing judgements, absence of mechanisms for tracking accuracy, and the failure to embed outputs into delivery plans.

Lack of expertise, training or infrastructure (36%)

Closely following in importance, many participants flagged the lack of training and accessible guidance as a major barrier to sustained adoption. While the workshops helped in understanding the benefits of forecasting, participants feared that without ongoing support, poor or inconsistent implementation would follow. Several noted the need for structured resources — cheat sheets, question prompts, and templates — especially for teams under pressure or who lack analytical skills.

A related infrastructure issue was raised around access to a forecasting platform. While the Swift Centre platform was praised for its usability, participants worried that it would be difficult to scale forecasting across their team or department without continued access to a secure version that could handle OFFICIAL-SENSITIVE content. Concerns about facilitation quality, low domain knowledge, poor question design, and inadequate diversity of sources also reflected the broader theme that good forecasting relies on capability and infrastructure, not just interest.

Insufficient political and senior buy-in (15%)

Participants expressed concern that even well-constructed forecasts could be sidelined by cultural discomfort with probabilistic thinking. Several noted that probabilistic statements such as "20% chance of banning X" could be seen as politically risky or open to misinterpretation by ministers or senior officials. Others feared that forecasting might expose uncomfortable levels of uncertainty at odds with the perceived need for "objective" advice.

A recurring sentiment was that without senior leadership buying into the benefits and process, forecasting would face an ongoing struggle to be implemented. It was noted that forecasting may be seen as "quirky" or too speculative to be taken seriously by seniors, especially in high-profile areas. Some respondents connected this directly to a lack of understanding of what forecasting is for: not a crystal ball, but a structured tool for reasoning under uncertainty.

Lack of time or money (5%)

Though less commonly cited, time and resourcing constraints were flagged as practical barriers. Some participants noted the difficulty of getting the right people in the room, the challenge of making time to complete forecasts for busy teams, and the risk that time-poor teams would not have capacity to learn or facilitate forecasting properly. It was noted that forecasting may be perceived as disproportionate in terms of time spent for smaller or lower-cost projects.

No noticeable improvement in decisions (4%)

A smaller group of participants raised concerns about whether forecasting would lead to different or better decisions in practice. One person feared people would provide forecasts that already align with their own policy preferences, while a couple of others questioned the added value if the process simply reconfirmed what the team already wanted or expected.

Not aligned to the problems trying to be solved (3%)

Finally, a few respondents questioned whether forecasting was always suited to the types of decisions being made in their teams. This included concerns about using forecasting for outlier events or "unknown unknowns", where the nature of the problem wasn't well defined, or it being misapplied in some scenarios.

Conclusion

The Swift Centre workshops showed that balanced, structured forecasting techniques have the potential to significantly improve how government departments reason through uncertainty, make decisions, and plan for the future.

Compared with previous attempts, such as the UK's Cosmic Bazaar which had thousands of forecasters and days- or weeks-long lead times, Swift's structured workshops and platform enable the creation of accurate forecasts from initial risk identification, through question generation, to forecasting in a matter of hours.

This was not just seen as a technical improvement: participants repeatedly emphasised that forecasting sharpened judgement, encouraged transparency, clarified assumptions, and improved collaboration.

Given the scale and complexity of decisions made within government — from allocating billions in public spending to navigating technological change and geopolitical risk — even a small improvement in decision quality can translate into significant real-world impact. The potential returns on improving institutional reasoning are therefore immense.

However, realising this benefit requires further collaboration between institutions themselves, the forecasting research community, and applied forecasting providers such as the Swift Centre. The workshops revealed a clear enthusiasm to adopt balanced, decision-relevant forecasting techniques, but also surfaced a set of recurring barriers that risk relegating forecasting to a one-off exercise unless they are directly addressed.

To embed forecasting as a sustained part of institutional decision-making, this project recommends four priority areas in need of further investment and investigation.

Balance the governance and management of internal forecasting systems

The troubled legacy systems of Cosmic Bazaar, ICPM and others have typically been dominated by admins, top-heavy, and slow to respond to the needs of decision-makers. Swift's rapid, bottom-up approach balancing the input of actors allows for rapid production of forecasts. As many have noticed, there is a lot of room for improvement in government risk analysis and decision-making; but a giant forecast platform slowly churning out resolvable tournament-style questions is a poor solution compared with rapid iterative forecasting conducted in tandem with decision-makers.

Build capability through training, guidance and facilitation

While top-down systems have struggled, many teams lack the confidence, experience, or tools to independently carry out high-quality forecasting. Participants highlighted a strong need for tailored training that builds understanding of forecasting techniques and provides guidance materials, such as worked examples, question templates, and facilitation guides. A crucial element of this is support in generating the right questions to forecast, ensuring that institutions are able to identify the problems they really care about and then generate forecastable questions for them. The importance of this cannot be understated, given a primary benefit found during the sessions was the ability to produce conditional forecasts in option or scenario analysis, which is highly dependent on good exploration and design of questions.

Additional research and effort should be put towards differentiated training courses and materials for analytical and non-analytical officials that enable them to forecast to assist with their day-to-day decisions. To help bridge this gap, additional investment to access external support — expert forecasters, for instance — could also help bridge early gaps in capability and confidence, especially in relation to effective question generation.

Provide the infrastructure to scale and sustain use

Good forecasting needs infrastructure: not just knowledge, but tools. While participants praised the Swift Centre platform for its usability and how easy it was to make and aggregate forecasts, they also raised concerns about how to use such a platform to scale forecasting across teams given constraints around handling OFFICIAL-SENSITIVE material. To drive adoption there would need to be greater collaboration between providers of forecasting platforms and departments' internal IT and procurement teams, who can then explore how secure platforms can be used throughout the department.

Secure senior buy-in and link forecasting to real decisions

The critical insight from these workshops and our further conversations and investigations is that rapid, flexible forecasting where roles and governance are balanced seems to be the optimal way to use forecasts in governments. We have clearly identified improvements in processes to generate relevant forecasts where teams work directly with decision-makers.

But further work is needed on how to sensitively and clearly link forecasting to other institutional processes, such as strategy development, risk registers and investment appraisals. As part of this, the research community and applied forecasting organisations should seek to build and expand on relationships with leaders of large institutions to highlight the benefits of forecasting, and there should be more effort to identify champions beyond the relationships we have already built within departments who can help make the case for forecasting internally and help drive its adoption.

Final reflection

Forecasting cannot solve all the problems and risks of institutional decision-making. However, we have demonstrated that it is possible to make massive efficiency gains versus the status quo of government forecasting projects. The workshops showed that there is appetite for implementing the techniques presented and a strong appreciation that they can improve decision-making, analytical rigour, and support more collaborative and inclusive thinking.

Given the magnitude of decisions made in government, even modest gains in decision quality could yield substantial returns. To unlock this potential, forecasting must move from an interesting workshop to embedded practice. That requires deliberate investment in people, platforms, and leadership. Without it, forecasting will remain a promising idea — rather than an approach that reshapes how institutions think about and deliver the future.

---

This project was funded by Open Philanthropy. Swift Code 20250221.

References

1. Tetlock et al., Perspectives on Psychological Science. ↩
2. Samotin, Friedman and Horowitz, Crowdsourcing and intelligence analysis. ↩
3. RAND Forecasting Initiative, UK collaboration. ↩
4. Cosmic Bazaar introductory slides, 9 May 2022. PDF. ↩
5. Dart-Throwing Chimp, Asking the right questions. ↩
6. Asterisk, The death and life of prediction markets at Google. ↩
7. Cosmic Bazaar introductory slides, 9 May 2022. PDF. ↩
8. Responses shown were provided by officials who participated in the Swift Centre workshops. ↩