As extreme weather continues to dominate the headlines across the world, there is still a lack of confidence in the ability of AI to predict what is coming.

New research by the UK’s Met Office, thought to be the first of its kind, identifies a confidence gap between existing forecasting methods and emerging AI techniques, helping inform how future innovations are introduced.

The study, published in Artificial Intelligence for the Earth Systems, investigated public confidence in using AI forecasts, as well as perceptions of accuracy of AI weather forecasts and highlights the need for public engagement if potential benefits of AI-enhanced weather forecasts are to be fully realised.

The study added AI is likely to play a significant role in future forecasting. The Met Office currently produces forecasts using physics-based modelling and is actively looking at how these can be augmented by AI models. This future blend of approaches could accelerate improvements to speed and accuracy, so the identification of concerns over public confidence using AI forecasts is an important step for enabling future innovation.

The survey of over 6,000 UK adults asked respondents to rate their confidence in using different types of forecasts for everyday decisions as well as high-stakes decisions. They were also asked to assess the perceived accuracy of both existing physics-based methods and (potential) AI-generated forecasts for daily and severe weather.

Greater uncertainty around the use of AI forecasts for weather-related decision making was highlighted. For physics-based models, around 80% said they would be confident in making weather-related decisions, for both daily and important events, while this dropped to around 40% for potential AI forecasts.

This confidence gap was reinforced by the public’s perception of accuracy. Around 78% rated physics-based models as accurate for both daily and severe weather forecasts, compared to around 45% for forecasts that could be generated using AI techniques in the future.

Met Office science fellow and lead author of the paper Dr Edward Pope said: “This paper deepens our understanding of current public perceptions as we approach a crucial juncture where AI-based weather models are demonstrating their potential to work alongside physics-based methods. This was a unique opportunity to compare public perceptions of established and emerging approaches to forecasting the weather.”

“The gap in confidence and perceived accuracy highlighted in the paper demonstrates the need to clearly and transparently demonstrate the value of new approaches in ways that matter to people. AI techniques are likely to play a significant part in the next generation of weather forecasts, but these improvements will only be fully realised if the public continue to have confidence, and importantly act on, the weather forecast they see.”

The Met Office added trust is fundamental to its work and has been highlighted over the impactful heatwaves of recent months. For the Red Extreme Heat warning in June this year, post-event surveys revealed that 96% of people in the area were aware of it, and 97% of those took action as a result.

Taking action relies on being confident using forecasts for making decisions, which, in turn, is partly underpinned by the perceived accuracy of forecasts for events that affect people. This is why the new research considered those aspects in detail.

Met Office chief AI officer Professor Kirstine Dale said: “We are exploring ways of blending physics-based and AI-based modelling to deliver the forecasts that we all rely on. As with all science developments, we will robustly evaluate and validate any changes to our approach to weather forecasting before we introduce them into the model.

“This stringent, and well established, evaluation process builds and maintains public trust and confidence when using the weather model to make decisions. Maintaining public confidence is critical and this study provides us with valuable insight into what’s needed to ensure that we fully realise the benefits of the integrated approach.

“At the Met Office we will continue to push forward the frontiers of weather and climate intelligence so that we can serve the public and will continue to publish peer-reviewed articles, such as this one, to ensure that we move forward as a community.”