As the debates around the risks of AI continue Simon Gilbert CEO of Elmore Brokers says the insurance industry needs to show flexibility to support clients as deployment and development continues.

Many companies are integrating artificial intelligence (AI) into their core business operations. Its uses are growing including talking to customers, analysing sensitive data, writing code, supporting operational decisions, spotting fraud and automating workflows across connected systems.

AI is now helping employees work faster and, in many cases, being given access to data, applications and infrastructure. In some cases, these systems now have the ability to make decisions and take actions with limited human input.

We all recognise the hype and excitement around this new capability, but the technology’s usage is also changing the level and nature of cyber risk.

AI is increasingly connected to the same data, applications and infrastructure that businesses have spent years trying to protect. Organisations need to understand what their AI technology can access and what decisions it can influence, and what they can do if something goes wrong.

Generative AI vs agentic AI

Generative AI creates content and has become incredibly popular. However, it has introduced new risks such as inaccurate outputs, disclosure of confidential information and manipulation through malicious prompts.

Agentic AI takes actions to achieve a goal. It can use tools, retrieve information, access databases, communicate with other systems and carry out tasks with limited human intervention. Depending on its permissions, it might send an email, update a customer record, execute a transaction or change a business process.

That ability combined with permissions introduce a very different risk profile.

A malicious instruction hidden inside a document or website could alter an agent’s behaviour. Excessive or incorrect permissions could let the AI access sensitive information it doesn’t need, and poor controls could mean confidential data gets disclosed or the wrong action carried out.

These are both cyber security questions and questions of operational resilience, governance and financial risk.

The challenge for businesses is that AI adoption is happening extremely quickly and governance and security controls aren’t necessarily keeping pace.

What can businesses do to reduce this new risk?

Traditional cyber security remains essential. Businesses still need strong identity management, access controls, secure infrastructure, employee training and incident response.

But AI introduces new requirements, not least red teaming to test how systems behave when someone deliberately tries to make them fail.

Red teaming applies adversarial techniques to an AI system to identify weaknesses before they are exploited in the real world. This can include attempts to bypass instructions, extract sensitive information, manipulate outputs or persuade an AI agent to take an action outside its intended purpose.

Testing is particularly important because the behaviour of an AI application depends on much more than the underlying model. Its prompts, data sources, permissions, integrations and business context can all change the risk.

A system that appears secure in isolation may create a very different exposure once it is connected to customer records, payment infrastructure or internal applications. The aim of any testing, therefore, should be to understand credible failure scenarios and the potential financial consequences attached to them.

Security issues continue after deployment

It’s also important for businesses to understand that AI risk changes over time. Models get updated, applications get connected to new data sources and business teams find new use cases for them. Threat actors also keep developing new ways to manipulate these systems.

Security assessments conducted before deployment cannot be relied on over the medium or long term and businesses need controls that operate while AI is being used.

These can include restrictions on the information an AI application can access, controls over the actions an agent can take, monitoring of inputs and outputs, human approval for sensitive actions and safeguards that intervene when suspicious behaviour is detected.

The principle is familiar to cyber security in that a company should assume preventative controls will sometimes fail and design additional protections around them. For autonomous AI, this becomes particularly important and the greater the authority given to a system, the stronger the controls around that authority need to be.

Insurance needs to be flexible enough to follow the risk

The evolution of agentic AI has important implications for insurers and brokers.

AI-related losses will not necessarily sit within a single insurance category. An incident could involve cyber loss, privacy liability, professional liability, crime, business interruption or regulatory exposure. Understanding how AI is being used therefore needs to be part of understanding the overall insured business.

Brokers need to ask detailed questions such as what AI systems are being used, what information can they access, and are they customer facing or able to initiate actions? Understanding how they are currently tested, what controls are in place, and what happens when something goes wrong are also important considerations.

The right questions and answers should lead to better risk assessment. Insurance should then address the financial exposure that remains after sensible controls have been applied. It cannot substitute for security and security cannot eliminate every possible loss.

As businesses give AI greater responsibility, the insurance market will need to understand the technology, how it creates cyber exposures and how it should be risk assessed. AI will continue to become more capable and autonomous, and risk management now needs to keep pace with it.