The past week has seen the threat of artificial intelligence rear its head with experts predicting that the technology has the potential to kill off humanity.

AI company Anthropic have called for the pace of development of artificial intelligence models to slow down and to be closely monitored.

Ten company’s  head Dario Amodei wrote in an online essay that developing AI was not in question, but that the risks associated with it were “serious” and that companies and governments must be given time to address them.

The bosses of two rival AI firms, Sam Altman of OpenAI and Elon Musk, have both said they agree with Amodei.

There have been growing concerns recently about the technology’s potential risks. An AI researcher who left Anthropic told the BBC that “if we don’t slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future”.

Earlier this year Anthropic safety researcher Mrinank Sharma announced his resignation in an open letter to colleagues, which stated: “I continuously find myself reckoning with our situation. The world is in peril.”

There are warnings from experts there is currently a10% chance AI kills humanity within 10 years.

The warnings came as the UK’s Parliamentary Joint Committee on Human Rights that there are now a range of human rights risks that the” existing regulatory and legal framework are ill-equipped to prevent or respond to”.

Committee chair, Alex Sobel MP said:  “AI is heralded as an unprecedented era of technological development with the potential to transform our lives for better or for worse. It is moving with such speed and complexity that its impact is hard to accurately predict. What is clear is that at present we are unprepared to deal with its consequences however potentially dire they may be.

“Nowhere in the world, including the UK, has a current legislative and regulatory approach to AI that is fit for purpose. New legislation is needed to establish a comprehensive set of protections that deal with the entire AI supply chain and its lifecycle. A single AI regulator should be established to set policy, monitor performance and with the teeth to ensure enforcement.

“We also recognise that AI has the potential to provide great benefits to society, including improving how human rights are protected. That is why it will be important to ensure that regulation is proportionate and targets the areas of AI development and usage that pose the greatest risks.

“Fundamentally, this is about making sure that you, as an individual, know when AI is being used in the decisions that affect you. We also want to make sure that if something does go wrong then avenues of redress will be available. We need these protections in now, it cannot wait until fear human rights risks become reality.”

Where does this leave businesses and their insurers? AI can and will deliver benefits but there are also significant risks, and the latest warnings will do nothing to calm the concerns of some who believe that AI will cause far more problems than it will solve.

Will there be a slowdown in development or is the momentum simply too rapid for the brakes to be applied.

Jon Guy, Editor,

Emerging Risks