The Risk of AI: Navigating Liability and Governance in Insurance

In the rapidly evolving landscape of artificial intelligence (AI), the recent comments from researchers warning of AI's potential to threaten human existence have sent shockwaves through various sectors, including insurance. This alarming narrative, while dramatic, has highlighted a crucial and more immediate concern: how AI's development affects liability, governance, and underwriting practices in the insurance industry. As AI systems become increasingly autonomous and capable of self-improvement, the question of accountability and risk management looms large for insurers, policymakers, and consumers alike.

The discourse around AI's existential risks has transitioned from niche academic circles to mainstream political conversation, with lawmakers and industry experts urging a closer examination of the technology's implications. Matthew Hill, CEO of the UK's Chartered Insurance Institute, emphasizes that the discussion should not merely be about the sensationalist headlines but rather focus on the practical implications of AI's capabilities, particularly in how they intersect with insurance policies and practices.

The Shift Towards Autonomous AI Systems

The notion of recursive self-improvement (RSI) in AI—where AI systems enhance their own capabilities with minimal human oversight—poses significant challenges for the insurance industry. Recent developments at AI research organizations like Anthropic and OpenAI indicate that AI is advancing faster than many anticipated. For instance, Anthropic has reported that over 80% of the code integrated into its production systems is now generated by its AI models, a staggering increase in efficiency.

The Implications for Insurance

The implications of such rapid development are profound. As AI systems begin to operate with less human intervention, the traditional frameworks for risk assessment and liability are being tested. Tom Hughes, Director of Underwriting at the International Underwriting Association (IUA), notes that AI-generated outcomes can be unpredictable, creating new challenges for organizations that rely on these technologies. This unpredictability raises critical questions about how insurers will manage risks associated with AI-driven decisions.

Regulatory Frameworks and Their Challenges

In response to the growing influence of AI in various sectors, including insurance, regulators have been proactively working to establish governance frameworks. The National Association of Insurance Commissioners (NAIC) has developed a Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, mandating that insurance companies maintain documented AI governance programs. This initiative aims to ensure accountability, risk management, and oversight of third-party vendors.

The State of AI Regulation

As of 2023, 25 states and the District of Columbia have adopted the NAIC's framework, marking a significant step towards standardized AI governance in insurance. However, the regulatory landscape is evolving, with federal-level interventions complicating the situation. An Executive Order by President Trump in December 2025 aimed to establish a national policy framework for AI, which has raised concerns among state regulators about potential overreach and the undermining of state-level regulations.

The Challenge of Liability and Insurance Coverage

One of the most pressing issues facing insurers is determining liability in the event of AI-related incidents. As technology evolves, so do the risks associated with it. Insurers are grappling with how to address potential claims arising from AI's autonomous decisions—especially in sectors like technology errors and omissions (E&O) and professional liability. Some insurers, such as CFC, have started incorporating explicit AI language in their policies to address these emerging risks. In contrast, others, including AIG and Great American, have sought regulatory approval to limit their liability concerning AI-related claims, reflecting apprehension over the potential for massive claims in the billions.

Creating Standards for AI Insurance

Establishing clear standards for AI applications is crucial for the insurance industry's future. Drawing parallels to past innovations, Hughes highlights how the insurance sector historically collaborated with manufacturers to develop safety standards for new machinery and urban infrastructure. Developing a similar approach for AI could help mitigate risks and provide clarity in coverage, ultimately benefiting insurers and consumers alike.

Legislative Actions and Industry Responses

In addition to regulatory frameworks, legislative efforts are underway to address potential algorithmic discrimination in insurance pricing. The PAID Act, recently reintroduced by bipartisan lawmakers, seeks to prohibit insurers from using non-driving-related factors in auto insurance pricing. This legislative movement underscores the need for accountability in how AI is employed in underwriting and pricing models.

California's Groundbreaking Legislation

California has taken a significant step by enacting two bills that establish the first framework for independent audits of AI systems. This legislation mandates accredited third-party audits for AI used in consequential decisions, including insurance. If successful, this model could prompt other states to adopt similar measures, further influencing the regulatory landscape and potentially reshaping the NAIC model bulletin.

Key Takeaways

• AI's rapid evolution poses challenges for insurers as traditional liability and governance frameworks struggle to keep pace.
• Regulatory bodies are actively developing governance frameworks to address the implications of AI in insurance, but federal interventions complicate the landscape.
• Insurers are reevaluating their policies to address emerging AI-related risks, with some adding specific language to cover AI-related incidents.
• Legislative efforts are underway to ensure accountability and prevent algorithmic discrimination in insurance practices.
• California's new laws on AI audits could set a precedent for other states, shaping the future of AI governance in insurance.

Frequently Asked Questions

What are the main risks associated with AI in insurance?

The primary risks associated with AI in insurance include unpredictability in AI-generated outcomes, potential biases in underwriting and pricing, and challenges in determining liability when AI makes decisions. As AI systems become more autonomous, insurers may struggle to predict and manage these risks effectively, leading to uncertainties in coverage and liability.

How are regulators addressing AI risks in the insurance sector?

Regulators, particularly the NAIC, are proactively establishing governance frameworks that require insurers to document their AI governance programs. This includes risk management strategies and oversight of AI systems. However, federal-level interventions may pose challenges to state regulations, potentially affecting the consistency and effectiveness of these governance efforts.

What steps are insurers taking to adapt to AI developments?

Insurers are taking several steps to adapt to the evolving landscape of AI, including incorporating explicit AI language in their policies, reevaluating underwriting practices, and seeking regulatory approval to limit AI-related liability. Additionally, some insurers are participating in discussions around creating industry-wide standards for AI applications to ensure accountability and reduce risks.

What impact could California's AI legislation have on other states?

California's new legislation on independent AI audits could have a significant impact on other states by setting a precedent for similar regulatory measures. If successful, this framework may encourage other states to implement their own auditing requirements, further influencing the regulatory landscape for AI in insurance and potentially leading to more uniform standards across the country.