Hallucinations in Gen AI: What Insurers Should Know

‍

‍Understand what hallucinations in gen AI are and how they happen. Explore how these issues affect AI in insurance and get tips for managing them effectively.

‍

What is Hallucination in Gen AI?

Generative AI (Gen AI) refers to a type of artificial intelligence that can create new content, like text, images, or music, based on the patterns it has learned from existing data. 

For instance, it can write stories, generate artwork, or even compose music by understanding and mimicking styles it has been trained on.

In the context of Gen AI, "hallucination" happens when the AI creates information that isn't accurate or doesn’t exist. It's like when the AI makes up details or fabricates facts that aren't based on real data.

‍

Types of Hallucinations in Gen AI

Factual Inaccuracies

The AI might give incorrect facts, like wrong historical dates or false scientific details, even though it sounds like it's correct. For instance, it might say, "Health insurance policies cover all types of cosmetic surgeries," despite the fact that most policies typically exclude cosmetic procedures unless they are deemed medically necessary.

Confabulation

The AI creates completely fictional stories or details that seem believable but are not true. For instance, the AI might state, "ABC Insurance offers a life insurance policy that guarantees payouts for individuals over 150 years old," which is completely fabricated and implausible.

Overgeneralization

The AI makes broad statements that aren’t accurate, like generalising trends from limited information, which can lead to misleading conclusions. For example, it might assert, "All life insurance policies pay out regardless of the cause of death," while neglecting to mention that many policies exclude specific circumstances, such as suicide within the first year or death due to illegal activities.

Misleading Statements

The AI provides correct but confusing information that can lead to misunderstandings. For example, when it states, "Most auto insurance policies offer full coverage," it fails to clarify that "full coverage" often refers to liability, collision, and comprehensive coverage, which may not encompass every scenario, such as rental cars or roadside assistance.

Out-of-Context Responses

The AI might give answers that don’t fit the question or situation, making them seem irrelevant. For instance, when asked, "What are the key benefits of a family health insurance plan?" it might respond with, "Cryptocurrency is transforming global payments," which is completely irrelevant.

‍Contradictions

The AI provides responses that contradict itself or previously given information, making them seem irrelevant. For example, it might say, "Flood damage is covered under standard homeowners’ insurance," but then later claim, "Flood damage requires a separate insurance policy," creating confusion about the coverage.

Fabricated Sources

The AI might invent sources or data, like fake authors or studies, that don't actually exist. For instance, it may state, "According to the 2022 Allianz Global Risk Report, pet insurance is the fastest-growing sector," even though no such report exists or contains that information.

‍

Why Do Hallucinations Happen in Gen AI?

Hallucinations in generative AI occur for a variety of reasons. These include:

‍

Limitations in Training Data

AI relies on the information it's trained on. If this data is incomplete, outdated, or incorrect, the AI can provide inaccurate or misleading answers. 

For instance, a gap in the data available between 2014 and 2018 due to a less-than-successful technology transition in the training dataset for the required model may lead to inaccuracies or outdated information.

Pattern Recognition Errors

Generative AI creates content by recognizing patterns in the large amount of data it has learned from. 

Sometimes, it applies these patterns incorrectly or links unrelated ideas, leading to mistakes. For instance, if it frequently sees two concepts together, it might wrongly assume they’re always connected, resulting in inaccurate information.

Lack of True Understanding

AI doesn't truly "understand" content like humans. 

It generates responses based on patterns in data, not real knowledge. 

So, it might produce plausible-sounding but incorrect information, such as suggesting two historical figures met based on misleading context, even if they never did.

Overfitting

Overfitting happens when AI becomes too focused on specific examples from its training data. 

This makes it struggle with new situations and apply patterns incorrectly, leading to errors. 

For example, if the AI learned that a particular car brand is famous for sports cars, it might wrongly claim that any new model from that brand is also a sports car, even if it’s a family sedan.

Context Loss

Generative AI often struggles to maintain long-term context in conversations or across complex queries. 

In a lengthy dialogue, it might lose track of what was said earlier, causing it to generate responses that contradict previous information or seem unrelated to the current discussion.

‍

Hallucination in Gen AI: Challenges for Insurance Use Cases

Insurers are increasingly using gen AI assistants for tasks like personalizing customer interactions across sales, policy servicing, claims, among others. While this boosts efficiency and personalization, there are challenges to address. Here’s a look at some of the key issues.

‍

Claims Assistance

Some of the ways gen AI chatbots help customers with claims include answering questions and providing guidance. 

However, if the AI gives incorrect information about policy coverage or claim procedures, it can cause confusion and frustration for customers who need accurate details to proceed.

Renewal Assistance

AI can suggest renewal options and additional coverage by reviewing past policies and preferences. 

If the AI makes errors, it might recommend irrelevant or non-existent options, confusing customers, affecting service quality, and impacting potential renewal sales.

Customer Service

AI-powered assistants handle a variety of service tasks like answering questions about policies and billing, among others. 

If the AI gives incorrect information, it can mislead customers about their coverage or billing, eroding trust and negatively impacting their service experience.

Document Drafting

Generative AI can also be used to create summaries of complex policy documents or draft contracts based on customer information. 

If the AI hallucinates, it might produce incorrect policy terms or clauses, which could lead to legal issues or compliance problems. 

Accurate document generation is essential to ensure that all terms are clear and legally sound.

Himanshu Gupta, Chief Operating Officer (COO), says, "Given the nature of Large Language Models, challenges like 'hallucinations' are difficult to avoid. However, this doesn't mean it's a dead-end. By combining human intelligence and experience with a well-planned feedback framework, we can build the necessary guardrails to improve the models' performance and reduce the risk of leaving a business process red-faced.”

He goes on to say, “Therefore, hallucinations should in no way be seen as a deterrent to embracing the transformative benefits of gen AI in the insurance domain, which far outweigh this concern, ultimately enhancing both insurance operations and customer experience."

‍

Guidelines for Insurance Companies to Handle Hallucinations in Generative AI

While insurance companies rely on AI developers for core fixes, they can still take important steps to handle issues with AI hallucinations: 

Select Domain-Focused AI Providers

• Pick Vendors with Domain Experience: AI providers come from various backgrounds. However, those with a track record in the insurance domain are likely to be more hands-on with actual business processes and more reliable in their implementation. Check their industry-relevant experience before signing them up.

• Regular Reviews: Benchmark important metrics on scalable performance and accuracy. Continually assess their model's performance against these metrics and develop a feedback loop to maintain high-quality standards.

Implement an AI Watchdog for Enhanced Oversight

An AI watchdog is a secondary AI system that checks the outputs of the main generative AI. It finds errors and inconsistencies to make sure the primary AI is accurate, compliant, and reliable.

• Real-Time Error Detection: Use an AI watchdog to catch and fix mistakes instantly, especially in key areas like claims and underwriting. For example, if a generative AI generates an auto insurance policy that incorrectly states coverage limits, the watchdog detects the error before the document is sent out, halting the process and alerting the underwriting team.

• Escalation for Major Errors: Establish clear steps for serious errors flagged by the watchdog to be reviewed by human experts.

• Regular Watchdog Audits: Regularly check how well the AI watchdog detects errors and look at any missed issues to improve both the Gen AI system and the watchdog.

• Contract Safeguards: Ensure contracts with AI vendors clearly define responsibilities for fixing errors flagged by the watchdog, including how quickly they must be resolved. For example, a contract may state that major issues must be fixed within 4 hours.

• Transparent Customer Communication: If the AI watchdog finds errors affecting customers, promptly explain the issue, the fix, and how similar mistakes will be prevented to the affected customers. For example, If the Gen AI gives incorrect premium rates, the insurer promptly informs customers of the mistake, provides the correct rates, and outlines how similar errors will be avoided.
  

• Support Decisions: Use AI to assist with decision-making, but ensure final decisions, particularly those affecting customers, are reviewed by human staff.

Documentation

• Documentation: Keep detailed records of how the AI works and potential issues. This aids in quickly identifying and fixing errors.

Communicate Your Use of AI to Customers

• Customer Awareness: Let customers know about AI’s role and its limitations in your processes. Encourage them to verify important information with human representatives.

• Reporting Issues: Provide clear methods for customers to report any problems or mistakes with AI-generated information.

‍

In conclusion, Generative AI offers exciting benefits for the insurance industry, like automating tasks, boosting efficiency, and improving customer experiences. However, AI can sometimes produce incorrect information, which we call "hallucinations." 

To make the most of AI, insurers need to be aware of these issues and take steps to handle them. As AI technology improves, it's important to keep making it more accurate, reliable, and transparent. This helps build customer trust and get the most out of AI’s benefits in the long term.

‍