AI adoption is a critical focus in the healthcare industry yet concerns about trust often hinder progress. There are legitimate fears that deploying an AI model prematurely could cause harm.
A study published by HiLabs in the Journal of the American Medical Association last year revealed that listings for 81% of providers nationwide have inconsistencies (such as a provider listed at different locations in different directories). Enhancing directory accuracy on this scale is only feasible with AI solutions designed to function as experienced subject matter experts. Therefore, building trust in these models is essential, and there are three main pillars a trusted AI solution should encompass: Accuracy, Reliability, and Transparency.
Pillar #1: Accuracy
1. Source Diversity: Using diverse input data sources enhances accuracy. Since provider data is found on organization websites, aggregators (i.e. Zocdoc), claims data, and national databases, these should all be utilized.2.Large Language Models (LLMs): LLMs can mimic experienced analysts by extracting relevant information. A provider data management solution can use LLMs to read complex provider pages, improving the overall accuracy of the AI's outputs.
3. Learning from Users: Users should easily be able to provide feedback to the AI, for example, when an analyst agrees with the engine if it says an address is inaccurate. This feedback improves the algorithm and fosters trust.
Pillar #2: Reliability
1. Incorporation of Existing Business Rules: A solution capable of integrating existing business rules makes recommendations more actionable. For example, the engine would know not to recommend the removal of a provider’s address if it is the only address in the directory.2. Simulation Environment: Users should be able to engage in “what-if” scenario simulations to test the impact of recommendations, such as network adequacy, before removing addresses. Such forecasting empowers teams to make cleansing decisions at scale, while increasing the technology’s adoption.
Pillar #3: Transparency
Transparency is essential for building trust in AI solutions so they can scale. Subject matter experts should be able to understand an AI model’s decision-making process. The following can assist them:
1. Presentation of Evidence: AI models should provide analysts with clear evidence for their recommendations, such as claims or websites for a provider's address.
2. Confidence Levels: The AI should communicate the rationale for its recommendations, detailing the weights assigned to various factors such as reliability, recency, and relevance of different data sources.
3. Drill-Down Capability: Users should be able to drill down into the data for a more detailed view. For provider data accuracy, this includes detailed views of providers' profiles, addresses, networks, and other attributes with specific recommendations.
4. 10,000 ft View: Providing a high-level summary of data quality through dashboards and statistics is helpful for executives and can provide transparency around an AI solution's overall impact. This can include the total number of addresses removed or the overall increase in a directory’s accuracy.
HiLabs has built its provider data management solution, MCheck Provider, with these pillars and capabilities in mind. As a result, health plans have been able to achieve quick wins, prove ROI, and scale the solution to their different markets quickly. This has, in turn, built organizational trust around implementing AI solutions and paved the way for further innovation.
If you are interested in taking the first step in AI innovation or are interested in seeing MCheck Provider analyze your data, contact us today.