What It Takes to Trust an AI Risk Score: Key Lessons for Audit-Ready AI

The Imperative of Reproducibility in AI Risk Scoring

The increasing reliance on AI for risk scoring in areas like vendor assessment, transactions, and customer profiling necessitates a robust understanding of how these scores are generated and, crucially, whether they can be trusted. With regulations such as OSFI E-23 looming, internal audit and assurance professionals must be equipped to defend AI-influenced decisions to examiners. The core challenge isn't the AI's sophistication, but its reliability and transparency. A key takeaway from the author's experience is that an AI risk score that cannot be consistently reproduced is not a control; it's merely an educated guess.

Three Critical Questions for Evaluating AI Tools

The author's self-assessment of an AI vendor-risk tool revealed three fundamental issues that audit and assurance professionals should address when evaluating any AI system:

• Reproducibility as a Measured Number: Vendors must provide a quantifiable measure of reproducibility, demonstrating how consistently the AI generates the same risk score for identical inputs. This goes beyond marketing claims of "audit-ready" or "reliable" and provides concrete evidence of the tool's consistency.
• The Challenge of Hosted Models: When AI models are hosted externally via APIs, organizations lose direct control over model updates. A silent update by the provider can lead to divergent results for identical inputs, compromising the integrity of risk assessments. Organizations need a strategy to detect and manage these silent changes.
• Structured Forms vs. Free-Text Interpretation: AI models may prioritize free-text descriptions over structured data fields, treating the latter as mere decoration. This can lead to decisions being based on unintended inputs, undermining the control framework built around structured data. It's crucial to verify what data the AI is actually using for its decisions.

Implications for Internal Audit and Assurance

For internal audit and assurance professionals, these insights are vital for ensuring the governance and control of AI systems. When assessing AI tools, it is imperative to ask vendors specific, probing questions:

• What is the measured reproducibility number, and what were the exact inputs used for this measurement?
• How does the reproducibility number change when the underlying hosted model is updated by the provider?
• Is the AI's decision-making process genuinely driven by the structured data collected, or is it primarily influenced by free-text inputs?

Tools that cannot provide clear, data-backed answers to these questions are not truly audit-ready. Prioritizing these inquiries will help organizations select and implement AI solutions that meet regulatory requirements and provide reliable, defensible risk assessments.