Financial services firms are increasingly employing artificial intelligence to better not just their operational operations, but also business-related tasks, including assigning credit scores, identifying fraud, optimizing investment portfolios, and supporting innovations. AI improves the speed, precision, and efficacy of human efforts in these operations, and it can automate data management chores that are currently done manually. However, as AI advances, new challenges arise. 

The real issue is transparency: when individuals don't comprehend or only a few people understand the reasoning behind AI models, AI algorithms may inadvertently bake in bias or fail.

Analytics leaders today often witness hesitation from the leadership while deploying black box AI-powered solutions. This has accelerated the need for explainability in ML models across industries. In fact, according to Gartner, by 2025, 30% of government and large enterprise contracts for the purchase of AI products and services will require the use of explainable and ethical AI. 

While AI in financial services is currently limited to tasks such as process automation and marketing, it might get complex and evolve in the future, making the black box model approach risky. Black box AI provides no explainability and transparency, leaving the stakeholders with questions as to why a model failed or how it arrived at a particular decision. To deal with this, banking and financial institutions are exploring explainability in AI models across various applications.

What is Explainable AI?

Explainable AI models allow stakeholders to comprehend the main drivers of model-driven decisions and interpret the decisions made by AI and ML models. In fact, the European GDPR regulation states that the existence of automated decision-making should carry meaningful information about the logic involved. Explainable models must be able to explain their rationale, characterize their strengths and weaknesses, and convey an understanding of how they will behave in the future, overcoming the challenges of black box AI which cannot explain why and how a model reached a specific decision point.

Explainable AI overcomes trust issues and the possibility of bias creeping in due to prejudice of designers, faulty training data, or no proper business context. It makes ML algorithms transparent, robust and accountable. It does so by assigning reason codes to decisions and making them visible to users. Stakeholders can review these codes to both explain decisions and verify outcomes. Good explainable AI must be highly scalable, easy to understand, highly personalized, and comply with regulatory and privacy requirements as per the use case and country. 

Fig: Working of a Blackbox AI vs Explainable AI models

Why is Explainable AI Crucial in Financial Institutions?

Financial institutions follow strict regulatory policies, and any incorrect decision can cost millions of dollars and damage consumer confidence. It is, therefore, imperative for financial companies to subject AI models to rigorous, dynamic model risk management and validation. The bank must ensure that the proposed AI solution can provide the required transparency depending on the use case. 

Having a solid and practical explainable governance framework could help financial organizations to understand their obligations regarding AI explainability and how to operationalize them. Below we discuss a few use cases where AI in financial services is extensively used and why explainable models are crucial in each scenario. 

Consumer credit: A popular use case of AI in banking, it is used to decide the lending standards. It is crucial that the banks understand why or why not the AI is making decisions about offering or rejecting loans to customers. AI explainability ensures that decision is fair and not biased based on gender or race. 

Anti-money laundering (AML): AML extensively uses AI and demands explainability to understand the model output if it suggests anomalous behavior or suspicious activity in a transaction. 

Customer onboarding: Financial institutions lose millions of dollars due to insufficient customer onboarding processes, and AI ensures that the process is smooth with minimal loss. Explainable AI provides a system for eligibility checks and risk management while maintaining transparency.

Risk Management: Using the historical structured and unstructured data, AI helps the banks and financial institutions track fraud and signs of potential risks in advance. Model explainability must suggest why they identified an activity as risky to be able to manage it and maintain a better customer experience.

Forecasting: AI forecasting models help in monitoring and forecasting incoming financial transaction parameters in real-time. Explainability ensures the accuracy and dynamic nature of automated predictions. 

How to Ensure Explainability in AI Models? 

The more robust the AI technology, the more challenging it is to explain its reasoning because of the complex neural network architecture designed to execute AI. It also demands different stakeholders to have different levels of explanation. They must be able to provide a reason for each decision. For instance, in the case of loan approval AI systems, if the system denies a loan application, it should be able to explain why it denied the application while also suggesting if the outcome is correct or not. The model should be able to answer the following questions to understand if the model is working correctly or not. 

1. What algorithm is it using?
2. How does the model work?
3. Which data is it considering to determine output? 
4. Which variables contributed to the model decision?
5. Does the model decision hold true against regulatory guidance? 
6. Is there even a slight possibility that the model might be discriminatory against certain groups/gender/ethnicity? 

While explainable AI models are an ideal scenario, it might become increasingly challenging to create them as models grow in complexity and precision. There are also concerns about competitors reverse-engineering the proprietary machine learning models and algorithms. There may also be a risk of launching adversarial attacks leading to malfunction. To overcome these challenges many financial institutions are starting to leverage state-of-the-art algorithms while maintaining explainability. A deep understanding of data and processes can help data scientists build custom architectures and ensure explainability in AI models by 

1. Choosing the algorithms carefully keeping explainability in mind 
2. Controlling the span of the model without compromising accuracy
3. Building economic and regulatory assumptions in model training
4. Building MLOps pipelines
5. Ensuring strong model monitoring frameworks

Fig: Points to keep in mind to ensure explainability in AI models

Conclusion

Explainability and good model governance reduce risk and create the framework for ethical and transparent AI in financial services that eliminates bias. As AI use cases grow, it will be of paramount importance to create transparent and explainable AI models to explain critical decisions. Integrating AI and ML model explainability into the processes will pave the way for the future.

Financial institutions can choose AI partners with extensive experience to ensure explanation and transparency in AI models while meeting global compliance requirements at scale. The partner should be able to develop ethical safeguards for designing, developing, deploying, and operating AI systems. A transparent ML framework will bring accountability to models and reliance on complex AI solutions.

 
 
Yuktesh Kashyap AVP of Data Science at Sigmoid. He has almost a decade of experience in implementing machine learning-based decisions and monitoring solutions in financial services domains.