Tips to Evaluate an AI Agent for Your AML Compliance Program

Many banking and financial services organizations have an effective compliance program in place. But, most programs are highly inefficient and rely on too many manual processes that put a strain on the workforce.

As a result, organizations are adopting AI solutions to strengthen their AML operations. They recognize that AI and automation technology can manage and mitigate AML and sanctions risk more effectively while increasing efficiency and productivity. This, in turn, frees employees to focus on riskier customers, transactions, and higher-value work.

One such organization to adopt an AI solution for their AML operations is Valley Bank. They adopted an AI Agent from WorkFusion. Named Tara, this AI Agent comes prebuilt as an AI OFAC / AML expert that is laser-focused on protecting your organization from processing payments by sanctioned organizations and individuals.

In a recent webinar, WorkFusion Head of Product Marketing Kyle Hoback spoke with Chris Phillips, Director of AML Compliance, SVP at Valley Bank to gain insights into how Valley Bank arrived at the decision to hire Tara for transaction screening.

Evaluating process inefficiency via a risk-based approach

Chris noted that Valley Bank and many banks in their peer group still follow the industry norm of throwing bodies at inefficient processes and that AI can dramatically change that approach. “You’ve got to start by identifying your problem and ask, ‘What’s the problem we’re trying to solve? Where’s my risk? How do I pursue that?’” Banks can take that approach today, because AI agents are flexible automations. They can execute complex, multistep workflows across digital environments and apply reasoning. Moreover, beyond their combination of AI/ML and GenAI, they incorporate the critical element of human collaboration. They only call on humans as needed, enabling them to easily make decisions, plan, and adapt to achieve predefined goals – but while having the option to have a human investigate if a decision is too nuanced to make automatically. “For people that are contemplating starting this now, really rethink how you’re doing these things…we started with automating the investigation process,” said Chris.

Applying AI to transaction monitoring

Banks often identify their transaction monitoring process as highly inefficient and a strong candidate for AI-driven automation. After all, the vast majority of work in this process is associated with screening transaction sanctions alerts. Depending on the size of a bank, level 1 analysts face hundreds or even thousands of alerts per week. With unparalleled speed and accuracy, AI Agent Tara resolves false positives, escalates high-risk alerts to subject matter experts and helps stop sanctioned individuals or entities from participating in a bank.

Still, a modern AI solution can only integrate well into a bank’s environment if it can interact with older technologies. “Because you can’t replace your transaction monitoring system, you have to ask, ‘So how do I overlay it with an AI model that does alert suppression?’,” noted Chris. WorkFusion AI Agents were designed with this reality of banks’ installed legacy systems in mind. The WorkFusion AI Agent technology works alongside and integrates into a bank’s existing software, systems and a wide range of data sources – residing between them and the operations staff, collaborating with both sides to perform specific end-to-end compliance processes. Thus, Tara auto-closes false positive alerts – those that are highly unlikely to result in an escalation. In cases where an alert requires escalation to a human for detailed review, Tara automatically does so and provides rational for both auto-closed and escalated alerts.

Ensuring strong model governance

One of the main concerns among AML compliance operations leaders is their ability to govern the AI model(s) that enable AI Agents to make their own decisions. Chris gave this advice to his industry peers: “You have to govern that model, test it, and validate it…Sure, we can buy the technology, but how’s it going to interact with the other technology we have? How are we going to govern that?”

Here are the main elements of governance that surround WorkFusion’s AI Agents and why Chris felt comfortable deploying Tara in Valley’s AML operations:

Tara helps the bank process payments quickly and compliantly by automating entity recognition and name-matching for people, addresses and organizations, comparing them against ‘her’ decision matrix to determine whether or not an alert is a false positive. Her decision-making is driven by the creation of feature outputs from a machine learning ensemble model and a supporting rules-engine.

In line with FDIC supervisory guidance, the four AI/ML models which Tara employs incorporate the three core model components of Inputs, Processing and Reporting – with full explanations, audit trails and reporting that address each component. To further optimize AI explainability and minimize model risk, WorkFusion runs Tara through intensive and continuous testing of the models that underpin all of her decisions. It is a multi-step model testing methodology, as follows:

Step 1: Defining target metrics. For each ML model (I.e. Name matcher, Entity classifier, Decisioning, etc.), the target metric is defined in line with the business objectives. For example, the target metric could be “Precision” for a compliance team seeking to minimize errors.

Step 2: Cross validation and model selection. After defining the training and testing of datasets, the training dataset is subdivided into folds wherein one of the splits within the fold is used for testing the models and the remainder of the splits are used for training. This process is repeated until all the folds are used for testing. The average of these cross-validation results is then used for model selection (I.e., algorithm selection) to reduce selection bias.

Step 3: Testing on out-of-sample data and model tuning. Banks can only ensure implementation accuracy via thorough testing of the new technology. Testing occurs in a manner that is independent of existing rules/algorithms by replicating or re-executing the solution in a separate environment and following documented configuration settings and logic. A population-representative test set is used for hyperparameter tuning of all models.

Step 4: Model Effectiveness. Model effectiveness is typically measured through metrics and reporting. Banks should review the effectiveness of rules/algorithms in place by statistically assessing alerts. This will reveal any opportunities to improve efficiency by revising the thresholds, configuration settings and/or the rule’s logic.

Trends associated with changes in all metrics can be tracked over adjustable periods. In this way, alignment with MRM can remain constant and optimized. For compliance and monitoring purposes, Tara captures and saves, for each alert/hit, all decisions made throughout the automated business process as well as the rationale behind those decisions. Event logs for automation flow are also provided to the FI. Reports specific to Business Process Instances show information such as date of review, hit characteristics, scenario analysis, proposed commentary, decision logic, and model confidence score.

WorkFusion’s AI Agents are already proven and trusted at leading organizations around the world, including 4 of the top 5 US banks. They are substantially reducing the work time needed to resolve the endless flow of low-value customer and transaction alerts. By underpinning automation of many of the manual and repetitive tasks compliance analysts have suffered through for decades, AI Agents are freeing up millions of hours per year so that investigators can focus on actual risky activity. In addition, our AI agents are giving AML officers greater confidence that work is done consistently, backlogs don’t accumulate, and compliance with policy and regulations is significantly strengthened.

If you would like more information on WorkFusion’s AI Agents, please schedule a demo.