Handling Concept Drift in Interventional Machine Learning Systems

By Peter Baudains

Fraudsters are becoming increasingly ambitious and sophisticated in their financial crime endeavors, and fraudulent activity is becoming harder to differentiate from genuine activity. Losses are being realized in a shorter period, as fraudsters adapt to beat traditional fraud management techniques and strategies. This constant innovation by fraudsters to circumvent fraud prevention and detection systems remains an ongoing challenge for Machine Learning practitioners.

Machine Learning uses historical data to identify patterns that indicate risky behavior in incoming data streams. For some applications of Machine Learning, where these patterns either do not change or change slowly over time, the extraction of patterns from the past to predict future events is not a cause for concern.

However, when applying Machine Learning to human behavior­, for example in predicting fraud from payment data, we need to be aware that the behavior underlying the identified patterns is subject to change. The payments industry is brimming with innovations that have previously served as the drivers of such change (think – chip and PIN or mobile payments). The more that incoming patterns in payments data deviate from their historical counterparts, the more likely the model will become out of date, leading to a drop in performance. This is known as concept drift and can be resolved by updating the model with the latest patterns, as a form of course correction.

Supervised learning methods, which depend on a reliable set of labels to indicate either fraudulent or genuine behavior, pose the additional challenge that there may be a delay before those labels are available. In some cases, labels may be unavailable even after a change in behavior and a corresponding drop in model performance has been detected. But concept drift applies equally to unsupervised approaches, which typically aim to define and identify unusual (or outlier) behavior, and to characterize that behavior as risky.

Regardless of whether supervised or unsupervised methods are used, if the Machine Learning model is working in “detection mode” (­for example to detect and flag, but ultimately allowing a risky transaction to be made), then eventually a true label can usually be obtained for that behavior, which can then be used to evaluate model performance.

Where concept drift is more problematic is when Machine Learning models are deployed in “intervention mode.” This is where apparent risky behavior (as determined by the model) is prevented from occurring. For example, deploying Machine Learning models within the payment authorization cycle means that transactions can be declined based solely on the output of the model, and without any human involvement at the time of the attempted transaction.

In intervention mode, prior to model deployment, it is easy to demonstrate expected uplift in fraud prevented by using simulated performance on a test dataset that has not been observed by the model previously. In this scenario, we know the ground truth – whether behaviors observed are fraudulent or genuine – and we can easily calculate the counterfactual – if our model had been deployed over this period, what would have happened?

If no concept drift occurs, then the performance metrics obtained by this type of analysis would be expected to continue indefinitely. However, the ceaseless adaptation of the fraudster and the increasing sophistication of their attacks, ensures that this is rarely the case. Inevitably concept drift comes into play.

But now, because the model is preventing perceived risky behavior from occurring, we do not know whether those behaviors that the model has prevented would have turned out to be fraudulent or genuine. This means we cannot easily evaluate model performance. We do not know how much fraud has been prevented and we do not know how many false positives there have been. We have the ground truth (now whether the model has declined an attempted behavior) but can no longer obtain the counterfactual (what would have happened if the model was not deployed).

If we cannot evaluate model performance, then concept drift may continue undetected, with the model slowly impacting more and more genuine behaviors. Which is obviously bad news. Thankfully, there are a few tricks up the Machine Learning practitioner’s sleeve that they can use to fight back.

1. Monitor the distribution of the incoming data stream.

Substantial changes in the distribution of the incoming data stream – for example, big jumps in the average value of each transaction – can indicate that the historical data (on which the model has been trained) is no longer capable of discerning between genuine and fraudulent behavior.

Such a scenario could arise, for example, by opening a new payment channel or product line and not separating these additional transactions from the existing model. Although this form of concept drift may not be directly due to a change in fraudster tactics, it can be just as damaging to the model. Small distribution changes may require an update to the model on more recent data, while large deviations may require a full re-training of the model.

2. Monitor the distribution of the output of the Machine Learning model.

Focusing on the output of the model can enable the detection of fraudster behavioral change. This may be as simple as observing a decline in the number of prevented behaviors. If the model is not firing, then the fraudsters have likely moved on. Conversely, if the model gradually starts to prevent more behaviors (e.g., transactions) from occurring, then the business will quickly (and legitimately) start to question whether revenues are being impacted. Once a change has been detected, it’s time to re-train the model.

3. Use interpretable Machine Learning.

Knowing why the Machine Learning method has decided to prevent a behavior can help determine whether the Machine Learning model is still targeting the desired behaviors or whether concept drift has shifted the model off course. If the model explanations no longer make sense to a human analyst, then this is cause for further investigation and possible re-training.

4. Bring back the fraud analyst to estimate the counterfactual.

One of the many advantages for businesses after implementing Machine Learning methods is the reduction in analyst time spent reviewing transactions. But Machine Learning does not completely remove the need for human intervention. The best solution is a combination of human and machine. Using their subject matter expertise, a fraud analyst can review a sample of authorizations that have been declined by the Machine Learning model. By accounting for the micro-level behavioral context under which the authorization attempt was made, the analyst can make an informed decision on whether the authorization was indeed suspicious or may have been incorrectly declined by the model. This information builds up an estimate of the counterfactual and enables not only an evaluation of the model performance, but also a set of labels that might be used for updating the model.

5. Ask your customer.

Finally, the most direct route is to ask the customer whether the attempt was genuine or not. In this case, the customer can quickly review a transaction attempt and provide a reliable label for the declined behavior. But there is no guarantee that the customer will respond. In addition, with large transaction volumes, this may be an expensive and intrusive solution for which there is little business appetite. A solution that automatically sends an SMS or an online notification to the customer can work well but risks even smaller response rates.