For: Security & Risk Professionals
Stop Billions In Fraud Losses With Machine Learning by Andras Cser, April 6, 2015
Key Takeaways Legacy Fraud Management Mechanisms Fail In Today’s Economy In an economy that will continue to see new forms of electronic payments, you can’t adapt the risk scoring models or author new static rules of legacy fraud solutions fast enough to keep up with evolving fraud methods, particularly mobile payment fraud. Machine Learning Models Reduce The Frequency Of Model Updates Machine learning models improve their accuracy autonomously based on transactional data, navigational data, and analyst and investigator decisions. While they take time to “burn in,” long term they cost 30% to 50% less to operate than legacy models. Keep Vendors Honest And Compare Their Results With Existing Scoring Machine learning is a relatively new field in fraud management. While many of its algorithms have been in use for some time, unsupervised machine learning is a new area. Be sure to compare the risk scores vendors generate using machine learning with your existing scores to avoid any regression.
Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA 02140 USA Tel: +1 617.613.6000 | Fax: +1 617.613.5000 | www.forrester.com
For Security & Risk Professionals
April 6, 2015
Stop Billions In Fraud Losses With Machine Learning Machine Learning Is The Only Way To Keep Up With Resourceful Fraudsters Committing Cross-Channel Fraud by Andras Cser with Stephanie Balaouras and Jennie Duong
Why Read This Report Security and risk (S&R) professionals specializing in fraud find it increasingly difficult to develop new behavioral patterns and models to detect the telltale signs of cybercriminal activity across commerce channels — particularly mobile. This is exacerbated by many firms’ inability to find or afford enough fraud data scientists. Thus, when S&R pros must adapt their own fraud models, it’s a slow and inefficient process. The alternative is to wait for vendors to update the models in their commercial solutions. Both scenarios leave some businesses vulnerable to significant fraud losses for an extended period. In addition, with legacy models there is no way for S&R pros to know when the accuracy of the model has deteriorated. This is why there is so much excitement at the prospect of applying machine learning methods, algorithms, and models to fraud management. The hope is that machine learning will drastically reduce model update cycle times, which will not only improve fraud detection but give fraud analysts and investigators more time to focus their efforts on investigating suspicious transactions. In this report, we examine the promised benefits of machine learning and provide an overview of the vendors incorporating it in their technology.
Table Of Contents
Notes & Resources
2 Legacy Fraud Detection Methods Will Cost Firms Billions In Losses
Forrester interviewed 12 vendor and user companies: Accertify, ACI, BAE, CA Technologies, Feedzai, IBM, Kaspersky Lab, RSA, SAS, Skytree, ThreatMetrix, and WorldPay.
3 Machine Learning Overcomes The Limitations Of Legacy Methods 8 Case Studies In Machine Learning And Fraud 11 Navigate The Vendor Landscape recommendations
14 Track Machine Learning Efficiency In Vendor Proof-Of-Concept Demos WHAT IT MEANS
Related Research Documents Big Data In Fraud Management: Variety Leads To Value And Improved Customer Experience The Forrester Wave™: Enterprise Fraud Management, Q1 2013
15 Machine Learning Will Extend To All Types Of Transaction Fraud 16 Supplemental Material
© 2015, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To purchase reprints of this document, please email
[email protected]. For additional information, go to www.forrester.com.
For Security & Risk Professionals
2
Stop Billions In Fraud Losses With Machine Learning
Legacy Fraud Detection Methods Will Cost Firms Billions In Losses According to several studies, eCommerce fraud loss as a percentage of revenue ranges between .85% and .9%.1 Forrester estimates 2014 online retail sales of $294 billion in the US, CD$38 billion in Canada, and $5.7 billion in Mexico; given this, we can estimate that 2014 online fraud losses in just North America to be approximately $2.7 billion.2 Without an improvement in the fraud loss rate, North American eCommerce fraud losses will increase to approximately $4.2 billion by 2018. However, reducing fraud losses will be challenging. According to Forrester’s North American Consumer Technographics® Financial Services Survey, 2014, 72% of US online adults use at least one form of electronic payment regularly. In an economy that will see continued adoption of multiple cashless payment options, it’s very difficult to detect and prevent fraud across multiple channels, including web, mobile, phone, in-person, and kiosk, and across payment types. Legacy fraud techniques and methods are failing S&R pros because these techniques:
■ Can’t detect fraud quickly enough given an avalanche of customer data. Mobile devices,
cross-channel interactions, and customers’ social media activity generate an enormous amount of data. Using legacy tools to identify fraud trends while protecting against new and emerging fraud patterns is difficult, if not impossible, in an environment of fast-streaming, nonpersisted, and real-time data in huge volumes from a variety of sources — such as clickstreams, geolocation information from mobile devices, Facebook, and Twitter posts.3 Fraud management solutions and tools that require extensive, supervised training can’t keep up with these new fraud patterns.
■ Have yet to adapt fraud patterns to account for the popularity of mobile commerce. Many
legacy enterprise fraud management (EFM) platforms are only suites for traditional payment fraud detection and can’t easily integrate location, IP address, and social network data — not to mention sensor data such as acceleration and application use patterns churned out by mobile devices. Baseline differences from these data sources often indicate fraudulent activity.
■ Can’t easily identify new cross-channel fraud patterns. Cybercriminals love to use the mobile
channel to perpetrate fraud. The telephone/call center, in-person, and even ATMs are also still popular channels. In fact, fraudsters often take advantage of multiple channels to commit moresophisticated and -complex schemes.4 For example, a fraudster can sign up for online access, change the mailing address of the account online, then order an ATM card to drain the account. Thus, actions that may look innocuous on one channel suddenly become fraudulent if observed in the context of multiple channels.5 To tackle cross-channel fraud, legacy fraud management techniques (such as statistical models and rule sets) won’t suffice: They are slow, hard to retrain and maintain, and produce high false positive rates.
■ Require the work of expensive, hard-to-find data scientists to be successful. Forrester
interviewed a small, regional bank that said that its top concern regarding EFM implementation was the complete lack of data scientist employees and the prohibitively high labor costs of
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
3
Stop Billions In Fraud Losses With Machine Learning
such skilled workers. Only the largest banks can afford to employ armies of data scientists to continually refine statistical models and maintain rule sets in EFM solutions. To curb fraud losses and control the cost of manual fraud identification, analysis, and investigation, small and regional banks need to find alternative, less labor-intensive but equally effective, fraud management solutions. Machine Learning Overcomes The Limitations Of Legacy Methods Machine learning is a type of artificial intelligence (AI) that gives computers the ability to learn without human programming.6 Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data. Machine learning works by building automated analytical models using an iterative mathematical algorithm that learns from its mistakes in previous iterations using new data supplied.7 Here’s what’s different about machine learning compared with traditional fraud detection methods. Machine learning solutions:
■ Support real-time decision-making. Ten years ago, most banks and eCommerce firms were
able to live with nightly batch fraud screening processes. One of the biggest changes in EFM is the shift toward real-time decision-making and interdiction (less than a 1-second response time). Today, because of increased fraudulent activity and competition and faster (often instantaneous) bank transfers, this is no longer viable. The norm is real-time. Fortunately, machine learning algorithms coupled with powerful hardware architectures can support this requirement. Many algorithms that were once unfeasible to implement on old hardware, such as ensemble models and complex, neural networks coupled with in-memory processing, became operationalized weapons in the arsenal for fighting fraud.
■ Don’t rely on static rules, the manual adjustment of model thresholds, or blacklists. Machine
learning is not about maintaining rule sets for score adjustments or business decisions. It doesn’t rely on continuous human programming to manually adjust thresholds in statistical models or to maintain whitelists and blacklists. It also doesn’t rely on fuzzy matching using a predefined algorithm, or the simulation of “what-if ” scenarios.
■ Require much less training than statistical models. Older models that are statistical-based
systems require extensive “supervised training.” This requires fraud analysts to feed six months of training data into the system in order to tune the statistical model to identify known, confirmed fraud. This is a slow, expensive, skill-intensive, and rigid process. Machine learning models are ideal for “unsupervised training.” In this case, analysts feed production data into the system and allow it to learn and adapt continuously as it identifies anomalies and incorporates analyst and investigator feedback from confirmed fraud cases.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
4
Stop Billions In Fraud Losses With Machine Learning
■ Require less maintenance. Older, rules-based systems are very high maintenance. Legacy
fraud management systems often have 100 to 200 static rules. They require extensive and costly manual maintenance, such as tuning of thresholds, and, based on our interviews, have a 30% to 40% slower throughput at the same false positive rates than comparable machine learning systems. Forrester’s interviewees said that fraudsters immediately find rule thresholds, such as $300 of funds available immediately after depositing a check, and exploit them quickly, thus making rules less effective. Machine learning solutions don’t require rules and thus have a lower maintenance cost than systems with explicit rules.
■ Feed on large data sets and improve their accuracy with time. The more data you feed to a
machine learning algorithm, the better it becomes. Thus, big data makes machine learning not only possible but also more reliable than legacy models that could consider only much smaller data sets. In fact, the availability of rich contextual data from mobile operating systems and applications only serves to improve EFM systems powered with machine learning: The system can identify subtle fraud patterns based on geolocation, device usage patterns, application network traffic, and other contextual data.
■ Learn from analyst and investigator feedback. Machine learning models can learn from
analyst and investigator feedback and knowledge of prior transactions. If an investigator marks a case fraudulent, the machine learning can leverage that information in reviewing subsequent transactions with similar parameters to improve decisioning. In the first year after deployment of Accertify’s solution, European airline easyJet cut the percentage of fraud loss on revenue by 29%, and in the second year by 39%.8
Advanced Techniques Improve Detection And Automate New Pattern Identification Machine learning differs from legacy statistical and rules-based models in a number of other ways.9 Some features and functions help to improve detection and automate the identification of new fraud patterns at a lower cost. Machine learning:
■ Can automatically identify fraud patterns faster in transaction streams. Financial crime and
compliance applications of machine learning most often relate to pattern classification in which the goal is to divide data into groups that convey some concept of reputational, regulatory, or financial risk. Machine learning systems can identify relationships and causality between input variables (such as IP address) and an output variable (such as fraud or not) much faster than legacy methods can. Identifying patterns in transaction streams allows EFM solutions with machine learning to flag transactions that are suspicious based on trending. An example trend might include a fraudster trying to withdraw the following amounts in order — $1,000, $500, $300, $100 — to establish the fraud limits of the ATM.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
5
Stop Billions In Fraud Losses With Machine Learning
■ Allows for classification and grouping of transactions. With classification, the model
separates transactions into two or more classifications, such as fraudulent transaction, transaction to review, and legitimate transaction. Then, the algorithm produces a model that makes the call on unseen transactions and places them into the above classifications. Regression predicts continuous outputs (e.g., “How much is the predicted loss for a transaction?”). Clustering identifies previously unknown groups based on data only by “looking at transactions” and then attempts to put unknown transactions into the identified groups. The result? Classification allows EFM tools to predict if a particular transaction will be fraudulent or not based on its attributes like dollar amount and location.
■ Can operate supervised or unsupervised. Supervised learning means that the system attempts
to identify data elements on its own that have been labeled fraudulent by data scientists training the solution. This iterative process requires some manual tuning of the model by a data scientist. Unsupervised learning means that no one gives labels or indicators to the model, allowing the model to find structure automatically in the input. Classification and regression can be both supervised and unsupervised, while clustering is typically unsupervised. The benefit of unsupervised algorithms is that they don’t require extensive human labor for training and are less costly to maintain.
■ Identify predictor features automatically. Machine learning algorithms and methods can
identify predictor features automatically. For example, for a North American regional bank, contextual authentication machine learning has in the past identified the following predictor features automatically: 1) time the user has been known as a customer to the bank; 2) number of transactions in the past 10 minutes; 3) number of times the user is transferring money to the destination account; and 4) how typical this geolocation is for the user (deduced from the IP address). An EFM solution employing this method will have a lower cost of finding features than data scientists manually trying to identify predictor features.
Random Forests, Deep Learning Algorithms Advance In Machine Learning Machine learning algorithms matter. While regression methods and Bayes predictors have been around for a long time, S&R professionals don’t consider them robust enough for machine learning. Here is a quick overview and the pros and cons of the most common machine learning algorithm solutions for fraud management:10
■ Random forests are very well understood and fast to learn and score. Random forests
are based on decision trees and are one of the oldest machine learning technologies.11 Data scientists usually use them in credit and fraud scoring among many other applications such as medical diagnosis. Pros of the method include: 1) It can handle missing data; 2) its analytics is robust enough to resist the skewing of outliers; 3) it requires little tuning; and 4) it’s fast to train
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
6
Stop Billions In Fraud Losses With Machine Learning
and score. Challenges are really few and easy to overcome, but they are: 1) One cannot easily interpret decisions as rules or single trees; 2) the input requires labeled data; and 3) it handles a high number of features poorly.
■ Deep learning or neural networks provide excellent predictive power but are a black box.
Neural networks (or more recently called “deep learning systems”) are complex nonlinear models with very large numbers of parameters. The prediction equations are generally sums of exponentials. The equations behind neural nets tend to be so complex that they are effectively a black box. The highest performance fraud and credit scoring tools routinely employ some type of neural network. The benefits of neural networks include: 1) They have the ability to represent complex patterns; 2) they can provide great predictive power; and 3) they are parallel. The downsides are: 1) They have difficulty handling different input types; 2) they can’t handle missing data values; 3) they’re slow; 4) they require extensive tuning and retuning; and 5) they are hard to interpret.
■ Support vector machines (SVMs) decide which population a transaction belongs to. SVMs
assume that the outcome it predicts is binary, so repeated use is required when predicting multiple outcomes. Good data preparation is essential. Data scientists use them in credit card fraud management and credit scoring. Benefits of SVMs include: 1) They have the ability to detect nonlinear patterns; 2) they’re effective with many features (e.g., high dimensionality); 3) they have good predictive power; and 4) they’re not as prone to overfitting as neural networks. Challenges include: 1) SVMs can’t handle missing values; 2) they’re hard to scale; and 3) it is hard to find optimal kernels.12
■ Clustering algorithms perform well in unsupervised learning and group observations.
Clustering algorithms include KMeans, KMediods, and Kohonen (self-organizing) maps — all based on the KMeans algorithm. Data scientists use them for supervised learning in an iterative fashion to segment data in very large sets. Benefits of clustering algorithms include: 1) They have a high tolerance to missing data and outliers; 2) they have good predictive powers; 3) they offer fairly easy graphical representation; and 4) can reduce dimensionality quickly. Challenges of clustering algorithms are: 1) They have an inability to handle different types of input; 2) they require tuning; 3) they’re hard to interpret; and 4) KMeans requires specification of the number of clusters.
Machine Learning Improves Fraud Management Performance Good fraud management is becoming a competitive differentiator among banks, insurance companies, and healthcare organizations. More and more customers understand, look for, and demand robust security and fraud management practices at a bank. In light of the recent credit card breaches at Home Depot, Neiman Marcus, Target, and others, S&R professionals are increasingly paying attention to and becoming concerned about solid fraud management practices that keep the
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
7
Stop Billions In Fraud Losses With Machine Learning
customer experience intact. Your firm/s customers, who are rightfully worried about their online security and privacy, are more and more likely to go to a competitor if you can’t protect them. In order to find the tell-tale signs of fraudulent activity in the enormous data sets collected from mobile devices, phone conversations, and transactions conducted across many different channels, S&R professionals require machine learning algorithms because they:
■ Can catch behavior changes and patterns as they occur. You can only defend against threats
you can identify. Machine learning helps by automatically identifying changed behaviors and detecting new patterns long before an army of human analysts (or in fact older, typology-based algorithms) could. Machine learning allows for quick and effective ingestion of large data sets and allows for checking out different data segmentation types and what-if scenarios quickly and efficiently. Many times, machine learning can identify counterintuitive results that human analysts can’t. Machine learning can also identify patterns that senior management can use to make strategic decisions such as reorganization or a shift in go-to-market-strategy.
■ Allow you to measure the accuracy of your fraud model. Machine learning methods allow
you to measure how accurately you can predict the outcome of your classification. This is very important, especially with real-time interdiction when you may have to block transactions if you have a high confidence of a transaction being fraudulent. This can greatly contribute to reducing false negative rates and ensures that you don’t overlook the truly high risk and fraudulent transactions. Machine learning applied in iterations can also greatly improve data quality and fill out missing feature values in records.
■ Improve the performance and scale over rules-based or manual methods. Vendors Forrester interviewed for this report said that machine learning algorithms, especially when combined with each other, tend to perform 30% to 40% faster at equal false positive rates than equivalent rules-based and other AI systems. Given the recent drop in prices and explosion of computing power, many machine learning algorithms that have been previously cost-prohibitive from a computing perspective have become commonly available and thus significantly refined and more accurate.
■ Require no programming and tuning with unsupervised machine learning. Unsupervised machine learning (available with decision trees and random forests) goes a long way in those scenarios where a fraud team has no access to previous training truth data or when there is a lack of data scientists. Since most machine learning tools built into commercial fraud management platforms require no programming, they allow for a much lower cost and faster implementation of robust, industrial-strength EFM.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
8
Stop Billions In Fraud Losses With Machine Learning
Case Studies In Machine Learning And Fraud The following are three enterprise case studies demonstrating the best practices and benefits of machine learning in fraud management:
■ Global financial services companies monitor trading using self-organizing maps. Over the
past decade, over $15 billion in losses have been through rogue trading activity. BAE Systems developed and implemented the NetReveal Unauthorized Trading detection solution to address this at a number of companies; NetReveal uses unsupervised learning in the form of selforganizing maps. BAE used “unsupervised” SOM (self-organizing maps) as neural network methods to detect unauthorized trading. Maps show traders’ activity features like volume of trades, most frequently traded products, and other key risk indicators. It uses a neural network to automatically cluster traders exhibiting similar behaviors. This allows the solution to identify significant changes in trader behavior (e.g., a front-office trader behaving like a back-office trader). The solution has been proven to identify unauthorized trading before significant losses occur and predicts trader misbehavior for about six months — giving ample time to the bank to avoid actual losses (see Figure 1).
■ A US credit issuer uses high-performance models to cover cross-channel fraud. A top US
credit card issuer with 50 million accounts with a multilayered authorization system struggled to improve fraud detection without affecting customer experience. The issuer had both in-house and multiple third-party commercial fraud risk scoring engines. The growth in EMV smart card adoption led to fraud shifting to “card not present” transactions on alternative channels and online banking. Existing fraud risk scores were colliding, and this in turn resulted in a poor customer experience. A team of three full-time equivalents (FTEs) chose Feedzai random forest, SVM, and other machine learning models to define model features, extract training samples, and test performance in three weeks. The issuer improved detection rates by more than 40%, for a $125 million increase in savings. The solution detected that 68% of the issuer’s fraud was cross-channel, and its fraud models achieved a 20 millisecond response time at 10,000 transactions per second.
■ A US bank improves anti-money-laundering (AML) efforts with hybrid models. A US
bank, seeing high false positives in AML alerts, was frustrated with its inefficient AML investigation. The bank deployed SAS’ AML solution and applied a hybrid model of supervised and unsupervised logistic regression and decision trees. The hybrid model allowed the bank to simplify and reduce its filing workload of suspicious activity reports (SARs) using a transparent and auditable operational process that was also easily explainable to auditors. The solution produced $1 million savings in AML investigations in the first year, reduced the number of work items by 46%, and allows for autotriage of work items in queue in an objective and repeatable manner.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
9
Stop Billions In Fraud Losses With Machine Learning
Figure 1 Example Of Kohonen Map Interface Bar chart of average trader activity in cluster
OTHER
OTHER
OTHER
OTHER
OTHER
OTHER
120912
OTHER
SALES
OTHER
OTHER
OTHER
OTHER
Cluster of traders with similar behavior
SALES
OTHER
OTHER
OTHER
OTHER
SALES
SALES
SALES
SALES
OTHER
OTHER
SALES
OTHER
TRADING
SALES
SALES
OTHER
TRADING
TRADING
SALES
SALES
SALES
TRADING
TRADING
Source: Forrester Research, Inc. Unauthorized reproduction or distribution prohibited.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
10
Stop Billions In Fraud Losses With Machine Learning
Prepare For The Speed Bumps: Supervised Machine Learning Can Be Tough To Set Up Although effective, machine learning methods aren’t the end-all-be-all of solutions. If you’re considering using machine learning to improve fraud management, you should be aware that:
■ You may struggle to set up supervised machine learning. Supervised machine learning can be hard to set up because it requires reliable training data and time. Over-fitting the model (seeing patterns and signals where there are none) can be a problem as well. Neural networks can require a fair bit of computing resources to solve complex problems.
■ Machine learning solutions require a “burn in” time. Since models improve as more data is fed
to them, it takes time (a few months at least) for the model to reach its target accuracy, as the model needs to capture fraud patterns hidden in the data. S&R pros can avoid this by using parallel and legacy methods for risk-scoring during this silent period. Unsupervised learning algorithms usually require a lot of data before they can be as accurate as supervised learning algorithms.
■ Missing and outlier data or too many features can mislead some algorithms. Especially with
unsupervised training, and neural networks and support vector machines, missing feature values or incorrectly captured, erroneous feature values can cause the model to go astray, which can require manual intervention and retraining. When the number of dimensions of the model increases, the model’s size becomes large and sparse. This scarcity is problematic for any method that requires statistical significance. So, with greater volumes of data, results can be more difficult to correlate and irrelevant features can impede the model. Machine learning algorithms can also produce spurious and unstable results.
■ Easy of interpretation can suffer, leading to hard-to-explain fraud decisions. Machine
learning methods don’t always produce clear-cut explanations as rules-based systems or Bayesian predictors. The logic that the learning algorithm identifies is hard to explain, which causes natural tension between accuracy and interpretability. As a result, even though the model may be very accurate, the score may be hard to explain, which causes a perception of black-box decision-making. Thus, fraud management professionals may face difficulties explaining risk scores to affected customers, regulators, and auditors.
■ Machine learning requires big data expertise and tools. The percentage of transactions that
are fraudulent in a given payment activity or online banking system is small. Given that most transactions are genuine, the identification of the patterns associated with fraud using machine learning requires the analysis of very large sets of data. Big data analytics involves technologies that facilitate the economic and rapid analysis of these data sets. Traditional statistical methods can’t handle big data really well. This mandates that the company deploys and maintains a tall technology stack, consisting of many systems to pull in from and process data, including DBMS, Hadoop jobs, NoSQL data stores, Python, R, Weka, and streaming systems.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
11
Stop Billions In Fraud Losses With Machine Learning
Navigate The Vendor Landscape Vendors increasingly incorporate machine learning, especially unsupervised machine learning, into their offerings. Every vendor has to compete in multiple geographies where some of their clients are reluctant to provide training data for tuning the vendor’s EFM model — so machine learning algorithms increasingly substitute training data. Forrester’s interviews with vendors indicate a dynamic landscape in which:
■ Accertify uses deterministic and nondeterministic algorithms. Through its fraud solution,
clients that opt in can leverage positive and negative experiences across all participating Accertify clients. Accertify Index leverages a cross-client statistical model to provide a score to indicate the correlation of one or more data elements to a positive or negative experience by one or more clients. Accertify also creates custom machine learning models for clients, leveraging the many custom variables that clients send that are distinct to their business to provide improved and customized decisioning. Medium-to-large eCommerce companies use the vendor’s solution.
■ ACI tightly integrates its payment risk management offering with payment solutions. As
part of the Universal Payments Framework, and building on ReD’s existing models, ACI’s solutions enable machine learning to use transactional and peripheral data from the entire transaction life cycle, but ACI’s machine learning algorithms do not substitute for training data. The solution then creates and fine-tunes the detection triggers, which improves response times and accuracy. ACI has invested heavily in analytics research to further automate the creation of rules and modeling capabilities. Medium-to-large banks and financial services companies use the vendor’s solution.
■ BAE offers an integrated solution with a broad selection of models. The NetReveal machine
learning is a fully integrated solution that contains configurable analytics. The solution offers transparent models for feature relevance and dimensionality reduction, logistical regression, frequency profiling and outlier analysis, text mining and sentiment analysis, as well as rule induction, neural networks, stochastic gradient-based algorithms, and unsupervised clustering. The single view of the customer allows investigation not just at the level of the individual, but also around the company he/she keeps. Future plans include automating online learning, as offline approaches are increasingly less effective; automated model drift detection and correction; and improved visualization tools.
■ CA Technologies adds machine learning to 3D Secure implementation. CA Risk Analytics
uses machine learning to automatically create features from inputs in the current transaction and the behavior distillates on users. These features are used to power the neural network model suite that produces risk scores for making real-time operational decisions to stop authentication fraud. CA Technologies’ neural networks extend functionality of CA Risk Analytics to examine patterns across multiple issuers.13 CA Technologies uses machine learning in its risk-based
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
12
Stop Billions In Fraud Losses With Machine Learning
authentication and 3D Secure processing solutions. It plans to use machine learning to: 1) augment Risk Analytics with models for multiple international regions and 2) use real-time data from multiple issuers for fraud risk scoring.
■ Feedzai scales based on industry standard big data Hadoop and NoSQL/Cassandra
platforms. Feedzai’s Fraud Prevention Platform is a self-serve, end-to-end, big data modeling environment. The solution links online and offline behavior patterns to increase profile accuracy and maintains individual baselines of expected behavior regardless of channel. It also allows for white-box, model-based continuous machine learning: It continuously rebuilds behavioral profiles and continuously updates fraud patterns in real time.14 The vendor’s plans include: 1) easier and faster feature design using a point-and-click data science framework; 2) deep learning anti-fraud models that are closer to artificial intelligence and mimic how the human brain works; and 3) cloud API improvements.
■ IBM connects fraud management, security, and marketing BI with machine learning.
IBM launched a new Counter Fraud initiative in March 2014.15 Based on SPSS, the solution incorporates standard and proprietary machine learning and other types of algorithms and can integrate with R algorithms.16 It offers anomaly detection, and entity analytics algorithms establish relations between entities. IBM plans to: 1) invest in developing new algorithms; 2) integrate information security, identity and access management, and AML; 3) use Watson and other artificial intelligence techniques to process data; 4) apply machine learning algorithms to large, in-memory data sets and evaluate models against those faster; and 5) use the cloud as a delivery and computing platform.
■ Kaspersky Lab plans to use machine learning for biometric analysis against fraud. Kaspersky Lab uses machine learning techniques to identify global trends in malware, fraud in social networks, phishing campaigns, and fraudster behavior analysis. Kaspersky Lab’s access to advanced threat research provides a foundation for selecting which data to analyze and which machine learning models to apply to a given problem. Machine learning will take a central role in the future Kaspersky Fraud Prevention’s behavioral biometric analysis capabilities, including navigation, mouse clicks, and historical behavior.
■ RSA uses Bayesian models in its risk engine across the board. RSA hosted and on-premises
Adaptive Authentication solution’s risk model is self-learning — it learns from case resolution as well as genuine or failed authentication feedback and online live shared fraud intelligence data. The risk engine modifies its risk predictions based on case investigation results and authentication feedback then automatically updates the risk model to catch fraudulent activities that were missed or genuine activities that were wrongly flagged. RSA plans to: 1) enhance its device identification with machine learning; 2) enhance the use link analysis; 3) allow customers to use custom predictor features; and 4) use machine learning to create unsupervised algorithms for behavior anomaly detection.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
13
Stop Billions In Fraud Losses With Machine Learning
■ SAS combines and autoselects multiple models in its solution for the best fraud detection.
SAS Data Mining, SAS Visual Scenario Designer, and SAS Model Manager all work using multiple approaches to look for the best fraud detection lift, and then the technology recommends the best approach as opposed to a human bias that might adversely affect the results. In addition to SAS’ many machine learning methodologies, the capabilities supporting the model life cycle within the SAS solutions support data driven detection. Using its big data analytics platform, SAS plans to: 1) use ML tools in its new enterprise fraud management models; 2) make analytics more accessible to all business users by using visual interfaces; and 3) improve performance by using in-memory processing.
■ Skytree develops high-performance machine learning algorithms using big data. Skytree’s
specialty is creating high-performance and scalable machine learning (KMeans, SVM) algorithms.17 The vendor’s solutions are used for fraud and credit risk scoring of customers as well as pricing and churn analysis. The solution can identify the point of compromise by using big data sources and providing real-time insight into patterns. At a credit card network, Skytree helped the client move from annual model updates to daily or real-time model updates on nofrills Linux x86 hardware. Skytree’s focused on allowing analysts to create models rather than forcing them to write software.
■ ThreatMetrix provides web fraud with machine learning in its cloud platform. In its
largely banking- and eCommerce-targeted solution, ThreatMetrix collects malware-related information and information on device data, login, user name, address, phone number, email, product purchased, amount, and quantity. The solution uses machine learning algorithms on anonymized data to uniquely identify devices and to generate trust scores based on global behavior patterns. This year, ThreatMetrix is expanding this platform to include both supervised and unsupervised learning and create machine-learning-based real-time rules. API and userinterface-based analyst feedback refines the algorithms. ThreatMetrix TrustDefender Platform will combine integrated end-point intelligence, anonymized identity and context intelligence, visualization, case management, workflow, and machine learning.
■ WorldPay collects data from its payment gateway, fraud screening, and acquirers. Models
comprise sector-built profiles, global velocity controls, global split cross-reference controls, global fraud data pooling, custom rules, merchant and sector-built rules — all of these alongside ongoing merchant review/chargeback reviews. While RiskGuardian today does not offer all of the mathematical algorithm capability as may be technically defined for a machine learning system, WorldPay is planning to partner with a machine learning algorithmic-based services provider to run proof of concepts around the business case for implementing machine learning in its solutions in the first half of 2015.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
14
Stop Billions In Fraud Losses With Machine Learning
R E C O M M E N D AT I O N S
Track Machine Learning Efficiency In Vendor Proof-Of-Concept Demos Finding the right vendor to convert your legacy EFM portfolio to one powered by machine learning is no easy task — there are a lot of misconceptions and exaggerated vendor claims. Forrester’s conversations with EFM practitioners highlight that S&R pros should:
■ Demand proof that the vendor can do unsupervised learning — fast. Especially if your
fraud team lacks training data, data scientists, and statistician talent, the solution needs to be able to perform self-learning on existing offline data and streaming online data, and you must be able to deploy it in production in no more than four to five months. If you already invested in supervised machine learning, try using it in parallel first with unsupervised machine learning tools, and then shift your fraud management portfolio toward unsupervised methods; if they produce the same results as supervised algorithms, you will save on supervised machine learning training costs.
■ Ask for visibility into machine learning algorithms. The whole point of using machine
learning is to be able to avoid black box fraud risk scoring. No matter how complex, a machine learning algorithm should be understandable and customizable to your analysts and data scientists. While vendors will offer their extensive professional services for model creation and updates, it’s much less costly to quickly become operationally self-sufficient in this area.
■ Insist on free proofs of concept to get a taste of implementation complexity. Many
vendors don’t offer free proofs of concept in EFM with machine learning algorithms. They claim that their solution has been proven to work with many previous clients, and therefore, proof of concept is unnecessary. In many instances, this behavior implies that extensive customization and coding is required to get the solution off the ground. Any vendor claiming to have a working, “shrink-wrapped” EFM with machine learning platform should be able to stand up a simple proof-of-concept environment in two to three weeks.
■ Pool data within and across banks. Machine learning algorithms feed on data. S&R and
fraud management professionals need to work with network security, marketing, finance, and other departments within the company to create an integrated data warehouse that can feed and improve machine learning-based fraud management. Smaller eCommerce companies and banks can also benefit from sharing anonymized and encrypted transaction information, whitelists, and blacklists with each other to build higher quality input data for machine learning algorithms.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
15
Stop Billions In Fraud Losses With Machine Learning
W h at I t M eans
Machine Learning Will Extend To All Types Of Transaction Fraud Retail banking already uses many of the above machine learning techniques to prevent credit card fraud and is slowly extending to other forms of transactional fraud such as payments and checks. The larger volume of data from the online and mobile channel is stimulating demand for online, real-time, adaptive learning solutions. In the future, we expect that machine learning will:
■ Expand into corporate banking, wealth management, and investment trading. The risks here are so large that banks are nervous about being left behind. Unlike retail banking, the levels of known fraud incidents are low in number (although high in value); hence, supervised techniques are in use and the focus is on unsupervised approaches where the goal is to find interesting patterns in the data without knowing in advance what to look for.
■ Become a tool for fraudsters too. Fraudsters are employing technologically sophisticated
systems: They are using distributed systems, internal knowledge, big data, and even machine learning to detect weak spots and to discover ways to maximize their attacks. Old-school defenses are not “the safer bet”; in fact, they are anything but. Security teams and dedicated fraud teams that want to defend themselves against fraud need to have a solution that is better than their industry average, because fraud flows to the weakest points. In addition, your teams need to constantly evolve their solutions because fraud patterns change quickly and you need to have solutions that abstract away the complexity of their software antifraud solutions (arm fraud analysts with power of data science).
■ Migrate to the cloud for faster adoption and greater effectiveness. The on-premises EFM
transactional monitoring systems of the past live in a semivacuum: They can only learn from transactions, fraud, and analyst decisions that affect the firm where they are implemented. Cloud-based EFM solutions of the future can share machine learning algorithm parameters, best practices, and hotlists and inform all solution clients proactively of new fraudulent activity — without the need for a model update. Collaborating (instead of competing) on security and fraud by sharing data securely and privately will benefit all banks and eCommerce companies.
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
16
Stop Billions In Fraud Losses With Machine Learning
Supplemental Material Companies Interviewed For This Report Accertify
Kaspersky Lab
ACI
RSA
BAE Systems
SAS
CA Technologies
Skytree
Feedzai
ThreatMetrix
IBM
WorldPay
Endnotes Sources: “2014 LexisNexis True Cost of Fraud Study,” LexisNexis, August 2014 (http://www.lexisnexis. com/risk/downloads/assets/true-cost-fraud-2014.pdf) and “2014-2015 Online Fraud Management Benchmark Study,” CyberSource, 2014 (http://www.cybersource.com/resources/collateral/Resource_Center/ whitepapers_and_reports/CYBS-Fraud-Benchmark-Report.pdf).
1
2
For more information about the projected US and Latin American eCommerce market, see the “US eCommerce Forecast: 2013 To 2018,” Forrester report and see the “Latin America eCommerce Forecast, 2014 To 2019” Forrester report.
Big data will only increase the effectiveness of fraud management and regulatory compliance and in turn, directly improved the overall customer experience. For more information, see the “Big Data In Fraud Management: Variety Leads To Value And Improved Customer Experience” Forrester report.
3
Companies continue to lose money due to fraud issues such as chargebacks, uncoverable transfers, and time-intensive investigation on fraudulent transactions. For more information, see the “Market Overview: Fraud Management Solutions” Forrester report.
4
For example, consider this scheme. Fraudster: 1) steals a person’s bank account number and other personally identifiable information; 2) calls the bank contact center and establishes an online presence for the victim’s account; 3) changes the address on the account online; 4) calls into the call center and orders a new ATM/ debit card; and 5) empties the account at an ATM. If S&R and fraud management professionals only look at the phone channel, this fraud scheme is not detectable.
5
Source: “Machine learning,” WhatIs.com (http://whatis.techtarget.com/definition/machine-learning).
6
Source: “COS 511: Theoretical Machine LearningMachine learning,” Wikipedia Princeton University (http://www.cs.princeton.edu/courses/archive/spr08/cos511/scribe_notes/0204.pdfhttp://en.wikipedia.org/ wiki/Machine_learning).
7
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
For Security & Risk Professionals
17
Stop Billions In Fraud Losses With Machine Learning
Source: “How Accertify helped easyJet use fraud screening to cut their fraud loss on revenue by 39%,” Accertify (http://www.accertify.com/Publications/Case-Studies/easyJet1/).
8
Source: Rich Caruana and Alexandru Niculescu-Mizil, “An Empirical Comparison of Supervised Learning Algorithms,” Cornell University, Department of Computer Science (http://www.cs.cornell.edu/~caruana/ ctp/ct.papers/caruana.icml06.pdf).
9
For a more detailed overview of machine learning algorithms, visit the following URL. Source: “Basics of Machine Learning,” The University of Edinburgh School of Informatics (http://homepages.inf.ed.ac.uk/ vlavrenk/iaml.html).
10
For more information on decision trees, visit the following URL. Source: “Decision Trees,” MindTools (http://www.mindtools.com/dectree.html).
11
Kernel methods owe their name to the use of kernel functions, which enable them to operate in a highdimensional, implicit feature space without ever computing the coordinates of the data in that space, but rather by simply computing the inner products between the images of all pairs of data in the feature space. Source: Thomas Hofmann, Bernhard Scholkopf, and Alexander Smola, “Kernal Methods In Machine Learning,” Institute of Mathematical Statistics, 2008 (http://www.kernel-machines.org/publications/ pdfs/0701907.pdf
12
The sharing of data depends on how the contract between the vendor and the issuer is structured.
13
A white-box model means that the vendor exposes and documents how the model works and how model variables impact the algorithm, and this documentation is available to the customer organization.
14
For more information, see the “Quick Take: IBM Announces A Streamlined Fraud Management Portfolio” Forrester report.
15
Source: The R Project for Statistical Computing (www.r-project.org).
16
The vendors claims that its models are one to two orders of magnitude faster than competitors’ models.
17
© 2015, Forrester Research, Inc. Reproduction Prohibited
April 6, 2015
About Forrester A global research and advisory firm, Forrester inspires leaders, informs better decisions, and helps the world’s top companies turn the complexity of change into business advantage. Our researchbased insight and objective advice enable IT professionals to lead more successfully within IT and extend their impact beyond the traditional IT organization. Tailored to your individual role, our resources allow you to focus on important business issues — margin, speed, growth — first, technology second. for more information To find out how Forrester Research can help you be successful every day, please contact the office nearest you, or visit us at www.forrester.com. For a complete list of worldwide locations, visit www.forrester.com/about. Client support For information on hard-copy or electronic reprints, please contact Client Support at +1 866.367.7378, +1 617.613.5730, or
[email protected]. We offer quantity discounts and special pricing for academic and nonprofit institutions.
Forrester Focuses On Security & Risk Professionals To help your firm capitalize on new business opportunities safely, you must ensure proper governance oversight to manage risk while optimizing security processes and technologies for future flexibility. Forrester’s subject-matter expertise and deep understanding of your role will help you create forward-thinking strategies; weigh opportunity against risk; justify decisions; and optimize your individual, team, and corporate performance.
Forrester Research (Nasdaq: FORR) is a global research and advisory firm serving professionals in 13 key roles across three distinct client segments. Our clients face progressively complex business and technology decisions every day. To help them understand, strategize, and act upon opportunities brought by change, Forrester provides proprietary research, consumer and business data, custom consulting, events and online communities, and peer-to-peer executive programs. We guide leaders in business technology, marketing and strategy, and the technology industry through independent fact-based insight, ensuring their business success today and tomorrow. 120912
