F E A T U R E A R T I C L E
Credit Scoring And The Next Step ALBERT FENSTERSTOCK
Albert Fensterstock is Managing Director of Albert Fensterstock Associates, a consulting firm that specializes in the application of the Internet and the utilization of decision support technology for improving credit department risk analysis capability and collection department efficiency. He can be reached at 516.313.1020 or via e-mail at
[email protected].
I
NTRODUCTION
The use of credit scoring for evaluating credit risk is at an all-time high in American businesses. This is probably due to several factors inherent in the use of this technology. Among them, credit scoring is more efficient than having credit department personnel evaluate every potential sale, which saves money—a major consideration for corporate America these days. Additionally, Sarbanes-Oxley and its requirements point to the use of credit scoring as a good method for improving internal control over risk-based decisions. For the most part, however, regardless of the type of scoring system in place, businesses are not getting everything they can from the implementation of a credit scoring system. So, there is more that can be done, however, before we get into what it is, and why it should be done, let’s briefly review what types of credit scoring systems are available and the differences between them. If the reader needs more than a brief review, you may want to take a look at an article I wrote, Credit Scoring Basics, that appeared in the March 2003 issue of Business Credit. This article covers the nature of a credit scoring model, credit scorecards, the requirements for maintaining a credit scoring system, and how to determine whether to build or buy, subjects that will not be covered here.
What Is Credit Scoring?
BUSINESS CREDIT MARCH 2005
Essentially, credit scoring is a systematic method for evaluating credit risk that provides a consistent analysis of the factors that have been determined to cause or affect the level of risk. Factors are usually determined through an analysis of historical bill payment activity as well as various descriptive parameters that may be used to classify an account into one of several defined categories or customer classes. Once the analysis necessary to create the credit scoring model—or in most instances models—has been accomplished, information about an applicant company and its credit experiences are entered into the model, when a credit decision is needed. Then, credit scoring software, usually provided by a company that specializes in this type of analysis, helps to predict:
46
• Whether the debtor company is likely to pay its debts on time, and/or • The likelihood the debtor company will file for bankruptcy or default on the debt.
What Are The Different Types Of Credit Scoring Systems? The different types of credit scoring systems most usually used to evaluate credit risk in B2B situations are: • Judgmental/Rules-Based Systems • Neural Network-Based Systems • Statistical-Based Systems • Genetic Algorithm-Based Systems They all have strengths and weaknesses and the nature of your business, the amount of risk you are willing to take, and the funds available to develop and implement the scoring system will play a major part in determining which technology is best for your company to employ. To be sure that you employ the one that is best for your company, you should review all of the options before you make a decision.
Judgmental/Rules-Based Systems Judgmental/rules-based systems evaluate credit worthiness using a formula or a set of rules based upon internal and external credit experience. The determination of the rules used and their associated values is a manual process that is dependent upon certain individuals’ intuition and past experience. Some of the strengths of this procedure are: • It is based on accepted credit risk standards (judgment) and takes such items into consideration as: - Customer payment history - Bank and trade references - Credit agency ratings - Financial statements - Various financial ratios • Past experience is the basis of the factors and weights used • Provides a consistent alternative for evaluating new credit applications Some of its weaknesses are:
F E A T U R E
• It is inefficient to build as all of the work is done manually • The factor weights can be biased towards data elements that may not be mathematically relevant to risk • The effect of a specific factor can not be accurately measured and, therefore, the sensitivity to a change in that factor can not be accurately determined • Risk cannot be quantified as expressed by the probability of payment within a specific time period or of default • It is difficult, if not impossible, to determine the source of error if the system’s predictions are not accurate, therefore, updating the system in an effort to improve results is a hit and miss proposition
Neural Network-Based Systems These systems utilize artificial intelligence algorithms applied to historical data to find relationships between account characteristics and the probability of default. They have the capability to classify accounts into various credit risk classes such as: good, indeterminate, delinquent, charge-off and bankrupt.
A R T I C L E
• Can deal with non-linear relationships • Correlations between factors are accounted for • Models are adaptive in that the nature of forecast errors (i.e., bad model decisions) can be easily evaluated and the models improved accordingly Some of its weaknesses are: • Results are essentially a “black box” • Credit personnel have no idea of the structure of the credit scoring model, i.e., the weights connecting nodes between model layers and, therefore, cannot properly evaluate the model’s decisions • Model output must be accepted by credit personnel based on the word of an “expert”, i.e., the consultant who developed the model, who is rarely a competent credit analyst. • Almost all applications in the credit industry have been in consumer-based risk evaluation
Statistical-Based Scoring Systems Some of the strengths of this procedure are: • Has the ability to determine the characteristics that are most important in predicting credit risk • In some circumstances, may be more flexible than standard statistical techniques as no assumptions are made about the relationships of the risk factors prior to analysis
NACM’S
GOVERNMENT BUSINESS GROUP
Statistical-based scoring systems utilize statistical analysis, usually in the form of a multivariate non-linear regression model, (remember Y = a + bX, only with more variables, hence multivariate), to estimate the probability a customer will default or become delinquent. These systems are very similar in operation to judgmental systems except that the factors used and their assigned weights are based on statistical analysis—
PRESENTS
Don’t Gamble on Your Government Business! APRIL 20 - 22, 2005 GOLDEN NUGGET • LAS VEGAS, NV Learn the most effective ways to sell and get paid (ON TIME!) by the Federal Government. • Intensive training sessions • Invaluable networking opportunities • Government experts with answers
Register now to take advantage of early bird discounts!
47
BUSINESS CREDIT MARCH 2005
Strength in Numbers
F E A T U R E
A R T I C L E
not human judgment—of a company’s past operations together with other data considered by the analyst as pertinent. Some of the strengths of this procedure are: • Can use and evaluate any internal and external data to score accounts • Many factors are considered individually and simultaneously • The model development process calculates and analyzes the correlation between all of the variables to identify factor tradeoffs and eliminate redundant relationships • Variables can be chosen by the statistical analyst to ensure the relationships makes financial sense (e.g., older firms are less risky then younger firms) • Users are able to identify sources of estimation error and improve model accuracy Some of its weaknesses are: • Need someone with a statistics background and experience with credit information to build and implement the model • Bulk of analyst’s time may be spent in data preparation and analysis of statistical relationships, rather than in model building • If there are a lot of variables to consider, an analyst may need to predetermine the important variables based on a separate analysis • Some statistical models can be difficult to implement
• Extremely limited knowledge of this technology within the business community, almost all practitioners are university-based • The few business-based practitioners have little experience in applying GAs to credit risk analysis • In-house statisticians may resist the implementation of a technology they do not know or understand
Why Use Scientific-Based Credit Scoring? With respect to neural, statistical and genetic algorithmbased credit scoring systems, the main reasons for using one of them are: • They provide a significantly more accurate analysis by delivering 10 percent to 30 percent—or more—improvement in predicting customer credit risk over non-scientific methods • The cause of prediction error in scientific-based models can be identified and these models can be improved and kept current. Updates can be scheduled on a regular basis, or as needed • They can provide the basis for meeting certain requirements of Sarbanes-Oxley Providing the work discussed in the next section is accomplished: • They can help in the development of optimal strategies for dealing with at-risk account. • Scientific-based credit scoring can dramatically increase credit and collection department productivity
Genetic Algorithm-Based Systems
The Next Step
This is the new guy on the block and may offer, in certain situations, a superior solution to any other method. Genetic Algorithms (GA) are based on the principle of “survival of the fittest” ala Darwin. Like selective breeding, a GA breeds an initial “generation” of random predictive models. The models are tested for fitness against user-defined criteria. The better models are more likely to be selected for breeding. Following fitness evaluation, the GA will apply the basic principles of genetics to breed the next generation of models. These consist of cloning, mating, mutating and the introduction of random models.
We submit that the implementation of a scientific-based credit scoring system is only the first step in improving credit and collection department efficiency. To really achieve the maximum benefits available from these technologies you need to go the next step, and sadly very few companies have.
An extensive “trial and error” process is utilized that may breed millions of models resulting in a final model that can be used to estimate the probability of payment within a given time horizon. Basically, GAs are very sophisticated search algorithms that by means of an iterative procedure search through all of the possible scoring models in an attempt to locate the best scoring model.
Consider the obvious. Credit scores in a given B2B environment will range from very low (high risk) to very high (low risk). Yet, in most instances, the collection procedures applied within a company are very much the same for all accounts shipped, until an account starts falling behind in their expected payments. Then, the collectors spring into action to do that which they should have expected they would have to do at the time the account was shipped. We propose a different strategy, whereby a given score predetermines the collection strategy to be applied from the time of shipment. If different scores indicate a different level of risk, doesn’t it make sense that they also imply that different collection procedures be utilized?
Some of the strengths of this procedure are:
BUSINESS CREDIT MARCH 2005
A Procedure For Achieving The Next Step • Can use and evaluate any internal and external data to score accounts • GAs can use 100% of the available data rather than an analyst selected subset • The interaction between variables is not a problem • As with statistical-based systems, users are able to identify sources of estimation error and improve model accuracy Some of its weaknesses are:
48
Assume that the new scientific-based scoring system has been developed and implemented; here is one approach to finishing the job: 1. Determine a set of collection procedures that might range from mild to very aggressive. Let’s assume that you have defined three different strategies, where one of the strategies is what you currently do.
F E A T U R E
2. Score all of your accounts using the new system, and classify them into homogeneous groups or strata. If we assume scores can range from 0 to 100, then perhaps one strata contains all of the accounts that scored from 0 – 25; a second strata might contain all of the accounts that scored from 26 – 50, and so on. You may also want to segment the strata if there are different divisions, i.e., even though accounts have the same score there is another characteristic that you believe makes them different; such that they might respond differently to the same collection strategy. 3. Within each stratum, select a random sample of some percentage of the accounts. The size of the sample can be determined based upon the variance within the strata. I would suggest that this variance be based upon a discounted cash flow (DCF) analysis of the accounts past payment history converted into a percentage of the maximum DCF possible, where the maximum DCF occurs if the invoice is paid at the time of shipment. Realistically, this is the statistic you are trying to improve in that it measures the time-value of money. The sooner after shipment an invoice is paid; the greater the percentage of the maximum possible DCF is achieved. Your credit scoring systems provider can help you with this. 4. Next, divide the sample within each stratum into three groups. You can do this by assigning every third account to a different group. For one of the groups, within each stratum, apply your normal collection procedures. For each of the other
A R T I C L E
groups, apply one of the other procedures you determined in 1. above. 5. Ninety to 120 days after you have started the test, compute the percentage of DCF achieved for each group and compare it to the historical group average. You can also compare the percentage DCF on an account-by-account basis. There is a standard statistical test you can utilize to perform this comparison that will determine whether the difference between your history and the current results are significant, i.e., the alternative procedures have produced a measurable change. 6. If the alternative collection procedures have produced a meaningful change in the bill payment activity of certain classes of accounts, you’ll need to consider whether you should change your collection activities relative to all of the types of accounts where a more favorable percentage of DCF has been realized. A simple rule-based program that can determine the type of collection procedure required on an account-by-account basis and then direct the personnel assigned to these accounts how and when to handle them can be readily designed and implemented into an inhouse or ASP based computer system. If you want to achieve all of the benefits promised by scientific-based credit scoring, an analysis similar to the one described above should be part of any credit scoring project. While this procedure may seem a little complicated, anybody with a basic knowledge of statistical sampling should be able to help you.
Strength in Numbers
–Expanded Knowledge –Career Opportunities –Heightened Professional Recognition
Professional Certification– A Recognized Success Path For details visit www.nacm.org or call 410.740.5560.
The Right Way 49
BUSINESS CREDIT MARCH 2005
–Standards of Excellence
