Analytics Magazine

H T T P : / / W W W. A N A LY T I C S - M A G A Z I N E . O R G

DRIVING BETTER BUSINESS DECISIONS

JANUARY / FEBRUARY 2014 BROUGHT TO YOU BY:

ANALYTICS CAREERS & CONSULTING • Eighteen things nobody tells you about solo practice • Certification: What it means for employers, practitioners • Analytics-driven culture: Why it’s a corporate no-brainer ALSO INSIDE: • Predictive analytics in the cloud • Analytics & health management

Special Supplement: CAP Candidate Handbook

• Dealing with missing values in data Executive Edge Verisk Innovative Analytics President Marty Ellingsworth on the future, big data and bigger analytics

INS IDE STORY

Big dreams, small data Like everyone else involved in the analytics space, we’ve been yapping endlessly about “big data” in this column. You all know the story – unfathomable amounts of data coming in from multiple sources at incredible speed have analysts everywhere scrambling to make sense of it all. Let’s face it, big data is the elephant in the room in any discussion of analytics, and the elephant is only going to get bigger (think hybrid data, including video, images, sound, text, etc. from countless sources and sensors). But wait, there’s more; there’s a “small” angle to the “big data” story. Even in the Big Data Era, many companies do not have the data they need to make data-based decisions. A start-up, for example, almost certainly does not have the historic data that an established firm has collected. Even well-established companies probably lack the data they need when considering introducing a new product or service or entering a new market. With that in mind, Analytics magazine will launch a new column by Brian Lewis in the March/April issue that will address the issue of insufficient data and how to overcome it. The name of the column: “Big Data Dreams, Small Data Reality.” Chew on that concept for a minute. Lewis, chief data scientist and co-founder of Fractal Sciences, provides more details in an introductory column in this issue. 2

|

A N A LY T I C S - M A G A Z I N E . O R G

Of course, big data remains the big fish in the analytics pond, so we’ll continue to cover it and all of its ramifications. For example, in this issue’s Executive Edge column, Marty Ellingsworth, president of Verisk Innovative Analytics, discusses the “promise of big data and bigger analytics” that “will drive the future” as the corporate world shifts from a company-centric to a customer-centric culture. Meanwhile, INFORMS, publishers of Analytics magazine and the world’s leading organization for high-end analytics, will present its inaugural INFORMS Conference on Big Data in San Jose, Calif., June 22-24. The conference will focus on the business of big data and making the journey from data-rich to decision-smart. For a preview of the conference, click here. The issue also includes a couple of “career-builder” feature articles that should pique the interest of any analytics professional looking to get an edge in a competitive environment. Veteran analyst Doug Samuelson outlines some of the consulting lessons he’s learned the hard way, while Polly MitchellGuthrie and Scott Nestler give an update on INFORMS’ Certified Analytics Professional program and how it can help employers and clients of analytics professionals, as well as analytics professionals themselves.

– PETER HORNER, EDITOR peter.horner@ mail.informs.org W W W. I N F O R M S . O R G

OPTIMIZE YOUR BUSINESS WITH UNPRECEDENTED SPEED IDEA

IN A FEW HOURS

MISSION CRITICAL ENTERPRISE APP

IN A FEW MONTHS

PUBLISHED INSTANTLY TO YOUR ENTERPRISE OPTIMIZATION APP STORE

PROOF OF CONCEPT

IN A FEW DAYS

OPTIMIZATION APP

IN A FEW WEEKS

To learn more about AIMMS Optimization Apps, visit aimms.com. [email protected] | +1 425 458 4024

C O N T E N T S

DRIVING BETTER BUSINESS DECISIONS

JANUARY/FEBRUARY 2014 Brought to you by

FEATURES 30

PREDICTIVE ANALYTICS IN THE CLOUD By James Taylor Research survey: Ability to deliver ROI solutions more cost-effectively is driving cloud deployment.

36

ADVENTURES IN ANALYTICS CONSULTING By Doug Samuelson Eighteen things nobody tells you about solo practice that you need to know before you take the plunge.

44

CERTIFIED ANALYTICS PROFESSIONAL By Polly Mitchell-Guthrie and Scott Nestler Usage guide for employers and clients answers key questions about first-of-its kind CAP® program.

52

ANALYTICS & HEALTHCARE By Rajib Ghosh Apply predictive analytics to address population health management and medication adherence problems.

58

MISSING VALUES By Gerhard Svolba The origin, detection, treatment and consequences of missing values in analytics.

36

44

SPECIAL SUPPLEMENT Certified Analytics Professional candidate handbook

52 4

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

N Di ew m AS en P sio V2 na 01 lM 4w in excel ode ith lin g

AnAlytic Solver PlAtform easy to Use Predictive and Prescriptive Analytics

How can you get results quickly for business decisions, without a huge budget for “enterprise analytics” software, and months of learning time? Here’s how: Analytic Solver Platform does it all in Microsoft Excel, accessing data from PowerPivot and SQL databases. Sophisticated Data Mining and Predictive Analytics Go far beyond other statistics and forecasting add-ins for Excel. Use classical multiple regression, exponential smoothing, and ARIMA models, then go further with regression trees, k-nearest neighbors, and neural networks for prediction, discriminant analysis, logistic regression, k-nearest neighbors, classification trees, naïve Bayes and neural nets for classification, and association rules for affinity (“market basket”) analysis. Use principal components, k-means clustering, and hierarchical clustering to simplify and cluster your data.

Help and Support to Get You Started Analytic Solver Platform can help you learn while getting results in business analytics, with its Guided Mode and Constraint Wizard for optimization, and Distribution Wizard for simulation. You’ll benefit from User Guides, Help, 30 datasets, 90 sample models, and new textbooks supporting Analytic Solver Platform. Analytic Solver Platform goes further than any other software with Active Support that alerts us when you’re having a problem, and brings live assistance to you right where you need it – inside Microsoft Excel. Find Out More, Download Your Free Trial Now Visit www.solver.com to learn more, register and download a free trial – or email or call us today.

Simulation, Optimization and Prescriptive Analytics Analytic Solver Platform also includes decision trees, Monte Carlo simulation, and powerful conventional and stochastic optimization for prescriptive analytics.

Tel 775 831 0300 • Fax 775 831 0314 • [email protected]

DRIVING BETTER BUSINESS DECISIONS

REGISTER FOR A FREE SUBSCRIPTION: http://analytics.informs.org

66

76

DEPARTMENTS 2 Inside Story 8 Executive Edge 14 Analyze This! 18 Forum 22 Big Dreams, Small Data 26 INFORMS Initiatives 66 Conference Previews

INFORMS BOARD OF DIRECTORS President Stephen M. Robinson, University of Wisconsin-Madison President-Elect L. Robin Keller,
University of California, Irvine Past President Anne G. Robinson, Verizon Wireless Secretary Brian Denton, University of Michigan Treasurer Nicholas G. Hall, Ohio State University Vice President-Meetings William “Bill” Klimack, Chevron Vice President-Publications Eric Johnson, Dartmouth College Vice President Sections and Societies Paul Messinger, CAP, University of Alberta Vice President Information Technology Bjarni Kristjansson, Maximal Software Vice President-Practice Activities Jonathan Owen, General Motors Vice President-International Activities Grace Lin, Institute for Information Industry Vice President-Membership and Professional Recognition Ozlem Ergun, Georgia Tech Vice President-Education Joel Sokol, Georgia Tech Vice President-Marketing, Communications and Outreach E. Andrew “Andy” Boyd, University of Houston Vice President-Chapters/Fora David Hunt, Oliver Wyman

INFORMS OFFICES www.informs.org • Tel: 1-800-4INFORMS Executive Director Melissa Moore Meetings Director Laura Payne Marketing Director Gary Bennett Communications Director Barry List Headquarters INFORMS (Maryland) 5521 Research Park Drive, Suite 200 Catonsville, MD 21228 Tel.: 443.757.3500 E-mail: [email protected]

72 Five-Minute Analyst 76 Thinking Analytically Analytics (ISSN 1938-1697) is published six times a year by the Institute for Operations Research and the Management Sciences (INFORMS), the largest membership society in the word dedicated to the analytics profession. For a free subscription, register at http://analytics.informs.org. Address other correspondence to the editor, Peter Horner, [email protected]. The opinions expressed in Analytics are those of the authors, and do not necessarily reflect the opinions of INFORMS, its officers, Lionheart Publishing Inc. or the editorial staff of Analytics. Analytics copyright ©2014 by the Institute for Operations Research and the Management Sciences. All rights reserved.

6

|

ANALYTICS EDITORIAL AND ADVERTISING Lionheart Publishing Inc., 506 Roswell Street, Suite 220, Marietta, GA 30060 USA Tel.: 770.431.0867 • Fax: 770.432.6969 President & Advertising Sales John Llewellyn [email protected] Tel.: 770.431.0867, ext. 209 Editor Peter R. Horner [email protected] Tel.: 770.587.3172 Art Director Jim McDonald [email protected] Tel.: 770.431.0867, ext. 223 Advertising Sales Sharon Baker [email protected] Tel.: 813.852.9942

N Di ew m AS en P sio V2 na 01 lM 4w od ith excel eli ng

AnAlytic Solver PlAtform from Solver to full-Power Business Analytics in

The Excel Solver’s Big Brother Has Everything You Need for Predictive and Prescriptive Analytics From the developers of the Excel Solver, Analytic Solver Platform makes the world’s best optimization software accessible in Excel. Solve your existing models faster, scale up to large size, and solve new kinds of problems. From Linear Programming to Stochastic Optimization Fast linear, quadratic and mixed-integer programming is just the starting point in Analytic Solver Platform. Conic, nonlinear, non-smooth and global optimization are just the next step. Easily incorporate uncertainty and solve with simulation optimization, stochastic programming, and robust optimization – all at your fingertips.

Comprehensive Forecasting and Data Mining Analytic Solver Platform samples data from Excel, PowerPivot, and SQL databases for forecasting and data mining, from time series methods to classification and regression trees, neural networks and association rules. And you can use visual data exploration, cluster analysis and mining on your Monte Carlo simulation results. Find Out More, Download Your Free Trial Now Analytic Solver Platform comes with Wizards, Help, User Guides, 90 examples, and unique Active Support that brings live assistance to you right inside Microsoft Excel. Visit www.solver.com to learn more, register and download a free trial – or email or call us today.

Ultra-Fast Monte Carlo Simulation and Decision Trees Analytic Solver Platform is also a full-power tool for Monte Carlo simulation and decision analysis, with a Distribution Wizard, 50 distributions, 30 statistics and risk measures, and a wide array of charts and graphs.

Tel 775 831 0300 • Fax 775 831 0314 • [email protected]

EXE CU TIVE E D G E

How analytics will drive the future Some businesses, afraid of taking on new risks, stick to their own market niches and avoid new ones – preventing their abilities to be competitive in the larger marketplace. Others are blazing trails using newer technologies, data sources, modeling approaches and electronic connectivity to better manage risks in an increasingly mobile world.

BY MARTY ELLINGSWORTH

8

|

Whether you’re steering an enterprise, championing an analytics program, driving a venture capitalfunded data-modeling product or piloting your own consulting practice, there is no way you have not become aware of the promise of big data and bigger analytics. Companies often deal with an opaque marketplace where riskiness is rated less accurately in some organizations than others. That said, some businesses, afraid of taking on new risks, stick to their own market niches and avoid new ones – preventing their abilities to be competitive in the larger marketplace. Others are blazing trails using newer technologies, data sources, modeling approaches and electronic connectivity to better manage risks in an increasingly mobile world. The following framework helps identify how companies are structured to compete on analytics. SPECTRUM OF PREDICTIVE ANALYTICS CAPABILITIES Now we’re at the beginning of a long rally race of analytic improvements – from newer, better data to smarter, faster algorithms. Enhancements will include broader, more scalable platforms and will access

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

New Solver for Office 365 Excel: Free for Your Tablet or Phone New

With our Solver App for Office 365, SharePoint 2013, and Excel 2013, you have all the capabilities of the Excel Solver on your desktop, laptop, tablet or phone. It works in the Excel Web App with all popular Web browsers, and solves your model “in the cloud” on Windows Azure. And it’s free and available now! Just visit solver.com/app or the Office Store online, or use Insert Apps for Office in Excel 2013.

Analytic Solver Platform: Multidimensional Models with PivotTables New

Analytic Solver Platform 2014 brings multi-dimensional optimization modeling to Excel. Easily create dimensions or index sets and cubes of computed values, using regular Excel formulas – extended to operate over multiple dimensions. Use PivotTables, created in Excel or from databases with PowerPivot, to populate your model with data. Easily create new PivotTables of optimization results.

Find Out More, Download Your Free Trial Now. Visit www.solver.com to learn more, register and download a free trial – or email or call us today. Frontline Solvers – The Leader in Spreadsheet Analytics

Tel 775 831 0300 • Fax 775 831 0314 • [email protected]

EXE CU TIVE E D G E

Now we’re at the beginning of a long rally race of analytic improvements – from newer, better data to smarter, faster algorithms. Enhancements will include broader, more scalable platforms and will access unique sensors – spectra, spatial, temporal – and micro/macro levels of structured and unstructured data.

10

|

unique sensors – spectra, spatial, temporal – and micro/macro levels of structured and unstructured data. All that will help generate insights into individualized, massively personalized and localized information while getting even more power out of grouped predictive parameters. For example, if you have operated a car or other moving object, you undoubtedly have been assessed for your risk of loss as an operator of that vehicle and, in some manner, likely have been insured. In the past, that insurance-based risk assessment has blended wide bands of information on a few historically available generic characteristics to achieve a generalpurpose estimate of prospective loss risk estimates. In the future, that historical benchmark will be segmented into ever-more granular and accurate assessments. Those will then again be reinvented, recombined and refined to enhance the data-driven process, culminating in an adaptive analytic that adjusts expectations to the level of risk in each operating scenario encountered or intuited. In the world of big data and bigger analytics, insurers will come to view vehicles as instrumented platforms and operators as real-time learners whose risk may change over time. Operators may drive safer and make smarter decisions about moving between locations, or they may permit distractions into their cockpit (such as texting, talking, smoking, eating and so on). How you drive, when you drive, where you drive and how much you drive are all becoming part of the context in your individual risk profile. Some businesses apply detailed telemetry, routing algorithms, real-time weather and traffic alerts and driver/crew pairing models to manage more

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Figure 1: Companies vary widely in their abilities to create and use predictive information. There are seven stages of development for predictive analytic capabilities, and each has a level of investment and an expected return. The companies with the most mature capabilities will have invested in all seven stages shown in the illustration and, depending on individual jurisdictional restrictions, will have deployed analytic models to serve their customers and compete for others. effectively the logistics of moving people, packages and pallets. Similarly, individual consumers make daily choices to move themselves, their passengers and their belongings along the same roadways and flyways and use all sorts of new navigation and alerting applications and devices to do so. (I’ve seen a mobile tablet computer go from a plane to a car to a sofa all in the hands of the same individual within one morning.) Peering into the future, if a submersible helicopter car becomes commonplace, we’d have a truly three-dimensional A NA L Y T I C S

driving experience. And on those journeys, we might need to dodge Amazon’s Octocopter self-driving delivery micro bots along the way. In the ubiquity of an instrumented world, such a trend is unstoppable. Our challenge will be how we will use analytics to interact with decision-making. If consumers continue to make their own decisions, it’s a certainty that marketing analytics, advertising effectiveness and brand campaigning will merge into mobile and content messaging more than ever before. The next frontier will involve J A N U A R Y / F E B R U A R Y 2 014

|

11

EXE CU TIVE E D G E

The paradigm shift transitions from companycentric to customer-centric and from “we always have done ‘IT’ this way” to real time. That shift must be the focus of top management, which needs to take the offense and drive resource allocation for innovation and productivity to a customer-focused, realtime strategy.

the layered sensing of the temporal and spatial context surrounding the customer. Decisions that address emotional desires of customers resonate in the behavioral economics that underpin our financial world. The closer we can come to a customer’s desires, the better – and better still to be able to influence demand by making customers aware of opportunities they did not know exist. That holds true for business-to-business decisions as well. The paradigm shift transitions from companycentric to customer-centric and from “we always have done ‘IT’ this way” to real time. That shift must be the focus of top management, which needs to take the offense and drive resource allocation for innovation and productivity to a customer-focused, real-time strategy. Executives who embrace such a process of optimization that both considers maximizing enterprise performance while minimizing risks will effectively revitalize every decision opportunity in marketing, production, distribution, logistics, operations, servicing and sales. And they will find there is no finish line when generating more shareholder value – only a continual cycle of improvement and a corporate culture of datadriven, sustainable excellence. Marty Ellingsworth is president of Verisk Innovative Analytics, a division of Verisk Analytics (www.verisk.com). Verisk Innovative Analytics is a member of the INFORMS Roundtable, and the author is a long-time member of INFORMS.

Join the Analytics Section of INFORMS

For more information, visit: http://www.informs.org/Community/Analytics/Membership

12

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Explore and solve complex problems for FREE FICO® Xpress Optimization Suite… FREE to Academic Institutions and Their Students Solve enormously complex problems, faster Xpress can handle complex problems with millions of variables and constraints. You’ll quickly narrow down choices to the very best (or the N-best) solution when there are innumerable possible solutions that would otherwise be difficult to compare. Explore “What If” Scenarios Xpress offers improved solution sensitivity analysis, making it possible to efficiently explore larger quantities of “what if?” scenarios. It finds the strongest levers by adjusting variables and constraints to measure how far they move optimal operating points. Take advantage of unique features for large scale optimization Xpress offers true 64-bit support for optimization and modeling, allowing users to solve ultra-large-scale models with more than 2 billion coefficients. Learn more about the Xpress Academic Partner Program: fico.com/app © 2014 Fair Isaac Corporation. All rights reserved.

ANALY ZE T H I S !

What is ‘real’ analytics? Preparations for a career in analytics should be built on a three-legged stool of computing skills, analytic capabilities and business effectiveness skills.

BY VIJAY MEHROTRA

14

|

A member of our MBA Program Task Force was talking about recent alums who had been successful on the job market, and early in her discussion, she cited the students “who had, you know, taken Vijay’s classes.” This did seem a little weird – my classes have course numbers and names, after all – but we were all in the midst of a very busy semester, and so I happily let it go. A couple of weeks later, when an MBA staff person came by my office to propose adding a section of one of my MBA electives, she mentioned the great demand for “classes in my area.” I suggested that we simplify things by just referring to them as analytics courses (while my department’s name has changed almost annually, the word “analytics” has always been part of it). She responded equivocally, and looked terribly uncomfortable doing so. Then, just before the holidays, I arrived a few minutes late to a meeting of the Graduate Programs Committee (I was giving a final exam that ran slightly over), expecting to present my proposal for a new MBA course in data mining. However, as I organized my handouts, a colleague seated nearby informed me with a chuckle that my new “non-analytics” course had already been approved. I wondered: “Why all this weird verbal tap dancing?” Well, after some digging around, I got an answer, though it was not a very satisfying one. During the last academic year, my school had launched a new Master of Science in Analytics (“MSAN”) program. The administrator who owned the program had apparently sought to differentiate the content of his program by explaining to

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

anyone who would listen that the courses that we teach to MBAs are not “real” analytics courses, since these classes do not require any computer programming (outside of the Excel environment, which is viewed pejoratively by techies) and do not delve deeply into the algorithmic details behind techniques such as optimization, regression or cluster analysis. This is just ridiculous. First of all, in this kind of rhetorical response, one is required by custom to provide a definition, and mine comes from Davenport and Harris’ book, “Competing on Analytics”: Analytics, they state, is “the extensive use of data, statistical and quantitative analysis, explanatory and predictive models, and fact-based management to drive decisions and actions.” Based on this definition, it is clear that the skills needed for successful analytics professionals are both broad and deep. George Roumeliotis, an analytics leader at Intuit, believes that a good data scientist needs to be a skilled business consultant who also has a broad array of technical skills for data management, analysis and modeling [1]. What this means is that preparations for a career in analytics should be built on a three-legged stool of computing skills (including the ability to gather, merge, clean and manage data), analytic capabilities (with a special emphasis on basic probability and statistics, data mining, dimensionality A NA L Y T I C S

reduction methods and fundamentals of optimization) and business effectiveness skills (such as leadership, problem framing, teamwork, project management, communication skills and negotiation). Any academic program that purports to be focused on preparing students for a career in analytics must strive to address each of these three competencies in some meaningful way, though there are an infinite number of ways to combine each of these somewhat orthogonal vectors. While I was thinking about all this, I came across a blog entry on Forbes.com entitled “Business Analytics Beyond BI: Rise of the MBAs” [2]. The author, John Furrier, is a tech industry veteran and the founder of the website SiliconAngle. com, which pays an awful lot of attention to analytics and Big Data [3]. Though this relatively short article covered a lot of ground, a handful of interconnected “money quotes” caught my eye: 1. “Every department within a company today is itching to apply data-driven systems to their workloads.” What he’s saying here – and what my business school colleagues are slowly starting to understand – is we’re moving toward a time when most professionals will have to be conversant in working with data and interpreting models. We will need to start expecting more of our MBAs in J A N U A R Y / F E B R U A R Y 2 014

|

15

ANALY ZE T H I S !

I do my best to train them to think critically – and wherever possible, to utilize user-friendly tools – to address a variety of data- and model-driven business problems.

16

|

these areas, and to keep innovating to deliver it. 2. “Automation will empower the data scientist to empower everyone else at the company, and they’ll need the help of software.” Automating the data scientist role has been discussed ad nauseam [4], but Furrier has a slightly different take on it: Automation is essential so that these critical human resources can be better focused on changing managerial behaviors, rather than having so much of their time consumed with managing data. 3. “The role of the data scientist plays an important part in setting the tone for collaboration within an organization, as these multidisciplinary problem solvers will need to communicate clearly with each other, as well as every other department.” That is, if the more technically trained analytics professionals can’t work well with less technically trained professionals, an organization’s analytic capabilities will fall far short of their potential. Back here at USF, my colleagues in the MSAN program have made the choice to emphasize the computational and statistical aspects of analytics, which as expected has led to incoming students and outgoing graduates who are suited for very technical roles. My MBA students, however, do not have either the programming skills needed to implement algorithms from the ground up or the inclination to acquire them. Instead, their focus is on the business rather than technology. Instead, I do my best to train them to think critically – and wherever possible, to utilize userfriendly tools, which will be flooding the market for years to come – to address a variety of data- and model-driven business problems, while also working through data quality and management issues as needed. Given the welldocumented talent shortages, it is not surprising that both

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

groups are finding good (though very different) opportunities in today’s marketplace. But let’s be clear: Both of these types of programs (and both of these types of students) are just as worthy of the term “analytics.” And in the future, we can expect that these folks will be working closely together on all sorts of things. Vijay Mehrotra ([email protected]) is an associate professor in the Department of Analytics and Technology at the University of San Francisco’s School of Management. He is also an experienced analytics consultant and entrepreneur,

an angel investor in several successful analytics companies and a longtime member of INFORMS. NOTES & REFERENCES 1. For more from George’s view on what makes a good data scientist, check out http://onlinebehavior.com/emetrics/marketing-metrics-intuit 2. www.forbes.com/sites/siliconangle/2013/12/10/ big-data-beyond-bi-rise-of-the-mbas/ 3. http://siliconangle.com/?s=big+data 4. For example, see http://www.allanalytics.com/ author.asp?section_id=1408&doc_id=251420, http://www.forbes.com/sites/gilpress/2012/08/31/ the-data-scientist-will-be-replaced-bytools/, and http://smartdatacollective.com/ radhikaatemcien/111596/data-scientist-scarcityautomation-answer.

Job Seeker Benefits CAREER CENTER

• Access to high quality, relevant job postings. No more wading through postings that aren’t applicable to your expertise. • Personalized job alerts notify you of relevant job opportunities. • Career management – you have complete control over your passive or active job search. Upload multiple resumes and cover letters, add notes on employers and communicate anonymously with employers. • Anonymous resume bank protects your confidential information. Your resume will be displayed for employers to view EXCEPT your identity and contact information which will remain confidential until you are ready to reveal it. • Value-added benefits of career coaching, resume services, education/training, articles and advice, resume critique, resume writing and career assessment test services.

http://careercenter.informs.org A NA L Y T I C S

POWERED BY

J A N U A R Y / F E B R U A R Y 2 014

|

17

FO RUM

Analytics-driven culture Left-brainers vs. right-brainers: In the Information Age, it is not wise to run a company with half a brain.

Philosophically, business analysis is in its Romantic Era – an era in which analysis is applied hither and yon in a tactical swashbuckling manner. Corporations aspiring to improve their decisionmaking to become more analytics-based will want to foster a more analyticsdriven culture.

BY RANDY BARTLETT

18

|

“In the final analysis, the root cause of Japan’s defeat, not alone in the Battle of Midway but in the entire war, lies deep in the Japanese national character. There is an irrationality and impulsiveness about our people which results in actions that are haphazard and often contradictory.” – Mitsuo Fuchida and Masatake Okumiya [1] Business analysis dissolves in an IT culture and in other cultures too. Philosophically, business analysis is in its Romantic Era – an era in which analysis is applied hither and yon in a tactical swashbuckling manner. Corporations aspiring to improve their decision-making to become more analytics-based will want to foster a more analytics-driven culture. They should seek a culture that: 1. Rewards analytics-based decision-making as in a meritocracy. 2. Integrates analytics into their strategy. 3. Embraces the pace of dynamic change during this analytics phase of the Information Age. 4. Accepts that understanding data analysis is part of understanding the business. 5. Fosters experimentation and continual learning about the business.

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Corporate culture can be defined as “how we do business.” An analyticsdriven culture necessarily blends analytics and company know-how. We can raise the analytics content of the culture by adjusting the leadership, specialization, delegation and incentives [2]. Analytics-driven cultures have built a legacy of seeking great financial opportunities based upon the numbers. They have learned to accept or tolerate the scientific method, plan for analytics, and enable analytics to drive decision-making. They are more deliberate in collecting appropriate data for their decisions. Rather than passively reacting to the data available, their proactive planning includes thinking ahead to seek new types of data that does not yet exist. A crude measure of a corporation’s acceptance of analytics is the extent to which analytics professionals are spread through the company. If a corporation wants to develop a more analytics-driven culture, then it needs to expose people to business analytics and spread analytics professionals throughout the company – growing the culture by spreading the approach. A NA L Y T I C S

LEFT BRAIN–RIGHT BRAIN CULTURAL CLASH The explosion of information implies that we need to apply scientific tools; this is not about pottery or poetry. Left-brain purveyors of the scientific method sound like Mr. Spock or today’s modern icons, Dr. Samantha Carter and Dr. Daniel Jackson of Stargate SG1. Analytics professionals are trained to accept their ignorance, value humility in presenting results and qualify their statements. They are often self-made. At a large bank, a group of predictive modelers was told never to say, “I don’t know” when answering questions from senior management. Similarly, they were told not to include caveats in their presentations. All these confessions of ignorance and qualified statements appear like “doubt speak” to the right-brainers. Do you have the answer or not? Captain Kirk, or the sensibly upgraded Dr. Elizabeth Weir of Stargate Atlantis, just want the answer so that they can “decide already.” Should we put our phasers on stun or close the stargate? Was that so difficult? We can benefit by thinking through these cultural differences. J A N U A R Y / F E B R U A R Y 2 014

|

19

FORUM

What you need to know if you are dating a right-brainer: Left-brainers, you need to go to charm school and forget about impressing others with your level of preparedness, intelligence and impeccable logic. Okay, we get it; analytics is subject to uncertainty. Now start socializing analytics so it is not so threatening. Making other people feel stupid does not make you appear very smart. Stop qualifying your results. You dwell much too much on the fact that if your analysis is correct, then there is still a chance that the conclusion is wrong – incomplete information being what it is. Finally, if you are in a non-analytics culture, then you need to do more than write a glossary of acronyms and speak the local language. You must walk the walk, too. You need to behave as much like the right-brainers as you can stand – conform a little, sadly. Just deal with it. What you need to know if you are dating a left-brainer: Right-brainers, you need to appreciate that the leftbrainers have the lonely responsibility for getting the facts right in the face of messed-up, incomplete information. Going forward, you need to evolve, to accept more of the communication burden, to think differently. Is this so threatening? You want to embrace or at 20

|

A N A LY T I C S - M A G A Z I N E . O R G

least accept uncertainty. When reading analysis, you should interpret signs of intellectual humility as signs of intellectual humility and not weakness. If you want the left-brainers to explain things simply, then you can help by reminding them that you realize their work is complex. Keep asking them the same question until you get it. Do not give up. However, you cannot expect them to divulge their secret techniques. If the above was not enough for you, left-brainers will want to share all of the bad news they have discovered. Just deal with it. What we all need to know: In practice, we all have left- and right-brain behaviors, and anyone who thinks that some group of people is homogeneous does not know much about them. Now that we are in the Information Age, it is no longer wise to run a company with half a brain. Today, our medieval corporate cultures from the Dark Ages place too much of the burden on the left-brainers to get the numbers right and explain it so that a right-brainers can understand it [3]. This is unreasonable – or at least not optimal. Instead, we should ask for the caveats and accept the “I don’t knows” on our way to cashing in on analytics. In analytics, there can be something suspicious about someone with all of the answers; they are not W W W. I N F O R M S . O R G

left-brain. The cultural change we seek is to be both left- and right-brain. DENYING THE SERENDIPITY OF STATISTICS Before purchasing expensive data or executing a sophisticated analysis, you should plan how you are going to use this information or how you are going to analyze a business problem. Having a plan makes sense – just not perfect sense. No one sat down and wrote a detailed plan for the discovery of penicillin. It was a complete accident. Many great discoveries happen by chance. Holding a data request up to the standards of a mathematical proof is a bit much. This is a chronic breakdown point and the site of many a discombobulation. In an analyticsdriven culture, it should be sufficient for a plan to entail what you expect and emphasize the economics of the possible exploratory work. We may need to make numerous attempts on our way to success. Finally and foremost, we must resist the temptation of allowing people to present other peoples’ analytics work. This delays acclimation and creates a deceptive culture. At a number of corporations, this is the standard. No one below a certain rank is given the privilege of presenting to senior management, and the token A NA L Y T I C S

few qualified analytics professionals will always be below that rank – whatever it takes. This senior management intends to stay insulated in the “executive management bubble,” all right-brain. Randy Bartlett (Randy.Bartlett@ BlueSigmaAnalytics.com), Ph.D., is a business analytics/big data leader with Blue Sigma Analytics. He has more than 20 years of experience, which includes leading and organizing analytics resources, reviewing advanced analytics results and providing advancements in business analytics. Bartlett delivers presentations and writes about business analytics, including the article “The Business Analytics Revolution,” co-authored with Girish Malik, that appeared in the May/June 2013 issue of Analytics magazine. Bartlett is also the author of a book, “A Practitioner’s Guide to Business Analytics,” from which this article was adapted. Reprinted with permission from McGraw-Hill Professional. Bartlett is a member of INFORMS. NOTES & REFERENCES 1. It is an amazing feat to write a book about a naval battle and tie the outcome to a cultural characteristic. See “Midway: The Battle That Doomed Japan” by Mitsuo Fuchida and Masatake Okumiya (1955). 2. “Competing on Analytics: The New Science Of Winning,” “Analytics At Work: Smarter Decisions, Better Results,” “Data Driven: Profiting from Your Most Important Business Asset,” and “Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart,” among others, have made it clear that analytics is too understated in the blend. 3. This is the right-brainers saying they cannot be bothered to think in a left-brain manner for a single moment.

Subscribe to Analytics It’s fast, it’s easy and it’s FREE! Just visit: http://analytics.informs.org/

J A N U A R Y / F E B R U A R Y 2 014

|

21

CAL L FO R TO P I C S

Big data dreams, small data reality Because of this abundance of data, many “best practices” rely on making data-based decisions. Yet there are still many situations where we unfortunately do not have sufficient data to make such decisions.

There is no doubt that we live in an era of dig data. We seem to have mountains of data about everything from business operations to customer behaviors, from personal health to global disease outbreaks. Because of this abundance of data, many “best practices” rely on making data-based decisions. Yet there are still many situations where we unfortunately do not have sufficient data to make such decisions. So what do you do when you have big data dreams but a small data reality? This is the focus of a new Analytics magazine column, debuting in the March/ April 2014 issue. Through this column we will explore how to deal with situations where we need data, but it’s limited or nonexistent. AT SCALE: BIG DATA

BY BRIAN LEWIS

22

|

I’ll give a specific example from my current work as chief data scientist at Fractal Sciences, a marketing automation software company that optimizes digital advertising and engagement (think Facebook and Twitter ads, but a lot more). Without giving too much away and without getting too technical, Fractal’s ad optimization algorithm is based in part on a proprietary feedback loop that uses prior ad campaign data to automatically predict, recommend, create and target new ads in order to maximize a customer’s ROI for their advertising spend. As a result, our customers’ ad campaign results get better and better the more they use our product.

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

In our internal data science meetings, we love to think about, tinker with and invent our next generation algorithms for when a customer is “at scale.” When at scale, a customer has run enough ad campaigns that have created enough data that we can finally apply some of our cutting-edge predictive analytics, machine learning and optimization algorithms. That is, we actually have some big data to work with. EARLY ON: SMALL DATA Long before a customer is at scale, they are essentially in a start-up phase. In this phase, terabytes and petabytes are replaced by mere megabytes. A/B/n testing is replaced by just … A. Predictive analytics is replaced by anecdotal evidence. And sample sizes are so small that the concept of statistical significance is, well, insignificant. From a data science perspective, we refer to this as small data. But despite the lack of data during this start-up phase, customers still expect our platform to optimize their ad campaigns. So how do we approach this situation? We will address this and other similar situations in the “Big Data Dreams, Small Data Reality” column. A few other obvious examples include planning for new businesses, new products or services and new business processes. A NA L Y T I C S

A start-up almost certainly lacks the historical data that an established company has collected about its operations, finances or sales and marketing strategies. Yet a new business still needs to plan its future: which products or services to launch, which customers to target, how to set pricing policies, how to promote the brand, how to layout the website, how much to staff up, and so on. All of these decisions could be aided by data, if only you had some. In the absence of data, one of the most important parts of planning to make data-driven decisions is how you structure your decision model. Did you include the right objectives, constraints and other assumptions? Even though you have no data, you still have to populate your model with something, for example industry benchmark data, data from public company SEC filings, probability distributions (if you want to use something more sophisticated like Monte Carlo simulation), and yes, even gut-feel values. As you start gathering data, you can transition from those external data sources to your own internal data. But when do you make this transition? How much data is enough data? In contrast to new businesses, well-established companies, such as the Fortune 500, have databases upon J A N U A R Y / F E B R U A R Y 2 014

|

23

CAL L FO R TO P I C S

You have no data with which to accurately predict results such as operational efficiencies or inefficiencies. So you make assumptions, use some sort of data that is external to the new process, and then slowly transition to your direct data as you gather it.

databases of data. Yet when they launch new products and services, they also start off with no direct data to work with when making their design, launch and planning decisions. If the new product is similar to existing products, you might use data from those products as proxy data until you have collected sufficient data about your new product. Examples of proxy data include historical sales data for building demand forecasts or customer profiles for deciding to target audiences for advertisements. Once again, you must start with some form of external data and then transition to your real data over time. Introducing a new business process is no different than the two cases described above. You have no data with which to accurately predict results such as operational efficiencies or inefficiencies. So you make assumptions, use some sort of data that is external to the new process, and then slowly transition to your direct data as you gather it. CALL FOR TOPICS FOR FUTURE COLUMNS The vision for this column is to be a community discussion about all types of small data situations at all types of companies and organizations. If you have a small data situation that you would like discussed in this column or even want to be interviewed for the column about your small data situation, please let me know! Until then, keep big data dreaming. Brian Lewis, Ph.D. ([email protected]), is chief data scientist and co-founder of Fractal Sciences, a digital marketing automation and optimization software company.

24

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Connect with the earned expertise of business forecasters and practical research from top academics from around the globe.

Each issue of Foresight contains articles that you’ll use in your day-today work, whatever types of forecasting you do. Here’s what our readers say:

““ “ “ “

The information is relevant to practitioners and is presented in a way that is not overly academic but with significant credibility.”

Issue 26 Summer 2012 THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING

- Thomas Ross, Financial Analyst, Brooks Sports

Foresight make(s) important research findings available to the practitioner.” - Anirvan Banerji, Economic Cycle Research Institute

...an important forum for practitioners to share their experiences....” - Dan Kennedy, Senior Economist, Connecticut Department of Labor

5 Setting Internal Benchmarks Based on a Product’s ForecaStaBIlIty DNa 18 Regrouping to Improve Seasonal Product Forecasting 32 Forecasting Software that Works For – Not against – Its Users

38 Book Review Abundance: The Future Is Better Than You Think 41 reliably Predicting Presidential elections

Issue 27 Fall 2012 THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING

I find Foresight very useful! I use it as a teaching resource to bring theoretical forecasting techniques to life for the students.” - Dr. Ilsé Botha, Senior Lecturer, University of Johannesburg

Put Foresight to work to improve your forecasts and rally support for the people, processes, and tools that accurate forecasting requires.

Subscribe or Renew today! forecasters.org/foresight/subscribe/

5 Special Feature: Why Should I Trust Your Forecasts? 23 Tutorial: The Essentials of Exponential Smoothing 29 S&OP: Foundation Principles and Recommendations for Doing It Right

40 New Texts for Forecasting Modelers

Issue 29 Spring 2013 THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING THE INTERNATIONAL JOURNAL OF APPLIED FORECASTING

5 Forecasting revenue in Professional Service Companies 14 Forecast value added: A Reality Check on Forecasting Practices 19 s&oP and Financial Planning 26 cPFr: Collaboration Beyond S&OP 39 Progress in Forecasting rare events 50 Review of "global trends 2030: alternative Worlds"

INFO RM S INI T I AT I VE S

World’s best analytics teams compete for Edelman honors The Edelman Award is the world’s most prestigious recognition for excellence in applying advanced analytics to benefit business and humanitarian outcomes.

26

|

The Institute for Operations Research and the Management Sciences (INFORMS) named six finalist organizations that will compete for the 2014 INFORMS Franz Edelman Award. The Edelman Award is the world’s most prestigious recognition for excellence in applying advanced analytics to benefit business and humanitarian outcomes. This year’s finalists include: • The Energy Authority for “Hydroelectric Generation and Water Routing Optimizer” • Grady Memorial Hospital, with the Georgia Institute of Technology, for “Modeling and Optimizing Emergency Department Workflow” • Kidney Exchange at the Alliance for Paired Donation, with Stanford and MIT, for “Kidney Exchange” • NBN Company, with Biari, for “Fiber Optic Network Optimization at NBN Co.” • Twitter, with Stanford University, for “The ‘Who to Follow’ System at Twitter: Strategy, Algorithms and Revenue Impact” • The U.S. Centers for Disease Control and Prevention (CDC), with Kid Risk, Inc., for “Using Integrated Analytical Models to Support Global Health Policies to Manage Vaccine Preventable Diseases: Polio Eradication and Beyond”

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

The six finalists will make presentations at the INFORMS Conference on Business Analytics and Operations Research in Boston, March 30-April 1. The winner will then be announced at the Edelman Awards Gala held in conjunction with the conference. Now in its 43rd year, the INFORMS Franz Edelman Prize competition recognizes outstanding examples of analytics and operations research projects that transform companies, entire industries and people’s lives. Using innovative advanced analytical methods, the teams were instrumental in helping their respective institutions make better decisions, providing a disciplined way by which management can improve organizational performance in a wide variety of situations and across both public and private organizations. INFORMS Franz Edelman finalist teams have contributed more than $200 billion in benefits to business and the public interest. The 2014 finalists were chosen after a rigorous review by accomplished verifiers, all of whom have led successful analytics projects. Finalists are chosen on the merits of how analytics methodologies were applied to solve problems, reduce costs or otherwise improve results in real-world environments. Additional information about the INFORMS Franz Edelman Award A NA L Y T I C S

competition, including video interviews with 2013 finalist executives, can be found online at https://www. informs.org/Recognize-Excellence/ Franz-Edelman-Award. NEW DATES FOR CONTINUING EDUCATION FOR ANALYTICS PROFESSIONALS Following its successful launch in 2013, INFORMS will once again offer its popular continuing education courses “Essential Skills for Analytics Professionals” and “Data Exploration & Visualization” in 2014. Essential Practice Skills for Analytics Professionals Gain essential tools for integrating your analytical skills into real-world problem solving. “The course was expertly presented and outlined methods that were immediately applicable to my everyday work. I would recommend this course to anyone looking to improve their problem-solving skills as well as their ability to present complex projects and problems in a clear, concise and logical way.” - Richard St-Aubin, P.Eng., IPEX Management Inc. Upcoming classes: Feb. 20-21 - Atlanta March 28-29 - Boston June 20-21 - San Jose, Calif. J A N U A R Y / F E B R U A R Y 2 014

|

27

INFO RM S INI T I AT I VE S

Data Exploration & Visualization Hands-on training that focuses on the critical steps in the process of analyzing data: accessing and extracting data, cleaning and preparing data, exploring and visualizing data. Upcoming classes: March 28-29 - Boston June 25-26 - San Jose, Calif. F o r mo re i nfo rma ti on on the courses and available discounts, visit www.informs.org/continuinged 2014 INNOVATIVE APPLICATIONS IN ANALYTICS AWARD The Analytics Section of INFORMS has named three finalists – Ford, IBM and Fiserv – for its 2014 Innovative Applications in Analytics Award that will be presented at the INFORMS Conference on Business Analytics and Operations Research in Boston, March 30-April 1. Sponsored by the section, the purpose of the award is to recognize creative and unique developments, applications or combinations of analytical techniques used in practice. The award competition promotes awareness of the value of analytics techniques in unusual applications, or in creative combination to provide unique insights and/or business value. Following a series of presentations, the three finalists were chosen by a 28

|

A N A LY T I C S - M A G A Z I N E . O R G

panel of judges from a competitive set of 31 submissions and nine semifinalists. Taken together, this work provides great examples of innovative applications and integration of a variety of analytical techniques that are making a difference in organizations. The three finalists will make another round of presentations in Boston before a winner is named. A closer look at the finalists and their work: Ford: “Enabling greater sustainability in vehicle fleet sales through customer-oriented analytics” Authors: Daniel Reich, Sandra L. Winkler, Erica Klampfl, Natalie Olson, Ford Motor Company Presenting author: Daniel Reich Abstract: Ford’s Fleet Purchase Planner (patent pending) is the first of its kind in promoting sustainability as a central purchase consideration for organizations with large vehicle fleets. In recent years, several new green vehicle technologies have emerged that present opportunities for increasing fuel economy, but this growing number of options is also making planning purchases a significantly more complicated endeavor. This decision support system is designed to simplify this process by optimizing for individual driving patterns. We are helping our fleet customers find the W W W. I N F O R M S . O R G

most cost-effective opportunities for increasing their sustainability. IBM: “A SMS text classification system for UNICEF Uganda” Authors: Prem Melville, Vijil Chenthamarakshan, Rick Lawrence, Solomon Assefa, Machine Learning Group, IBM Research, Yorktown Heights, N.Y.; James Powell, Sharad Sapra, UNICEF Uganda; Rajesh Anandan, US Fund for UNICEF, New York, N.Y. Presenting author: Rick Lawrence Abstract: U-report is an opensource SMS platform operated by UNICEF Uganda, designed to give community members a voice on issues that impact them. Data received by the system are either SMS responses to a poll conducted by UNICEF or unsolicited reports of problems occurring anywhere within Uganda. There are currently more than 200,000 Ureport participants, and they send up to 10,000 unsolicited text messages a week. The objective of the program in Uganda is to understand the data in real time and have issues addressed by the appropriate department in UNICEF in a timely manner. Given the high volume and velocity of the data streams, manual inspection of all messages is no longer sustainable. This talk describes an automated A NA L Y T I C S

message understanding and routing system deployed my IBM at UNICEF. We employ recent advances in data mining to get the most out of labeled training data, while incorporating domain knowledge from experts. We discuss the tradeoffs, design choices and challenges in applying such techniques in a real-world deployment. We conclude with a discussion of the societal impact that U-report is already driving in Uganda and discuss plans for future deployment. Fiserv: “Price & revenue optimization for one of the largest acquiring banks in South America” Authors: Suman Kumar Singh, Aditya Khandekar, Tarang Goyal, Fiserv Abstract: The Merchant Acquirer had about a million merchants in mass market. An opportunity existed to selectively optimize price for merchants based on certain attributes. A statistical segmentation was developed following which nonlinear differential price elasticity function was built. For each segment, price elasticity along with an optimization algorithm with complex constraints was developed to optimize price and achieve optimal revenue. – Pooja Dewan (committee chair of the Innovative Applications in Analytics Award) J A N U A R Y / F E B R U A R Y 2 014

|

29

RES EARC H SU R VE Y

Predictive analytics in the cloud Ability to deliver ROI solutions more cost-effectively is driving cloud deployment.

BY JAMES TAYLOR Decision Management Solutions (DMS) recently conducted research into predictive analytics in the cloud. Sponsored by FICO, Lityx and SAP, the research has at its core a survey of more than 350 respondents from a wide range of industries. Following up on a 2011 survey, the 2013 results make it clear that predictive analytics in the cloud is becoming increasingly mainstream, with broader and accelerating adoption.

D

30

|

A N A LY T I C S - M A G A Z I N E . O R G

The most striking result is that the number of companies reporting a positive impact from predictive analytics has risen dramatically since 2011. More than two-thirds of this year’s respondents have seen a positive impact from using predictive analytics in their business. It is also noticeable how much greater the reported impact is in 2013 relative to 2011. In 2013, many more companies reported transformative or significant impact than in 2011, while far fewer reported no usage or no plans as shown in Figure 1. W W W. I N F O R M S . O R G

Bucking the trend, 10 percent of the respondents said they still have no plans to implement analytics, and nearly a third have yet to put predictive analytics into production. As one respondent said, “[There is] still much user resistance to using Figure 1: Increasing impact from predictive analytics. [the] results of analytics. People still believe in the superiority of human judgment.” Matching this rise in overall impact from predictive analytics is a similar rise in both current and planned deployment Figure 2: Broad adoption of predictive analytics in the cloud. of predictive analytics in the cloud since 2011. The research divided predictive analytics survey respondents said they had dein the cloud into three use cases: ployed at least one of these predictive 1. Pre-packaged, cloud-based decisionanalytics in the cloud use cases – a sigmaking solutions that embed nificant increase over 2011. As Figure 2 predictive analytics. shows, an astonishing 90 percent said it 2. Cloud-based predictive modeling – was likely they would have at least one building models in the cloud. class of solution widely deployed in the 3. Cloud-based deployment of predictive next few years. Predictive analytics in analytics – scoring in the cloud. the cloud is going mainstream and may, These three scenarios leverage the in fact, already be there. scalability and pervasiveness of the cloud The primary driver for the use of as well as the growing use of the cloud cloud-based solutions was reduced cost. to deliver data. More than 60 percent of Advanced analytic applications have A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

31

ANALY TIC S I N T H E C LO U D

historically been both very high ROI and very high cost. There has been constant pressure on the market to deliver ROI solutions more cost-effectively, and this is clearly driving cloud deployments of predictive analytics. The typical obstacles to predictive analytics also came through in the survey: data security and privacy, along with regulatory and compliance concerns, remain the primary obstacles reported. As one respondent said, “Cloud based solutions mean either storing or transmitting our proprietary data to the cloud. Although there are safe ways to do this, our management is not convinced.” Predictive analytics has a strong history in credit risk and fraud detection. Recently, much of the market’s energy has been directed toward the use of predictive analytics for maximizing the opportunity from customer interactions, often positioned as cross-sell/up-sell. The big focus area for predictive analytics among respondents is in customer interaction; however, the particular focus of respondents was on customer satisfaction, customer retention and customer management rather than on increased sales. Many respondents use predictive analytics in marketing and cross-sell/up-sell, but the number one focus is using predictive analytics to improve customer engagement. 32

|

A N A LY T I C S - M A G A Z I N E . O R G

Given the importance of the cloud to big data, with so many new data sources being cloud based, it seemed appropriate to investigate the impact of big data on predictive analytics in the cloud. In particular, the survey explored the degree to which new data types (the variety aspect of big data) and “recent-cy” (the velocity aspect of big data) were impacting respondents. When asked what data matters most to predictive analytic models, the vast majority of most respondents indicated it was what you might call traditional data types, and structured data from their own internal systems was by far the most important. The survey also revealed a definite sense that unstructured data from internal systems was becoming mainstream, while no other data types were deemed particularly important. When more experienced analytic teams were separated out, however, and only those with existing deployments or significant impact were considered, the picture was quite different. These more experienced teams show much higher usage of new data types than in 2011. Social media, sensor, weblog, audio and image data types are all rated as much more important in analytic models among those with successful analytic deployments as W W W. I N F O R M S . O R G

shown in Figure 3. This probably reflects the use of new data types to improve the predictive power of existing models. Teams are still beginning largely with traditional data types, but they see increased value from new data types once they have Figure 3: More experienced practitioners use more data types. some success. With more successful, more established stored back into a database for later use teams using big data more broadly, it to real-time scoring. This shift is reflected seems likely that there will soon be a in the increased use of intra-day and realrapid and significant growth in the use time data in predictive models. As one surof new data types in building predictive vey respondent put it: “Intra-day data will analytics. More traditional structured be the most valuable to our company since data will likely remain broadly central we are open 24 hours.” to effective predictive analytic modScoring streaming data is not yet a els. One survey respondent put it this mainstream use case though it seems way: “Big data is a misnomer as data likely that the general shift to a more eventhas always been big. The challenge centric, real-time world will bring it squarely is making use of semi-structured and into the mainstream before too long. unstructured data in solutions. This Back in 2011 it was clear that early will be the next giant leap forward in adopters were going to get an edge. using data.” They were more likely to have plans for The velocity of data also matters. Prebroader deployment and saw predictive dictive analytics is increasingly focused on analytics in the cloud as more valuable. near real-time, operational data. This kind This trend strongly repeated in 2013. of data grew the most in importance beOnce again, early adopters with one or tween 2011 and 2013. This corresponds more use cases deployed were signifito the general shift in predictive analytcantly more likely to have plans to exics from batch scoring where results are pand deployment. Similarly those with A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

33

ANALY TIC S I N T H E C LO U D

experience were likely to rate the value of each scenario more highly. Exposure to predictive analytics in the cloud breeds enthusiasm – those who buy into the promise of predictive Figure 4: Decision management transforms results. analytics and get started like the results and want to do more. Therefore, organizations that get started quickly with predictive analytics in the cloud have the opportunity to create differentiation from slower-moving Figure 5: Decision management on the rise. competitors. One last result deserves a special call out. Recent years While a similar result was found in have seen increased interest in decision 2011, the percentage reporting the use of management or prescriptive analytics— this approach has risen significantly since the embedding of predictive analytics 2011 as shown in Figure 5. Decision maninto operational decision-making sys- agement (i.e., prescriptive analytics, with tems. The importance and value of this its systematic embedding of predictions trend was shown clearly in the survey into automated decision-making systems) results. More than 95 percent of survey is an effective approach to maximize respondents who adopted this approach the transformative power of predictive (tightly integrating predictive analytics analytics. into operations) reported transformative James Taylor (james@decisionmanagementsolutions. or significant impact. Putting predictive com) is CEO of Decision Management Solutions analytics to work in operations is strong(http://www.decisionmanagementsolutions.com/). This article summarizes the key results of the study. ly correlated with the most impressive For a full report, as well as a recording of a webinar results as shown in Figure 4. summary, click here. He is a member of INFORMS. 34

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

VISUAL See your data for all it’s worth. Explore billions of rows of data in minutes or seconds, visually represented in a way that brings hidden patterns into plain sight. Then easily create and share reports on the Web or mobile devices.

sas.com/vademo

Experience Visual Analytics firsthand.

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. © 2013 SAS Institute Inc. All rights reserved. S105651US.0213

ADVEN T U RE S I N C O NS U LT I NG

Eighteen things nobody tells you about solo practice BY DOUG SAMUELSON So you’ve decided to break free of whatever organization you worked for and go out on your own as an analytics guru, have you? Solo practice does have its attractions. You’ll have a boss you can always reason with. You get to set your own hours, rates and expectations. Freedom! But there are a few things you had better keep in mind.

S

– it’s all yours! You see to it or it won’t get done. This means: 2. Your time and attention are your most critical and scarce resources. Manage them accordingly. Lots of people will want to talk to you, usually about what they want, not what you want. Be polite and approachable, but learn to say “no,” preferably gently, but firmly when necessary. You’ll have to say “no” a lot. And, to make this aspect of your life more difficult:

1. Everything is your responsibility. Paying the bills, writing thank-you notes and holiday cards, marketing, sales, do3. Hordes of people have someing the taxes, even cleaning the toilet thing to tell you. Ralph Waldo 36

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Emerson famously wrote, “If you build a better mousetrap, the world will beat a path to your door.” True enough, but since he hadn’t tried it himself, Emerson did not realize who those pathbeaters were. Maybe a few want to buy your mousetraps, but many more want to sell you something they claim you need, or instruct you on moral and legal obligations you may or may not have as a business owner. Or they just want to tell you a story that means more to them than to you. They know someone who tried what you’re doing and succeeded . . . or failed, or got indicted, or developed a drinking problem. Again, you’ll have to say “no” a lot. Sometimes you’ll need to say it immediately after the other person’s “hello.” Or, in some extreme cases, even earlier. So …

5. Sometimes a little arrogance is a good thing. If other people could do what you do as well as you can, they’d be doing it, wouldn’t they? But openly displaying arrogance turns people off. What you want is quiet confidence: I’ve done things like this before; I can do this, too. I know you want what I can give you. This attitude is the sure cure for writer’s block and speaker’s terror, too. Speak quietly and modestly, but walk with assurance. But also remember:

4. Take all the good advice you can get, but remember that no one knows your business better than you do. A baseball player knows he’s in a slump when cab drivers who couldn’t make their high school team are giving him hitting tips. He knows he’s in a bad slump when he starts listening to those tips. The most useful information comes from people who have done something very similar to what you’re trying to do. Remember:

7. You still need to depend on other people. You’re free, free, free, but all those things you are responsible for and don’t want to do have to be delegated to other people. You’re not going to be good at everything, nor are you going to have the time and energy to do all the things you’re good at. And you’ll still need people to critique your drafts, help you debug computer code, fill in gaps in your expertise, introduce you to new clients and prospects – and so on and so on.

A NA L Y T I C S

6. You still need to please other people. You can usually disengage from a client with less difficulty and cost than you’d incur in leaving an employer, and you get more of a choice of teammates, but you still have to do things you don’t like in order to keep clients and colleagues happy. And …

J A N U A R Y / F E B R U A R Y 2 014

|

37

ADVEN T U RE S I N C O NS U LT I NG

As you become more senior, you’ll want to hire other people to do more and more things you do well. You may still be better at it, but your time is becoming more valuable. It’s advantageous to pay someone who can do it half as well as you at one-third the price. And your price is the opportunity cost, that is, the value of whatever you could be doing instead. Both in the marketplace and internally, you should be spending an ever-increasing proportion of your time doing things no one else can do – and for which, therefore, a high price is justified. Of course you have to keep selling to stay in business, and you need to focus on selling what you want to do and profit from. So keep in mind: 8. Your expertise isn’t the most important thing you’re selling. Your most critical asset is trustworthiness. This includes being right when you offer a technical solution or suggestion, but your client won’t even receive and process your technical input – in fact, probably won’t even engage you – until you’ve established that you know the client’s language, understand the client’s problem and point of view, and care about meeting the client’s needs. Perhaps you’ve been trained that you have an obligation to give the client your best technical solution. Wrong! You have an obligation to 38

|

A N A LY T I C S - M A G A Z I N E . O R G

give the client the best solution they can and will implement. Among other things, this means that every single decision variable in a model of a system has to be something the client can control, and every single data element your model uses has to be available to the client at the time a decision is to be made. Also keep in mind: 9. Most senior executives are not nearly as knowledgeable or as confident as they appear. Management is mostly quick decisions with incomplete information under intense scrutiny, where the first slip can be a career-breaker. Managers above the first-line supervisors generally have little or no contact with where and how tasks are actually done. They know they can’t rescue bad situations by themselves. They value people they can trust and depend on who know certain subject areas better than they do. On the other hand, if untrustworthy people seem to know something the manager doesn’t, the manager may perceive them as threats rather than assets. Most companies aren’t as badly managed as they appear. They’re worse – the managers know more than you do about where they messed up. But they have survived – respect that! The key is to avoid messing up on the relatively few requirements for which failure can totally kill W W W. I N F O R M S . O R G

your company quickly. For example, don’t let your phones get cut off, make timely tax payments and don’t issue bouncing paychecks. And understand that executives you serve have had to “triage” what they have to handle, too. Help them with one of their few key requirements and they’ll keep coming to you for more help. A closely related point: 10. Don’t assume the clients really know what they want. Even if

12-6 leeds ad alternate_Layout 1 12/20/13 8:13 AM Page 1

what you were to deliver was specified in excruciating detail, you can’t assume they’ll love it when they see it. You have to deliver partial answers (prototypes, preliminary analyses, simplified models) early and often, and engage the client throughout the process in refining the statement of objectives. Deliver multiple prototypes and let the client choose which to pursue further – you keep the client committed to the development process, and you elicit what their real priorities are.

Stand Out. Leeds alum Matt Emmi, founder of OneButton, Boulder-based tech company

Position yourself in a lucrative new career

with a master’s degree in Business Analytics or Supply Chain Management. Intensive nine month programs World-renowned faculty

Experiential projects with industry clients Personalized professional development

Be Boulder.

Get started now: www.leeds.colorado.edu/MS

A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

39

ADVEN T U RE S I N C O NS U LT I NG

I’ve done a number of software development projects for a firm fixed price, under a contract that comprised three tasks: Task one, it’s ugly and balky and frustrating, but if you can manage to get a full set of test data in, the output indicates that the computations are being done correctly. Task two, the program fully meets specifications: output in formats specified in the task order, user-friendly, doesn’t break under intense challenge testing. Task three, I fix everything they didn’t like after it met spec. The three tasks are equally funded. This contracting approach prevents the usual “requirements creep” and continuous wrangling over change orders. It has the added advantage of weeding out quite a few bad clients, as people will never sign a contract like this if they can’t acknowledge to their bosses that they can’t write a perfect spec. Now, as you prepare to go selling, remember these three things: 11. In a successful sales call, the salesperson does about 5 percent of the talking. Listening to the customer is critical, and not just to learn what you could say next. Once I saw a super-salesman interviewed on the Tonight Show. Johnny Carson asked, “How would you sell me this coffee cup?” The salesman promptly 40

|

A N A LY T I C S - M A G A Z I N E . O R G

responded, “Tell me all the things you could do with this cup.” 12. Humor can be a very good icebreaker. You have to have a good feel for what the other person thinks is funny, though. Humor with strangers is risky. That’s precisely why it’s so effective when you get it right. In any case, pay close attention to cues that let you know whether you’re breaking down barriers to trust. Remember you’re there to get them to believe you understand their problems and can solve them, not to entertain them or show them how smart you are. Techies often blow sales calls by talking too much about technical details. The right way to show your capabilities without talking too much: 13. You have to keep replenishing your intellectual capital. If you concentrate exclusively on billable work, your skills will be obsolete within five years. Take courses. Participate in professional societies – especially your clients’ and prospects’ professional societies. Read. Listen. Write articles, do presentations and pay attention to the feedback. Find knowledgeable people and ask questions. Mentor others; their questions will stimulate you. Learn new methods, new application areas and new ideas, W W W. I N F O R M S . O R G

Is the largest association for analytics in the center of your professional network? It should be.

▪ Certification for Analytics Professionals ▪ A FREE Community Membership

to join online visit http://join.informs.org

▪ Continuing Ed courses for Analytics Professionals ▪ Online access to the latest in operations research and advanced analytics techniques ▪ Unsurpassed Networking Opportunities available in INFORMS Communities and at Meetings ▪ Subscriptions to online and print INFORMS Publications ▪ INFORMS Career Center - the industry’s leading job board

JOIN

ADVEN T U RE S I N C O NS U LT I NG

not just extensions of what you already know well. And then how do you translate this knowledge into effective communication? Well … 14. Don’t always “be yourself.” This recommendation shocks people. Of course you shouldn’t pretend to be someone or something you’re not, right? Especially you shouldn’t claim expertise you don’t have. But consider how many former actors have succeeded in lines of work far removed from the stage. Acting teaches you to work out and feel how someone else would view the world, what they would do in a new situation, what they might mean by what they say. Wargaming helps you develop a set of skills highly similar to what you’d get from acting, as a role player or from directing, as a game designer or moderator. Maybe when you were in school you were advised, “If you want a B, learn the subject. If you want an A, learn all you can about the teacher.” Similarly, the most successful advisors focus on learning the client’s point of view, typically by some form of role-playing. Speaking of others’ point of view, here are some important points about your friends, family and financial backers: 42

|

A N A LY T I C S - M A G A Z I N E . O R G

15. Most people don’t share your risk preferences. Entrepreneurs are, by definition and by nature, risk-takers. Most people aren’t. This can create serious conflicts. To cite one particularly important example, your spouse probably doesn’t share your willingness to take risks, especially if you deliberately chose a mate whose steady job brings stability to your joint finances. When you explained what you were planning to do, your spouse may have assented without really believing you’d actually do it. Keep communicating about what risks you’re taking and what your plans are in case things go badly. Understand that you may have to change some plans in order to provide needed assurances. The same is true of friends who could back you financially: If they have significantly more money and financial stability than you do, they may be reluctant to risk their money with you. To ameliorate this problem: 16. Borrow money before you need it. Whether you go to a bank, a venture capitalist, your friends or whoever, the best time to borrow is when you don’t really need money. If you wait until you desperately need money, you look like a bad risk. If you borrow some money, or establish a line of credit, and repay it, you help to convince lenders that you will pay as agreed next time. And if W W W. I N F O R M S . O R G

you need to ask for another loan before you’ve paid off the last one, you’re likely to see a previously generous lender turn more reticent than Ebenezer Scrooge. And similarly … 17. Talk to people when you don’t need their help. If you get a reputation of only calling when you want something, you’ll soon have trouble getting your calls answered. You’re already enforcing the same rule on people who call you, aren’t you? And finally … 18. Remember the (Colin) Powell Doctrine: don’t get into anything unless you know how you’d get out of it. This is nowhere more true than in a business. Do you want to keep it for life and pass it on to your kids? If so, do they want it? Do you want to get bought out? If so, do you want that buyout to include a good job with the buying company? (Not usually something that works out for more than a year. You didn’t like working for someone else’s company, remember? If you go this route, be

prepared for getting quietly pushed out.) Do you want to go public and cash out? Do you want to make a certain amount of money and then quietly close down? Your exit strategy dictates a lot of other decisions. For example, if you want to retire in 10 years and don’t care whether the business continues after that, growing rapidly and hiring a bunch of people will head you into trouble. If you plan to go public, high-quality record-keeping is essential. If you want to stay for life, don’t get an ambitious board that might decide you’re holding the com pany back fr om gr eater success. If you have questions, I do happen to have some time available … Good luck! Douglas A. Samuelson (samuelsondoug@ yahoo.com), D.Sc., is president and chief scientist of InfoLogix, Inc., a consulting and R&D company in Annandale, Va., and a contributing editor of OR/MS Today and Analytics. He has been a federal policy analyst, a high-tech inventor, entrepreneur and executive, a consultant and a researcher. He is a senior member of INFORMS.

Request a no-obligation INFORMS Member Benefits Packet For more information, visit: http://www.informs.org/Membership

A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

43

CAP UPDAT E

Certified Analytics Professional A usage guide for employers and clients.

BY POLLY MITCHELL-GUTHRIE (left) AND SCOTT NESTLER, CAP (right)

T

The purpose of this article is to provide potential employers and clients of analytics professionals with useful information about the Certified Analytics Professional (CAP®) program offered by INFORMS, the leading professional membership organization in the world for advanced analytics. Beyond providing some updates regarding the CAP program, we also offer suggestions on how certification should (and should not) be used and attempt to answer questions 44

|

A N A LY T I C S - M A G A Z I N E . O R G

such as, “How can certification help you find and retain analytical talent?” and “Why should I support certification for my existing employees?” FOCUS ON SKILLS AND KNOWLEDGE There are many ways to qualify a new hire; certification is one that may have some advantages over other methods. First, the CAP certification is designed to test across the breadth of knowledge that is necessary in many analytics jobs. Some of this was detailed in an earlier W W W. I N F O R M S . O R G

publication on “The Shape of Analytics Certification.” While many of the newer degrees in analytics incorporate much of the same components in their curricular program, other related degrees may focus more on providing depth in one or more technical skill areas. The scope of relevant skills emerged from the CAP job task analysis (JTA), a methodological approach to determining what should be tested, that was developed to be a market-driven assessment of necessary skill areas for analytics professionals. In addition to addressing breadth of knowledge, the CAP certification tests skills, which can be defined as “the ability to use one’s knowledge effectively and readily in execution or performance.” This means that not only has the individual learned the knowledge through formal or informal education, but they also demonstrate proficiency in the application of that knowledge. While casebased interviewing is possible, it is tricky. So, while employers can ask a more concrete question such as familiarity with a particular algorithm, assessing whether a candidate knows when and how to apply this algorithm is tougher. The CAP tests this kind of scenario through the domains of business problem framing and methodology selection. Finally, students often learn the fundamentals in academic environments, which may not suffer from A NA L Y T I C S

The 5 E’s in CAP Many of the questions we receive from prospective applicants focus on the Exam, which tests skills and knowledge listed in the JTA. However, we would like to emphasize the importance of the other four E’s to employers and clients of analytics professionals. In order to apply for certification and take the exam, a candidate must demonstrate that they possess the necessary Education (B.S. or higher) and analytics-related Experience (3-7 years, depending on level of degree and field of study). Additionally, they must demonstrate Effectiveness, by having their soft skills validated by a current or former employer or client. Finally, they must agree to abide by a written Code of Ethics. the rigors of data that are often dirty, incomplete, messy, big, etc. The JTA covers the entire analytic life cycle, starting with the problem, working through the data, and finishing with model deployment and monitoring, thus addressing the challenges of real-world problems. Through a shared understanding of criteria and standards across the work environment, those employing analytics professionals will have greater confidence and assurance of organizational and individual qualifications. J A N U A R Y / F E B R U A R Y 2 014

|

45

CAP UPDAT E

CERTIFICATION SETS EMPLOYEES APART Besides demonstrating the recipient’s ability to apply their knowledge and skills to comprehend and analyze a real-life situation or issue, conduct an analysis and make a recommendation to the decision-maker, a certification such as CAP establishes the confidence and credibility of the people who are hired (i.e., right person for the right job). Most hiring managers and supervisors will agree that the ideal employee is much more than just an experienced worker. In particular, she is a dedicated professional with the right attitude as well. Good indicators of attitude include someone who is adaptable and willing to learn new skills, as well as someone who possesses the drive and confidence to demonstrate it, such as earning a relevant certification. An individual who has chosen to invest time, money and effort to prepare for and achieve certification demonstrates dedication beyond any doubt. Because the CAP certification requires completion of professional development units (PDUs) to maintain the certification, it is also an indication that an individual is keeping upto-date with developments taking place in the ever-changing analytics industry. This is certainly a quality that all managers look for in their potential employees. 46

|

A N A LY T I C S - M A G A Z I N E . O R G

CAP program update •1  2 exams conducted in 2013; six planned for early 2014. •P  ass rate: 91 percent (includes some “early adopters”) •C  urrent number of CAPs: 87 and growing •C  omputer-based testing is coming in 2014! • Study Guide now available online QUALITY ATTRACTS QUALITY As the saying goes, “birds of a feather flock together.” This is true in the workplace, where exemplary workers tend to attract other dedicated analytics professionals to your organization. According to Certification Magazine, organizations that consistently invest in training and certification establish a culture of excellence, including the ability to attract and retain top talent. These organizations will have the best people, which will be noticed by customers and competitors alike. And this then becomes a virtuous cycle, whereby great employees continue to attract great employees, resulting in continuous improvement. Furthermore, once you have attracted an outstanding cadre of employees, certification can be a worthy goal to help those with less experience advance in their careers, especially if the employer indicates willingness to invest in these careers by W W W. I N F O R M S . O R G

supporting this goal. Doing so tells employees that this organization cares about career advancement, continuing education, professional development and the ongoing learning required to maintain certification. As difficult as it is to attract good employees it can be equally hard to retain them, especially when demand outstrips supply. Since analytics professionals are knowledge workers they appreciate firms that share that value of knowledge and are willing to ensure they continue to acquire it.

SOME USEFUL TERMS To use certification appropriately it is important to clarify a few terms that are often used interchangeably but actually have specific meanings. A credential is a broad, umbrella term for recognition of having met a defined set of standards. Two common types of credentials are licenses and certifications. Licenses are mandatory credentials issued by states while certifications are voluntary credentials offered by professional organizations, such as INFORMS and its CAP program. One concern borne

Attention Companies Who Hire Analytics Professionals Reserve your space now for the industry’s premier, professional job fair!

Analytics Connect Brought to you by Analytics Magazine

INFORMS CONFERENCE ON

USINESS ANALYTICS & PERATIONS RESEARCH Applying Science to the Art of Business

March 30–April 1, 2014 |     

CAREER CENTER

The Westin Boston Waterfront | Boston, Massachusetts

Find the seasoned professionals you need – over 800 analytics professionals expected Provide your recruitment materials in a casual setting Arrange discreet on-site meetings in private booths Enjoy discounted combination pricing with the fall Annual Meeting Job Fair Enhance your visibility with an ad in Analytics or OR/MS Today

Questions? [email protected] or call (800) 4-INFORMs A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

47

CAP UPDAT E

of not understanding the difference is the misconception that the intent of the CAP is to restrict the pool of practitioners. While this may be indeed true for licenses (where health or safety concerns may apply and hence the credential is mandatory), this is not the case for voluntary certifications like CAP, which allow recipients to choose to differentiate themselves when they do not feel their experience alone does so. Another common mistake is thinking that certifications and certificates are equivalent. They are not; the latter is usually issued upon completion of a course

of training but may not actually reflect achievement of any other education, experience or testing standard. HOW NOT TO USE CERTIFICATION The CAP should be considered a possible order qualifier but not an order winner, meaning this designation alone shouldn’t signal that a resume with it should result in an interview or one without it be set aside. Many good candidates will determine that their education and experience are alone sufficient to demonstrate the skills the CAP seeks to tease out. A candidate who has a Ph.D.

CAP exam schedule The Institute for Operations Research and the Management Sciences (INFORMS) has scheduled the following exam sites for its Certified Analytics Professional program for the first half of 2014: Jan. 11, 2014: ITPG Education Center,
Vienna, Va. (suburb of Washington, D.C.)

 Jan. 29, 2014:
University of Alabama,
Business Analytics Symposium, Tuscaloosa, Ala.

 March 6, 2014:
Drexel University, James E. Marks Intercultural Center,
Philadelphia

 March 29, 2014:
INFORMS Conference on Business Analytics and O.R.,
Westin Boston Waterfront,
Boston March 30, 2014:
Gartner BI & Analytics Summit,
Venetian Resort Hotel & Casino,
Las Vegas June 21, 2014:
INFORMS Conference on The Business of Big Data,
San Jose Marriott,
San Jose, Calif.

 To apply, click on https://www.informs.org/Certification-Continuing-Ed/ Analytics-Certification/Apply-for-Certification
 For more information, click on https://www.informs.org/Certification-Continuing-Ed/ Analytics-Certification

48

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

in statistics and 15 years of experience in credit risk may seem at least minimally qualified, whereas it may be less obvious whether the theater major who started his career on stage actually has the abilities he professes to have gained from five years at an entertainment company in an analytics role. However, if the theater major achieved the CAP, employers can have greater confidence in this likelihood. So certification can be a helpful screening mechanism but should not be relied upon alone.

Likewise there are tasks the JTA concluded are essential to analytical jobs but are beyond the scope of the certification exam, such as testing and selecting approaches and running and calibrating models. Beyond even those concrete areas is the fuzzier area appropriately termed “soft skills,” because they are harder to define and measure. These skills can run the gamut from interpersonal skills to the ability to present and communicate results effectively. While the CAP does consider this area

INFORMS Today Podcasts with Host Barry List A series of podcasts with unexpected insights into the way that math, analytics, and operations research affect people like you and organizations like your own. In every segment, an expert explains how he or she changed the world by crunching the numbers.

https://www.informs.org/podcast

A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

49

CAP UPDAT E

CAP Special Supplement This issue includes a special supplement, “Certified Analytics Professional Candidate Handbook” (second edition). To view it, click here.

by requiring applicants to provide validation of these skills, this measure is currently qualitative and certainly bears further investigation by employers. The nature of the depth and breadth of skills needed in a given position merits more targeted interview questions by the potential employer. Finally, even more intangible considerations such as attitude and fit are far beyond the scope of a certification exam and up to each employer to find other ways to assess. The benefits of hiring Certified Analytics Professionals certainly go beyond just having the right person on the job. Certified employees elevate the credentials of an organization and improve customer confidence. For example, a CAP-credentialed senior analyst with the Coast Guard Research and Development Center, while helping work up a new center for analysis to full operational capability, indicated that certification would increase the credibility of his organization and the important work it is doing. Consulting firms in particular have shown strong interest in the program as a way to illustrate to their customers the quality of their bench. It’s clear that the benefits of hiring certified CAPs extend beyond the internal needs of the organization and extend to the perception of it by external stakeholders. For more information about the CAP program, visit the website or contact Dr. Louise Wehrle, INFORMS Certification Manager, certification@ informs.org, 443-757-3599. Polly Mitchell-Guthrie is lead strategist and customer liaison for advanced analytics at SAS and vice-chair of the ACB. She is a member of INFORMS. Scott Nestler, CAP is an Army operations research analyst and chair of the INFORMS Analytics Certification Board (ACB). He is a member of INFORMS.

50

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Get Started with Analytics

Find out how the use of analytics can improve your bottom line.

INFORMS can help!

Analytics is the scientific process of transforming data into insight for making better decisions. Our members help organizations like yours everyday use analytics to improve processes, save costs, and enhance revenues.

WANT TO KNOW MORE? Our Getting Started with Analytics website provides: • Over 80 searchable “quick-bite” success stories • Five signs you can benefit from analytics • Information on how to recognize the opportunity • Sources of analytics expertise • What to expect when working with an analytics professional Start Here and Contact us for more information.

http://www.informs.org/Sites/Getting-Started-With-Analytics

Getting Started With Analytics Home

Start Here

What Analtyics Is

What it Can Do For You

How To Start Using Analytics

Analytics Success Stories

The Edelman Award

More Resources

Search Analytics Portal

WHAT’S YOUR TOUGHEST, MOST COMPLEX PROBLEM? MORE ANALYTICS SUCCESS STORIES

Industry

Function

Benefit

START HERE »

Analytics in the News

No matter your organizational situation, INFORMS and the skilled use of analytics can help.

ANALY TIC S & H E A LT H CA R E

Population health management and medication adherence BY RAJIB GHOSH

I

In my last article I highlighted opportunities for health analytics entrepreneurs in the field of radiology services and revenue cycle management. This article focuses on two more opportunities, which are also part of the Centers for Medicare & Medicaid Services’ (CMS) reportable quality measures for provider organizations: managing population health and medication adherence. Some healthcare organizations have already achieved early success using analytics, but many have yet to adopt it. During the next three to five years, I predict we will see increased adoption of analytics in these two areas, driven by the need of compliance and the search for profitability. 52

|

A N A LY T I C S - M A G A Z I N E . O R G

HEALTHCARE’S MICROECONOMICS PROBLEM The Federal Health Insurance Exchange (Healthcare.gov) is at the center of an ongoing controversy – website glitches, broken promises, customer dissatisfaction and general skepticism about the Affordable Care Act (ACA). While we can argue on both sides, the opening of the marketplace to allow consumers to buy their own insurance after comparing plans is surely a paradigm shift in the healthcare industry. However, the bigger question is, what’s next? According to White House officials, the overarching goal of the ACA is to enroll more than 10 percent of the country’s population who are currently uninsured W W W. I N F O R M S . O R G

Figure 1: National supply and demand projections for FTE registered nurses (2018-2025). Source: http://www.aha.org/research/reports/tw/chartbook/ch5.shtml

into the umbrella of “affordable” healthcare through insurance. So what happens when all those people successfully buy insurance through their state or federal insurance marketplaces and seek care? Chances are many of the newly insured people have complex chronic conditions for which they rarely received care in the past. Is the chronic care delivery model equipped to handle that rising demand? Hospitals with an average of 4 percent operating margin [1] are consolidating at an accelerated rate to stay viable. In addition, nearly 50 percent of physicians and 45 percent of nurses are over 50 and are close to retirement. With a looming shortage of primary care physicians and nurses (Figure 1), who historically are the key providers for the chronic care A NA L Y T I C S

patients, the health systems in the United States are about to face a daunting task: keeping patients out of hospitals and other healthcare facilities while maintaining high patient satisfaction! This might sound like an oxymoron but that’s exactly what is needed. If providers cannot do that effectively, overcrowding in clinics and emergency rooms is inevitable, adding to the chaos. This brings us to the healthcare’s microeconomics problem: If demand outstrips supply, should we see direct cost rise to achieve equilibrium? Since there will be an artificial price ceiling, could it create more shortages and a decline in quality and patient satisfaction? Effective population health management is the answer, and therein lies the opportunity for analytics. J A N U A R Y / F E B R U A R Y 2 014

|

53

ANALY TIC S & H E A LT H CA R E

POPULATION HEALTH MANAGEMENT Population health management is about improving health outcomes for a group of individuals and removing health inequities. Caring for a population’s health is anything but trivial. A 1961 report, which was vindicated 40 years later in 2001, showed that only 27 percent of the patient population reports any health issue to clinical providers [2]. This means a health system managFigure 2: Steps in population health management. (Source: Population Health Management, Institute for Health ing a large patient populaTechnology Transformation) tion having one or more chronic conditions often does not have overweight and a smoker, and 3) patients access to accurate information about its who are relatively healthy with low utilizapatients. It takes patient data, interoperation of the health care resources. bility and analytics to address this problem. The middle group is the largest group, As Figure 2 demonstrates, many steps and Lisa Bielamowicz, M.D, chief mediand tasks are involved in managing popucal officer of the Advisory Board Comlation health effectively. The application pany, described them as the “rising risk” of health IT systems such as electronic patients. It’s just a matter of time before medical records (EMR), telehealth, disthey become part of a high-cost patient ease registries and analytics is a mandacluster. Effective management of this tory requirement for success. Any given group is important to reduce the health population can be divided into three main system’s total-cost risk exposure. Findclusters: 1) patients with high utilization ing out who they are and when they need of health care resources, 2) patients with intervention is a key to managing a risktwo or more primary diseases, typically based payment model [3]. Unlike bringing 54

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

patients into the medical facilities, this approach requires predictive insights and, therefore, presents a big opportunity for analytic solutions. Providers and thought leaders have been talking about this use case for healthcare analytics for quite some time. The looming migration toward a risk-based payment model where keeping patients in good health is rewarded by CMS through the Shared Savings program is now acting as the key driver. Some commercial payers (e.g., WellPoint, Aetna) have followed suit.

MEDICATION NON-ADHERENCE: THE $290 BILLION PROBLEM In a research report, the New England Health Institute (NEHI) has shown that patients not adhering to their medication regimen is a $290 billion problem in U.S. healthcare [4]. Medication management and improved adherence are not only critical tools to ensure hospital readmission rates stay under control and providers avoid penalties, but also to address compliance needs and financial incentives of stakeholders in the healthcare value chain.

2013 Innovative Applications in Analytics Award Finalist Presentations Online Organizations want to take advantage of tools and techniques to improve their decision making but don’t always know how to start. View these on-demand videos from top-notch analytics experts to improve your analytics maturity.

SECTION ON ANALYTICS

Join the Community for inside information.

http://analytics.section.informs.org

A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

55

ANALY TIC S & H E A LT H CA R E

Although better medication adherence can marginally increase a payor’s cost per patient, better outcomes and cost savings from preventable disease exacerbations save money in the long run. Analytics play a major role in both predicting patient cohorts who are at risk of non-adherence and the kind of intervention that works best for them.

56

|

Providers who also have pharmacies need EHNAC (Electronic Healthcare Network Accreditation Commission) certification, and medication adherence is an important part of certification. The HEDIS (Healthcare Effectiveness Data and Information Set) score that measures medication management is another key metric that payors pay attention to because it impacts their health plans and ultimately their profits. Payors who offer Medicare Advantage plans, for example, are concerned about CMS provided-STAR ratings for their plans because more stars mean a higher bonus amount (tens of millions of dollars for large plans). Medication adherence is a key aspect for STAR ratings. Although better medication adherence can marginally increase a payor’s cost per patient, better outcomes and cost savings from preventable disease exacerbations save money in the long run. Analytics play a major role in both predicting patient cohorts who are at risk of non-adherence and the kind of intervention that works best for them. Analytic models can consider hundreds of different factors in a patient population and improve accuracy of the prediction. Medication adherence is also a key component of achieving effective population health management, so it makes sense for providers, payors or accountable care organizations to apply predictive analytics to address both problems at the same time. Forward-looking health care organizations such as Partners Healthcare, Baylor Healthcare System, Kaiser Permanente and Intermountain Healthcare have already done that. The next few years will be a bit chaotic but exciting for the healthcare industry. Data and analytics will be harnessed

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

like never before to deliver better care for the population as a whole, as well as individual patients. Rajib Ghosh ([email protected]) is an independent consultant and business advisor with 20 years of technology experience in various industry verticals where he had senior level management roles in software engineering, program management, product management and business and strategy development. Ghosh spent a decade in the U.S. healthcare industry as part of a global ecosystem of medical device manufacturers, medical software companies and telehealth and telemedicine solution providers. He’s held senior positions at Hill-Rom, Solta Medical and Bosch Healthcare. His recent work interest includes public health and the field of IT-enabled sustainable healthcare delivery in the United States as well as emerging nations. Follow Ghosh on twitter @ghosh_r.

REFERENCES 1. AHA Research Report, “Aggregate Total Hospital Margins, Operating Margins, and Patient Margins, 1991-2011”; http://www.aha.org/research/reports/tw/ chartbook/ch4.shtml 2. L.A. Green, et al., 2001, “The Ecology of Medical Care Revisited,” New England Journal of Medicine, Vol., 344, No. 26, June 2001; www.nejm.org/doi/ pdf/10.1056/NEJM200106283442611 3. Mark Hagland, “Slicing and Dicing the Populations Within Population Health: One Industry Expert’s View”; www.healthcare-informatics.com/ article/slicing-and-dicing-populations-withinpopulation-health-one-industry-expert-s-view 4. NEHI, “Improving Patient Medication Adherence: A $290 Billion Opportunity”; www.nehi.net/ bendthecurve/sup/documents/Medication_ Adherence_Brief.pdf

BECOME A CERTIFIED ANALYTICS PROFESSIONAL www.informs.org/Build-Your-Career/Analytics-Certification

®

Be among the first to become a Certified Analytics Professional (CAP®). Make plans now to take the profession's first analytics certification exam.

BENEFITS OF CERTIFICATION

• Advances your career potential by setting you apart from the competition • Drives personal satisfaction of accomplishing a key career milestone • Helps improve your overall job performance by stressing continuing professional development • Recognizes that you have invested in your analytics career by pursuing this rigorous credential • Boosts your salary potential by being viewed as experienced analytics professional • Shows competence in the principles and practices of analytics

COST

• $495 INFORMS Members • $695 Non-Members • Bundled rates with meetings available

QUESTIONS? [email protected] A NA L Y T I C S

JANUARY 11 Northern Virginia

MARCH 29 Boston, MA MARCH 30 Las Vegas, NV

JANUARY 29 Tuscaloosa, AL MARCH 6 Philadelphia, PA

JUNE 21 San Jose, CA

APPLICATIONS

• Prepare to apply by reviewing Candidate Handbook & Study Guide Draft • Arrange now to secure academic transcript and confirmation of “soft skills” to send to INFORMS

UPCOMING CAP® EXAMS

DOMAINS OF ANALYTICS PRACTICE Domain Description Weight* Business Problem (Question) Framing 15% I Analytics Problem Framing II 17% III Data 22% IV Methodology (Approach) Selection 15% Model Building V VI Deployment VII Life Cycle Management *Percentage of questions in exam

16% 9% 6% 100%

N O V E M B E R / D E C E M B E R 2 013

|

57

WO RK IN G W I T H DATA

Missing values The origin, detection, treatment and consequences of missing values in analytics.

BY GERHARD SVOLBA Missing values – and how to deal with them – is an inevitable problem for statisticians, data miners or anyone working with analytical data. Missing values in the data create uncertainty for the analyst and the information consumer because decisions need to be made without having the full picture. Missing values should trigger a discussion about randomness and systematic patterns, as they might introduce more fuzziness and/or bias into the picture. Missing values can also reduce the number of usable records for the analysis, or force analysts to eliminate variables from the analysis. This happens for a technical reason, since many analytical methods

M

58

|

A N A LY T I C S - M A G A Z I N E . O R G

such as regression techniques, neural networks or cluster algorithms cannot deal with missing values, as they require numeric values to be used in a mathematical equation. Consequently, if an observation has a missing value in any of the required variables, the whole observation (data record) needs to be omitted from the analysis. Other options would be to exclude it from the analysis variable as a whole or to insert imputation values for the missing data points. (However, in descriptive statistics or with decision trees, missing values can simply be represented as a separate category.) Along with the technical consideration of what to do with missing values, it is also important, from a business point of view, to W W W. I N F O R M S . O R G

decide whether a sufficient number of records remain for the analysis to produce meaningful results. The following example illustrates how the true distribution can become distorted when the source of missing values is not identified properly. THE STORY OF MY AUNT SUSANNE (Or missing values in the “age variable” in the customer database of a telephone provider) My Aunt Susanne purchased her phone in the mid-60s. Her date of birth

c

ontinuing ducation

ESSENTIAL PRACTICE SKILLS FOR ANALYTICS PROFESSIONALS Topic areas:

» Problem Framing » Developing the Work Plan » Testing Recommendations

Next time I get a visionary assignment that needs some clarity, I’ll be using what I learned in this course to work towards a great solution! - Caroline Alexander, Fed Ex Corporation

”

A NA L Y T I C S

ANALYTICS PROFESSIONALS

INFORMS Continuing Education program offers intensive, two-day in-person courses providing analytical professionals with key skills, tools, and methods that can be implemented immediately in their work environment.

Topics areas:

This course will be held Atlanta, GA – February 20-21, 2014 Boston, MA – March 28-29, 2014 San Jose, CA – June 20-21, 2014

Learn more about these courses at: informs.org/continuinged

COURSES FOR

DATA EXPLORATION & VISUALIZATION

» Presenting Results » Impact Assessment

“

was not collected at that time as the philosophy of “know your customers” and the need for customer data was nowhere near as vital then as it is today. Things changed in the 1990s with the deregulation of the telecommunications market; suddenly, the analysis of customer behavior became important. Since then, it has become mandatory for customers to provide their date of birth on a new contract or with a contract change. My aunt, however, never changed her contract type or answered any customer questionnaire.

» Accessing Data » Understanding Raw Data » Cleaning & Transforming Data » Exploring & Visualizing Data » Dimension Reduction

“

The datasets on which the exercises are based are taken from real-life scenarios, are fun to work with and very challenging. The course provides a general framework for tackling data analysis and the instructors highlight the pitfalls one can made along the process. - Ivan Hernandez, Stevens Institute of Technology

”

This course will be held Boston, MA – March 28-29, 2014 San Jose, CA – June 25-26, 2014

J A N U A R Y / F E B R U A R Y 2 014

|

59

MIS S IN G VA LU E S

Figure 1:Distribution of variable “age” in a customer database.

Thus, the field “date of birth” is missing in the customer database of her phone provider, and we can assume she is not the only customer with a missing value. If an analyst now looks at the distribution of variable “age” in this customer database, he might get a histogram as shown in Figure 1. Additionally, he will see that he has 9.1 percent missing values. The question is how to treat these missing values. • Shall the mean be used as imputation value? • Shall different imputation values be sampled from the actual distribution? In our case, we can assume that the true age value for Aunt Susanne and her friends is not distributed over the whole 60

|

A N A LY T I C S - M A G A Z I N E . O R G

range of values. After a certain year it was mandatory to provide the date of birth with new contracts. So the missing values will mostly occur for a certain age segment (the older customers) and probably also for a certain behavior segment (those who did not change their contract type). In the Figure 2 histogram, the true distribution of the unknown age values is shown in red. We realize that we would make a wrong assumption when we treat the missing values as random, as we found out that there is a systematic pattern behind them. In order to qualify such a situation correctly, business and process knowledge is needed. This know-how is also important to formulate an adequate imputation rule as the imputation values should be from the W W W. I N F O R M S . O R G

Figure 2: True distribution of unknown age values shown in red.

interval 55 to 95 years rather than covering the whole age range. Calculating just the proportion of missing values per variable does not really help to uncover such situations. In this case we would just have seen 9.1 percent missing in the age variable. Such an analysis only tells us which characteristics are most commonly infected by the “missing value disease” in the data. For our purpose, we need to find a way that uncovers the relationship of the missing values to other variables or features of the customer. One method is to create an indicator variable “age missing YES/NO” and compare the distribution of other variables between these two groups. So we A NA L Y T I C S

might see that the missing age values occur with “old contract types” or have a specific phone behavior (Aunt Susanne is not making international phone calls or having data traffic, she is just phoning her friends from time to time). Another method that can efficiently be used to uncover the structure of the missing values is to analyze the “missing value patterns” and show these patterns in tile charts. For each record in the data a string of 0s and 1s is created, “1” indicates a missing value for the respective variable, “0” otherwise. The first character of this string could stand for variable AGE, the next for variable GENDER, the third for variable DATA TRAFFIC. Thus a string of “101” would indicate that for this J A N U A R Y / F E B R U A R Y 2 014

|

61

MIS S IN G VA LU E S

record the variables AGE and DATA TRAFFIC are missing and the variable GENDER is not missing. With such a representation (Figure 3), it can be seen at one glance that about 60 percent of the records don’t have a single missing value (pattern 000000000000), Figure 3: Representation of missing records. and that another 30 percent of the records have a missing value in only one of the variables (light blue). The little red cells show groups of records, where already five or more variables are missing. This information is important to decide whether missing values shall be imputed by analytical methods or not. Such a representation method is well suited to Figure 4: X-axis represents the increasing proportion of missing values, Y-axis shows the relative average response rate of the detect patterns of missing predictive model. values in the variables. It provides an answer to the question: Which missing values occur be treated differently in marketing actions together and helps to define segments in and probably have demand for specific the data? In our case we would probably hardware (phone with large keys, simple find a segment “Aunt Susanne and her usage, etc.). Or they may need special asfriends.” Customers in this segment should sistance through the customer care hotline. 62

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

SYSTEMATIC MISSING VALUES REALLY MATTER From the above example we learned that systematic patterns in the occurrence of missing values really matter. In order to illustrate the quantitative effect of random or systematic missing values on model quality, simulation studies have been performed. For scenarios with varying proportions of missing values and different types of missing values, the values have been set to “missing” in the analysis data. As a next step the missing data have been imputed with the mean before they were used in the predictive model. From the results shown in Figure 4, there is clearly a remarkable difference in model quality when dealing with different types of Figure 5 (top) and Figure 6 (bottom): Machine missing values; the blue and green downtime in a factory over 12 weeks. lines represent the scenarios with random missing values and lay example. Another customer may refuse to higher than their counterparts from the answer a question so no value is entered scenarios with systematic missing values in, for example, the field “number of chil(red and brown lines). dren.” Such cases can be easily detected How do I know that something is missand selected by database queries. But not ing? This question may sound trivial; a all missing values reveal themselves in missing value in a table can be recognized such an explicit way. Consider the Figure by the fact that a cell is empty. Aunt Su5 and Figure 6 that show machine downsanne’s missing date of birth value is one time in a factory over 12 weeks. A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

63

MIS S IN G VA LU E S

When looking at the course over time, we might assume that the diagrams in Figure 5 and Figure 6 represent different data (maybe different years or different factories). In truth the graphs are built with the same data that is shown in Table 1: WEEK 1 3 7 8 11

DOWNTIMES 12 16 8 7 15

Table 1 From a pure technical point of view, no values are missing and the “number of failures” is always filled. However, from a content point of view we do indeed have missing data: There is no entry for weeks 2, 4, 5, 6 and 10. It now has to be decided whether this means that in week 2 there were no failures or if no information exists for week 2. The red line in Figure 5 uses the data as they are and assumes a missing value for week 2 that is just interpolated with a straight line. The blue line in Figure 6 treats the missing weeks in the data as weeks with zero failures and interprets the data as shown in table 2: This example illustrates that in timeseries data we can have missing data in 64

|

A N A LY T I C S - M A G A Z I N E . O R G

WEEK 1 2 3 4 5 6 7 8 9 10 11 12

DOWNTIMES 12 0 16 0 0 0 8 7 0 0 15 0

Table 2 the form of missing records that are not explicitly visible. Only by reviewing the continuity of the time axis do we detect these missing values. Figure 7 shows methods that are frequently used to detect and impute missing values for time series data. The original data only had 12 records, the records for May, July and November 1949 where automatically inserted into the data and different replacement values were inserted. • ZERO VALUE: assuming that if a period is not represented in the data, no traffic occurred for that period • The LAST KNOWN VALUE: if no information for a period is available the value is assumed to be the same as in the previous period W W W. I N F O R M S . O R G

Figure 7: Methods to detect, impute missing values for time series data.

• The MEAN VALUE: assuming that a missing value is best represented by the mean of the existing values in the time series • The INTERPOLATED VALUE: here a most likely value is found based on spline interpolations The analysis has to be checked where data points are missing (where the time series has “holes”) and how these holes shall be interpreted from a business point of view. These considerations then lead to the decision of how the missing values shall be imputed. SUMMARY

the data. Business considerations are needed to decide how they shall be detected and handled. The aim is to get a more complete picture and to remove biases and patterns. Analytical methods help to detect missing values, to provide optimal replacement values and to simulate the consequences on model quality. Gerhard Svolba ([email protected]) works for SAS Austria as an analytic solution architect. He is the author of the SAS Press books “Data Preparation for Analytics Using SAS” and “Data Quality for Analytics Using SAS” and speaks at international analytics conferences about the necessary pre-steps before statistical analyses can start. To download the presentations click here.

In analytics, missing values are more than just a technical feature of A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

65

CO N FERE N C E P R E V I E W S

Davenport to headline INFORMS Analytics Conference Set for March 30-April 1, 2014, in Boston, event will feature “the best analytics work being done today.” “Our goal is to bring together the best analytics work being done today that helps organizations make smarter, fact-based business decisions. The conference emphasis will be on real-world analytics across industries and applications, along with skill-building tutorials, software training, and exceptional networking.” – Freeman Marvin, CAP, conference chair

Thomas Davenport, analytics pioneer and thoughtleader, will headline the 2014 INFORMS Conference on Business Analytics and O.R. with a keynote address on “Analytics 3.0: Where Big Data and Traditional Analytics Meet.” The meeting, scheduled for March 30-April 1, is expected to draw more than 900 analytics professionals to Boston for three days of intensive engagement and real-world learning on the science and art of analytics and operations research. Davenport’s 2006 Harvard Business Review article and best-selling book, “Competing on Analytics,” launched the revolution that has made analytics the hottest business trend and “data scientist” the sexiest job title in business today. His most recent book, “Keeping Up with the Quants: Your Guide to Understanding and Using Analytics,” has been called “the quantitative literacy guide” for managers. Davenport is the President’s Distinguished Professor of Information Technology & Management at Babson College, Fellow of the MIT Center for Digital Business, cofounder of the International Institute for Analytics and senior advisor to Deloitte Analytics. SHOWCASE OF AWARD-WINNING ANALYTICS In addition to Davenport’s keynote, attendees will see presentations from the “best of the best” in applied

66

|

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

analytics and O.R. Each IBM, Chevron, Amtrak, SAS, finalist for the 2014 Franz Chevron and UPS, as well Edelman Award, the highas leading universities and est international award for government agencies. The achievement in operations recommittee develops the topic search, will present their work tracks, selects speakers and on March 31, with the winner organizes the presentations announced at the Awards that comprise the heart of the Gala and Banquet that eveconference. Thomas Davenport’s ning. The 2014 champion “Our goal is to bring tokeynote address is will join a prestigious lineup gether the best analytics titled, “Analytics 3.0: of past Edelman Award winwork being done today that Where Big Data ners, including the Dutch Delhelps organizations make and Traditional ta Program Commissioner, smarter, fact-based busiAnalytics Meet.” HP, Memorial Sloan-Kettering ness decisions,” says MarCancer Center, General Motors and Nethvin. “The conference emphasis will be erlands Railways. Other high-impact work on real-world analytics across industries will be showcased throughout the meetand applications, along with skill-building ing in talks by finalists and winners of the tutorials, software training, and excepINFORMS Prize, Daniel H. Wagner Prize, tional networking.” UPS George D. Smith Prize, Innovative The conference committee has desApplications in Analytics Award, Gary L. ignated eight topical tracks for the 2014 Lilien Marketing Science Practice Prize invited speaker program: The Analytand the coveted Spreadsheet “Guru” Prize. ics Revolution, Healthcare Applications, HAND-PICKED TOPICS AND SPEAKERS The key to the conference’s ongoing success is its program committee, chaired this year by Freeman Marvin, vice president and executive principal analyst at Innova tive Decisions, Inc. The 36 members of the committee include analysts and managers from companies such as Google, Target, A NA L Y T I C S

Big Data, Marketing Analytics, Decision Analysis, Soft Skills and Supply Chain Management. New this year is a track organized by the INFORMS Roundtable that will feature world-class O.R. projects in established and mature analytics companies. The program will be rounded out by six tracks of member-contributed talks plus additional tracks on software solutions. J A N U A R Y / F E B R U A R Y 2 014

|

67

CO N FERE N C E P R E V I E W S

Focused Tracks • The Analytics Revolution • Big Data • Marketing Analytics • Healthcare Applications • INFORMS Roundtable: OR/MS in Practice • Supply Chain Management • Decision Analysis • Soft Skills

PRESENT A POSTER AND SAVE ON REGISTRATION While most speakers are hand-picked by the committee, interested students and professionals may submit a poster proposal and receive a discount off regular registration rates. The submission deadline is Jan. 17. The poster format lends itself to works-in-progress on which the presenter is looking for feedback, successful projects that may not be extensive enough for a 50-minute talk, and corporate or consulting work subject to nondisclosure restrictions that can still be presented at a high level. CAREER-BUILDING PROGRAMS

Plus: • Select presentations on a range of topics • Poster presentations

68

|

Two special programs within the conference are designed for future analytics leaders. The Early Career Connection, for junior faculty and young industry researchers, provides participants with insights into the critical problems facing business, helps them broaden their research agendas and provides important networking contacts. The INFORMS Professional Colloquium offers a full-day, interactive workshop on career guidance for master’s and Ph.D. students who are interested in careers outside of academia. The colloquium will be held March 29, and students may then attend the remainder of the conference. Participants in both programs can register for the full conference at a discounted rate. However, candidates must be nominated and selected to attend. Limited financial aid is available for the colloquium. The nomination deadline is Feb. 14.

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

WORKSHOPS ON TECHNOLOGY AND SOFT SKILLS While the conference formally kicks off with welcome receptions on Sunday (March 30) evening, the entire day is packed with valuable learning opportunities. Attendees can take advantage of in-depth technology workshops presented by software solutions companies, as well as a full-day workshop on soft skills. The Soft Skills Workshop follows a case study through various phases of an analysis project, providing participants with hands-on experience in the skills that can make or break a successful implementation. The INFORMS career service, called Analytics Connect, also kicks off on Sunday, enabling employers and job seekers to mingle in a casual environment. SUPER-SAVER CONFERENCE RATES Super-saver rates of $915 for INFORMS members and $1,150 for nonmembers are available until Feb. 10. Organizations can take advantage of the $827 team discount rate when they send three or more attendees. All meals for two days are included in the fees. The conference venue, the new Westin Boston Waterfront, is located in Boston’s exciting waterfront area, with great restaurants, shops and public green space – plus easy access to all the history and attractions that make Boston such a fascinating city. For additional conference information, visit: meetings.informs.org/analytics2014 A NA L Y T I C S

Special Programs within the Conference • Richard E. Rosenthal Early Career Connection: Exclusive networking program for junior faculty and young industry practitioners. Reduced conference registration rate $394; nominations due Feb. 14 • INFORMS Professional Colloquium: Full-day, intensive career guidance for master’s and Ph.D. students interested in practice. Reduced conference registration rate $394; nominations due Feb. 14 • Soft Skills Workshop: Fullday on the “soft” skills needed to work in teams and present results to decision-makers; $75. • Technology Workshops: In-depth training from leading solution providers. Free to conference registrations. • Birds-of-a-Feather: Facilitated discussion groups that provide the opportunity to share common challenges and solutions.

J A N U A R Y / F E B R U A R Y 2 014

|

69

CO N FERE N C E P R E V I E W S

From data-rich to decision-smart INFORMS conference to focus on the business of big data. The conference will be organized around several tracks that focus on business, with success stories and les­sons learned on real implementation of big data analytics. Sessions on big data 101 will offer tutorials on how to navi­g ate the big data ecosystem, how to se­lect and use the right technologies, as well as the challenges of building data science teams.

70

|

“Big data” is the latest business buzzword – but what’s missing among the talk and the hype is how a company gets from data discovery to real business value. The Institute for Operations Research and Management Science (INFORMS), the leading professional membership organization in the world for advanced analytics and publishers of Analytics magazine, is launching a new topical conference in 2014 that will put the focus squarely on the business of big data – how organizations can transition from just data-rich to decision-smart. The meeting will be held June 22-24 at the San Jose Convention Center in San Jose, Calif. The program is being designed by a committee of practitioners in the data arena, chaired by Margery Connor, CAP, senior operations researcher-Advanced Analytics at Chevron, and Diego Klabjan, CAP, professor and director of the Master of Science in Analytics Program at Northwestern University. Other committee members are analysts and managers at Dell, IBM, Booz Allen Hamilton, SAIC, Intel, SAS, Alcatel-Lucent and UPS. The committee will invite speakers to address defined topics in an intensive program that is geared to the

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

interests of business deciCase studies of big sion-makers, IT managers data projects that iland analytics professionals. lustrate the complete Bill Franks, chief analytjourney from business ics officer at Teradata Corpoproblem to analytics soration, will deliver a keynote lution will be a major address on “Putting Big Data component of the conferto Work.” At Teradata and ence. These sessions will throughout his career, Franks outline key steps such as has focused on translating scoping the problem, inBill Franks, chief complex analytics into terms frastructure issues, adanalytics officer at that business users can unvanced analysis on the Teradata Corporation, derstand and then helping data, visualization and will deliver a keynote address on “Putting organizations implement the supporting the decision Big Data to Work.” results effectively within their process. One of the conprocesses. His work has ference goals is to help spanned clients in a variety of industries, bridge the gap between business ranging in size from Fortune 100 comdecision-makers and analytics propanies to small nonprofit organizations. fessionals as they work together on Franks is author of the book, “Taming transforming data into insight and reThe Big Data Tidal Wave” and an active turn on investment. Other sessions on speaker and blogger. the program will look into the future, The conference will be organized exploring emerging technologies and around several tracks that focus on business trends. business, with success stories and lesThe conference will be held at the sons learned on real implementation of San Jose Convention Center in a new big data analytics. Sessions on big data expansion of the building that opened 101 will offer tutorials on how to naviin October 2013. Rooms are being gate the big data ecosystem, how to seheld for conference attendees at the lect and use the right technologies, as Marriott San Jose, which is connected well as the challenges of building data to the Center. science teams. Critical topics such as For continuing updates on INFORMS ethics and privacy requirements will Big Data, visit: http://meetings.informs. also be addressed. org/bigdata2014/ A NA L Y T I C S

J A N U A R Y / F E B R U A R Y 2 014

|

71

FIVE- M IN U T E A N A LYST

Markov’s nursery Because there are two children, there are a total of four states that the children could be in: 1. both sleeping; 2. Mary only sleeping; 3. Neil only sleeping; and 4. both crying.

BY HARRISON SCHRAMM, CAP 72

|

This month, we tackle a problem that may be familiar to some readers – the issue of getting multiple young children to sleep. We’ll also use this column to (re)introduce some neat mechanics – the generator matrix. Suppose that a family has two infant children, named Mary and Neil. Now, at bedtime, for analytic purposes, they exist in one of two states: “crying” or “sleeping” [1]. Because there are two children, there are a total of four states that the children could be in: 1. both sleeping; 2. Mary only sleeping; 3. Neil only sleeping; and 4. both crying. We would like to know the amount of time the system (nursery) spends in each state, particularly the proportion of time both children are asleep. There’s a minor twist to this problem – we assume that if one child is crying, it will reduce the amount of time that the other child is sleeping by half if they are sleeping, or lengthen the amount of time that the other child stays awake if they are currently awake.

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

Know what’s hot. The topic of analytics is on fire right now. With SAS, you can discover innovative ways to increase profits, reduce risk, predict trends and turn data assets — whether big or small — into competitive advantage. Decide with confidence. ®

sas.com/know

Learn why Forrester Research Inc. named SAS “an analytics powerhouse.” (Forrester Research, Inc., The Forrester Wave™: Big Data Predictive Analytics Solutions, Q1 2013) SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. © 2013 SAS Institute Inc. All rights reserved. S105652US.0213

FIVE- M IN U T E A N A LYST

There are several approaches we could take to solving this problem. One approach would be to simulate it, another would be to consider the discrete-time, discrete-space Markov chain and compute the limiting distributions the usual way. We’re going to take a different approach, which requires a bit of machinery but ultimately is more straightforward. We’re going to use the generating function of the continuous time Markov Chain, which we (and others!) denote as the G matrix [2]. This matrix differs from the more commonly used P matrix in discrete time chains because it is specified in terms of rates, not probabilities. Have no fear, knowledge of one fundamental matrix implies the other. The practical difference is that the rows of the P matrix sum to 1, while the rows of the G sum to zero, with the on-diagonal elements being negative. For instance, we might specify:

Where λ represent the sleeping times of the children, and μ represents their wakeful times. To make use of our model, we need to use P 't = tG , which is readily solved in matrix form as P(t ) = etG 74

|

A N A LY T I C S - M A G A Z I N E . O R G

To evaluate this expression, we need to consider what it means to take e to a matrix power. This turns out not to be any harder than taking e to a scalar power – you just have to use Taylor’s Theorem:

Depending on your programming environment, efficient algorithms exist to compute this numerically. As an example, if we take λm=.1λn=.125, μm=.2, μn= .2, and recalling that the rate parameters of exponential random variables are the reciprocals of their expectations, we see that Mary is a child who sleeps for 10 hours at a stretch (without interruption from her brother), and Neil is a child who sleeps an average of eight hours at a time. Both infants are up for an average of five hours at a time between sleep stretches. We can use these values to populate the G matrix mentioned earlier. To find the long-run behavior of the system, we choose a value of t which is large enough so that the system is stable but small enough to avoid problems with numerical stability. We select, somewhat arbitrarily, t = 30 to make sure that transients are out of the system. Now, it’s simply a matter of evaluating the expression. We find W W W. I N F O R M S . O R G

Figure 1: Probability that both children remain sleeping as a function of time, given that they were initially asleep. This chart was made by evaluating the matrix exponential at various points premultiplied by the scalar time, demonstrating the usefulness of this method. that in our example that both children are asleep 30 percent of the time, Mary only is sleeping 18 percent of the time, Neil only is sleeping 14 percent of the time, and both children are crying 36 percent of the time. Note that the children do not have equal sleeping behaviors. This is because Mary has a little lambda. We’ve (somewhat sloppily) found the limiting distribution, but we may do a great deal more. Suppose that both children are currently asleep. We wish to compute the probability that they will still be asleep in one hour. This is easy; we simply compute P(1) = eG and pre-multiply the result by A NA L Y T I C S

the initial condition vector (1,0,0,0), which strips off the top row, and we see that there is an 81 percent chance that both children will still be sleeping in an hour. Harrison Schramm (harrison.schramm@gmail. com) is an operations research professional in the Washington, D.C., area. He is a member of INFORMS and a Certified Analytics Professional (CAP).

NOTES 1. Real children, of course, exist in many states as they grow older. 2. “G Matrix” is another math-rap name ripe for the picking!

J A N U A R Y / F E B R U A R Y 2 014

|

75

THIN K IN G A NA LY T I CA LLY

Golf queueing

BY JOHN TOCZEK John Toczek is the senior director of Decision Support and Analytics for ARAMARK Corporation in the Global Operational Excellence group. He earned a bachelor of science degree in chemical engineering at Drexel University (1996) and a master’s degree in operations research from Virginia Commonwealth University (2005). He is a member of INFORMS.

76

|

Golfing can be an enjoyable and rewarding way to spend your time. Despite the attraction and fun of the game, there can be many challenges. One of the more common challenges for experienced players is waiting for slower players to finish a hole before the experienced player can start. As the owner of a nine-hole golf course, you currently have a first-in-first-out policy. In other words, faster players are not allowed to jump ahead of slower players. You are considering changing this first-in-first-out policy to a priority queueing policy to allow faster players to jump ahead of slower players in between holes. Players arrive at your golf course at an inter-arrival time of 10 minutes, exponentially distributed. The players on your golf course have three different skill levels. Fast players complete holes at an average of five minutes. Medium players complete holes at an average of seven minutes. Slow players complete holes at an average of 10 minutes. All distributions are normal and have a standard deviation of one minute. Player skill level is randomly distributed (one-third fast, one-third medium, one-third slow). Assume players start golfing as soon as they arrive on the course and that the system has achieved steady state. Each player is golfing individually (not in a group), and players must go in sequential order from hole 1 to hole 9. Players can only jump the queue if a slower player has not yet started the hole. Question: How much time on average (in minutes) will a player save if you convert to the priority queueing from first-in-first-out queueing? Send your answer to [email protected] by March 15, 2014. The winner, chosen randomly from correct answers, will receive a $25 Amazon Gift Card. Past questions can be found at puzzlor.com.

A N A LY T I C S - M A G A Z I N E . O R G

W W W. I N F O R M S . O R G

OPTIMIZATION

www.gams.com

High-Level Modeling The General Algebraic Modeling System (GAMS) is a high-level modeling system for mathematical programming problems. GAMS is tailored for complex, large-scale modeling applications, and allows you to build large maintainable models that can be adapted quickly to new situations. Models are fully portable from one computer platform to another.

State-of-the-Art Solvers GAMS incorporates all major commercial and academic state-of-the-art solution technologies for a broad range of problem types.

GAMS Integrated Developer Environment for editing, debugging, solving models, and viewing data.

PAVER 2: The next generation of the GAMS Performance Tools PAVER 2 automates the analysis and comparison of solver performance data. The use of the Python Data Analysis Library (http://pandas.pydata.org/) ensures platform independence, simple use, high performance, and flexibility. PAVER 2 highlights: • Easy customization of performance metrics • Computation and visualization of performance statistics • Automated handling of inconsistent solver outcomes • Integration with GAMS/EXAMINER solution point analyzer

Europe GAMS Software GmbH [email protected] USA GAMS Development Corporation [email protected]

http://www.gams.com

PAVER 2 is open-source and available at: http://www.gamsworld.org/performance/paver2/

To find an expert to help you, log onto INFORMS Find an Analytics Consultant Database at https://www.informs.org/Find-Analytics-Consultant/Search. INFORMS is the foremost association of analytics and O.R. experts. Our members literally wrote the book on how analytics and the principles of operations research are used to improve corporate or governmental decision making. If you're looking for an analytics professional, a software package based on analytics methods, or an expert witness, visit the database. You can search for independent analytics professionals and analytics consultants. You can also search by your application or industry need. Regardless of the scale or type of organizational challenge, there is an analytics professional who can deliver the value you're looking for. Two methods for finding an Analytics Consultant are available, a keyword search to seek matches on the entered keyword or filter checkboxes to filter consultants by their particular provider types, application areas, and vertical industry experience. INFORMS members may add themselves to the database. https://www.informs.org/Find-Analytics-Consultant/Search

Certified Analytics Professional candidate handbook

®

second edition

INFORMS Certified Analytics Professional (CAP ) Examination ®

CANDIDATE HANDBOOK

5521 Research Park Drive, Suite 200, Catonsville, MD 21228 USA 443-757-3500 800-4INFORMS (800-446-3676) Last revised 12/18/2013

INTRODUCTION....................................................................................................................................... 1 About INFORMS................................................................................................................................................ 1 About the INFORMS Certified Analytics Professional (CAP®) Program........................................................ 1 Vision and Mission Statements of the CAP® Program.................................................................................... 3 Nondiscrimination Policy................................................................................................................................... 3 Eligibility Requirements..................................................................................................................................... 3 About the Professional Job Task Analysis Process.......................................................................................... 4

APPLYING FOR AND SCHEDULING AN EXAMINATION...................................... 7 Steps in the Certification Process..................................................................................................................... 7 Application and Payment Submission Process................................................................................................ 8 Fees and Refunds............................................................................................................................................... 8 Scheduling an Examination............................................................................................................................... 9 Cancellation of an Examination........................................................................................................................ 9 Special Examination Arrangements................................................................................................................ 10

PREPARING FOR THE EXAMINATION............................................................................... 11 Sample Test Questions.................................................................................................................................... 11 References........................................................................................................................................................ 17 Continuing Education and Training Courses................................................................................................. 18

TAKING THE EXAMINATION..................................................................................................... 19 Exam Site Requirements and Instructions...................................................................................................... 19

AFTER THE EXAMINATION......................................................................................................... 20 Examination Score Reports and the Scoring Process................................................................................... 20 Hand Scoring.................................................................................................................................................... 20 Reexamination.................................................................................................................................................. 20 Security and Confidentiality............................................................................................................................ 20

APPEALS OF CERTIFICATION DECISIONS..................................................................... 21 Appeals Eligibility............................................................................................................................................. 21 Appeals Process............................................................................................................................................... 21 Appeals Panel................................................................................................................................................... 21 Appeals Decisions............................................................................................................................................ 21

i. ®

DISCIPLINE PROCESS....................................................................................................................... 22 CODE OF ETHICS/CONDUCT...................................................................................................... 23 CERTIFICATION RENEWAL PROCESS................................................................................ 25 Professional Development Unit (PDU) Requirements................................................................................... 25 Recording PDUs and the Audit Process......................................................................................................... 25 Transfer of Excess PDUs to the Next Renewal Cycle..................................................................................... 26 Changes in the Name and Address of Certificants....................................................................................... 26

USE OF THE CAP® CREDENTIAL.............................................................................................. 26 CONTACT INFORMS............................................................................................................................ 26 APPENDIX CAP® Certification Application and Agreement Confirmation Statement on Analytics Soft Skills Certification Examination Special Accommodations Form

ii. ®

About INFORMS

INFORMS is the world’s largest professional society for those in the field of analytics, operations research (O.R.), and the management sciences. INFORMS is committed to meeting the professional needs of those who develop, apply, research, and teach advanced analytics, O.R. and the management sciences. INFORMS works to promote a broader understanding and recognition of the field by encouraging, facilitating, and rewarding excellence; communicating appropriate achievements and capabilities; providing lifelong education and career development opportunities; and attracting the best people.

About the INFORMS Certified Analytics Professional (CAP®) Program

INFORMS is pleased to announce the development and implementation of a professional certification program that meets the needs of analytics professionals. The Certified Analytics Professional (CAP®) program was developed in 2011–2012. The first examination was conducted on April 7, 2013, at the INFORMS Analytics Conference, April 7–9, in San Antonio, Texas. INFORMS analytics certification program advances the use of analytics by setting agreed upon standards for the profession and advances the profession by providing a means for organizations to identify and develop qualified analytics professionals, by contributing to the career success and continued competence for analytics professionals, and by improving the credibility and visibility of the analytics profession. INFORMS defines analytics as the scientific process of transforming data into insight for making better decisions. Analytics is seen as an end-to-end process beginning with identifying the business problem to evaluating and drawing conclusions about the prescribed solution arrived at though the use of analytics tools and methodologies. Analytics professionals are skilled at this process. The CAP® examination measures acceptable performance across seven major areas or domains of practice that adhere to the analytics end-to-end process: business problem framing, analytics problem framing, data, methodology selection, model building, deployment, and model life cycle management. See the “About the Professional Job Task Analysis Process” section for more information about the end-to-end process that is covered by the exam. Because analytics is an end-to-end process, the CAP® examination assesses a breadth of knowledge across the seven domains, but not a depth of knowledge in any one methodology. Those interested in taking the CAP® examination should consider themselves to be analytics professionals or semi-professionals, and not analytics amateurs. They should be interested in adhering to the highest standards of good analytics practice and following a path of continual professional development in analytics. Candidates must have at least three years of experience in analytics depending on their academic degree. See the “Eligibility Requirements” section for more information on qualifying for the CAP® examination. The CAP® program formally began in August 2011, when INFORMS established the INFORMS Certification Task Force to plan and develop this new analytics certification.

1 ®

Following the first CAP® examination in April, 2013, an independant Analytics Certification Board (ACB) began administering the INFORMS analytics certification program. The ACB replaced the INFORMS Certification Task Force and became the official governing body. The ACB has independent authority to make all final decisions regarding program procedures, program content, approval of applicants and granting of certification independent of input from INFORMS governance including INFORMS Board of Directors. Members of the Analtyics Certification Board are listed below: • Chair: Scott Nestler, PhD, CAP, US Army • Vice-Chair: Polly Mitchell-Guthrie, SAS • Tom Davenport, PhD, Babson College • Bill Franks, Teradata • Jeanne Harris, Accenture • Terry Harrison, PhD, CAP, Penn State University • Lisa Kart, CAP, Gartner • Kathy Kilmer, Disney • Don Kleinmuntz, PhD, Strata Decision Technology LLC • Jack Levis, UPS • Paul Messinger, PhD, CAP, University of Alberta • Jonathan Owen, PhD, General Motors • Melissa Moore, INFORMS Exec. Director, Ex officio INFORMS is the first professional society to develop a professional certification program for analytics professionals. Key components of the CAP® program include the following: 1. Formal credentialing requirements, including a standardized examination and required renewal process. 2. Program content based on the findings of a job task analysis working group, whose members represent a broad background of analytics practitioners (see the section titled “About the Professional Job Task Analysis Process”). 3. Agreed-upon eligibility criteria that consist of academic preparation, work experience in analytics, and an attestation from an employer confirming adequate mastery of soft skills in analytics. 4. Certification program content that is both software and vendor neutral. 5. Successful completion of the certification process enabling analytics professionals (and their employers) to have confidence that they bring a core set of analytics skills to a project team.

2 ®

Vision and Mission Statements of the CAP® Program Vision To advance the use of analytics to transform the world by setting agreed-upon standards for the profession. Mission To advance the analytics profession by providing a high-quality program of certification and by promoting continuing competence for practitioners.

Nondiscrimination Policy

INFORMS does not discriminate among candidates on the basis of race, color, creed, gender, age, religion, national origin, ancestry, disability, military discharge status, sexual orientation, or marital status. INFORMS strives to adhere to all applicable laws and regulations pertaining to nondiscrimination practices. INFORMS will arrange for reasonable accommodation for any individual requesting it.

Eligibility Requirements

Eligibility requirements for the CAP® credential include the following. Experience •

Applicants must have o at least three (3) years of professional analytics related experience for individuals holding an MA/MS degree or higher in a related educational area (educational areas considered to be related to analytics are enumerated under “Education” below) OR o at least five (5) years of professional analytics related experience for individuals holding a BA/BS degree in a related educational area (educational areas considered to be related to analytics are enumerated under “Education” below) OR o at least seven (7) years of professional analytics related experience for individuals holding a BA/BS degree or higher in an educational area unrelated to analytics.

•

Applicants are expected to provide information on professional experience including the name of the company/employer, name and title of the immediate supervisor, contact information for the immediate supervisor, and dates of employment in a professional analytics position.

Education •

A BA/BS degree or an MA/MS degree in an analytics-related area is recommended for consideration for the CAP® program. These analytics-related areas include, but are not limited to, the following: analytics, operations research, management science, statistics, engineering, business (and directly related areas such as marketing, finance, etc.), theoretical or applied mathematics, information technology, computer science, decision science, and others deemed appropriate by INFORMS Analytics Certification Board. The degree must be obtained from a regionally accredited college or university recognized by the U.S. Department of Education or similar entities in other countries.

•

INFORMS requires submission of a photocopy or electronic version of an official university or college transcript to demonstrate compliance with all education requirements.

3 ®

Confirmation of Soft Skills An important trait of a Certified Analytics Professional is the demonstrated achievement of an acceptable level of “soft skills,” in addition to the knowledge, skills, and abilities covered by the CAP® examination process. These soft skills include, but are not limited to, the following: •

Ability to communicate with a client/employer regarding the framing of an analytics problem.

•

Understanding of the background of the client/employer regarding its organization and specific industry focus.

•

Ability to explain the findings of the analytics process in sufficient detail to ensure clear understanding by the client/employer.

The CAP® program relies on a confirmation by a person in a supervisory capacity that the candidate possesses the skills enumerated above. The confirmation process requires that all applicants obtain documentation of their acceptable soft skills proficiency in the official confirmation form (see Appendix A for the Employer Confirmation Statement.) It requires the signature of a current or previous supervisor of analytics work who is not a relative of the candidate and is designed for current or previous employers of individuals hired to provide professional analytics services. Individuals who can show that they are unable to contact a past employer, client, or acceptable substitute, may submit a written summary of a recent analytics project describing in detail the application of soft skills in the successful completion of the project and in the implementation of its findings. Candidates choosing this option should contact INFORMS staff for additional information. Representatives of the INFORMS Analytics Certification Board may also, in some cases, require a telephone interview with a candidate in addition to the written summary to assess an applicant’s soft skills.

About the Professional Job Task Analysis Process

The Job Task Analysis (JTA) study defines the current knowledge, skills, and abilities (KSAs) that must be demonstrated by analytics professionals to effectively and successfully provide these services. KSAs are validated according to their frequency of use and importance. The JTA also serves as a “blueprint” for the content (performance domains) of the INFORMS CAP® examination. INFORMS upholds stringent guidelines for the construction and implementation of the examination development and administration process. An 11-member panel of subject matter experts (SMEs) was selected to develop the first JTA for the CAP® credential. This group was called the Analytics Certification Job Task Analysis Working Group. The following leaders in the analytics profession comprised the first panel selected to participate in this important project: •

Arnold Greenland (IBM Global Business Services)

•

Bill Klimack (Chevron)*#

•

Jack Levis (UPS)*#

•

Daymond Ling (Canadian Imperial Bank of Commerce)N

•

Freeman Marvin (Innovative Decisions, Inc.)

•

Scott Nestler (U.S. Army)#

•

Jerry Oglesby (SAS)

•

Michael Rappa (North Carolina State/Institute for Advanced Analytics)# * INFORMS Board Member Tim Rey (Dow Chemical) # Credentialing Task Force Member Rita Sallam (Gartner)N N Nonmember of INFORMS Sam Savage (Stanford/Vector Economics)

• • •

4 ®

The findings of this working group were then validated by a random sample of practicing analytics professionals. Feedback from this survey resulted in slight modifications of the performance domains, tasks, and knowledge that comprise the test blueprint that determines the content of the CAP® examination. The table below includes the final domains and their representation on the certification exam that were derived from the JTA and a review of validation survey recommendations. Domain

Approximate Weight

I. Business Problem (Question) Framing

12%–18%

II. Analytics Problem Framing

14%–20%

III. Data

18%–26%

IV. Methodology (Approach) Selection

12%–18%

V. Model Building

13%–19%

VI. Deployment

7%–11%

VII. Model Life Cycle Management

4%–8%

The INFORMS CAP® examination is based on the following test blueprint derived from the JTA process. The final agreed-upon weights reflect the percentage of questions from each domain that will be included in each test form. The JTA and the test blueprint resulting from this process will be reviewed periodically and updated as needed to reflect current practices in analytics. The following list of domains also includes the key tasks associated with each domain.

(12%–18%)

Domain I

Business Problem (Question) Framing

(The ability to understand a business problem and determine whether the problem is amenable to an analytics solution.) T-1

Obtain or receive problem statement and usability requirements

T-2

Identify stakeholders

T-3

Determine whether the problem is amenable to an analytics solution

T-4

Refine the problem statement and delineate constraints

T-5

Define an initial set of business benefits

T-6

Obtain stakeholder agreement on the problem statement

(14%–20%)

Domain II

Analytics Problem Framing

(The ability to reformulate a business problem into an analytics problem with a potential analytics solution.)

5 ®

T-1

Reformulate problem statement as an analytics problem

T-2

Develop a proposed set of drivers and relationships to outputs

T-3

State the set of assumptions related to the problem

T-4

Define key metrics of success

T-5

Obtain stakeholder agreement

(18%–26%)

Domain III

Data

(The ability to work effectively with data to help identify potential relationships that will lead to refinement of the business and analytics problem.) T-1

Identify and prioritize data needs and sources

T-2

Acquire data

T-3

Harmonize, rescale, clean, and share data

T-4

Identify relationships in the data

T-5

Document and report findings (e.g., insights, results, business performance)

T-6

Refine the business and analytics problem statements

(12%–18%)

Domain IV

Methodology (Approach) Selection

(The ability to identify and select potential approaches for solving the business problem.) T-1

Identify available problem solving approaches (methods)

T-2

Select software tools

T-3

Test approaches (methods)1

T-4

Select approaches (methods) 1

(13%–19%)

Domain V

Model Building

(The ability to identify and build effective model structures to help solve the business problem.) T-1

Identify model structures1

T-2

Run and evaluate the models

T-3

Calibrate models and data1

T-4

Integrate the models1

T-5

Document and communicate findings (including assumptions, limitations, and constraints)

(7%–11%)

Domain VI

Deployment

(The ability to deploy the selected model to help solve the business problem.) T-1

Perform business validation of the model

T-2

Deliver report with findings; OR

T-3

Create model, usability, and system requirements for production

T-4

Deliver production model/system1

T-5

Support deployment

(4%–8%)

Domain VII

Model Life Cycle Management

(The ability to manage the model life cycle to evaluate business benefit of the model over time.) T-1

Document initial structure

T-2

Track model quality

T-3

Recalibrate and maintain the model1

T-4

Support training activities

T-5

Evaluate the business benefit of the model over time

Tasks that are beyond the scope of the CAP® certification exam and that will not be tested.

1

6 ®

Steps in the Certification Process 1. Potential applicant decides to apply for certification and determines whether he or she meets the eligibility requirements by reading the Candidate Handbook. 2. Applicant secures a photocopy or electronic version of an official transcript from his or her regionally accredited college or university documenting the required academic preparation for the CAP® credential. 3. Applicant provides contact information for a recent supervisor of analytics in the workplace who will confirm soft skills. 4. Applicant completes and submits the online application form and sends it to INFORMS with payment of the required application fee. (Application fee is waived for INFORMS members.) 5. INFORMS staff reviews application to determine whether the candidate is eligible to be scheduled for the examination. 6. INFORMS staff provides the eligible candidate with information on the available exam administration sites. 7. Candidate reviews available exam sites, chooses one, and remits exam fee to INFORMS. This step must be completed at least 10 days prior to the preferred examination date. 8. Candidate takes examination at the designated test site. All examinations are scored by a consulting psychometrician or INFORMS certification staff. 9. INFORMS certification staff sends the candidate a written pass/fail notice by postal mail marked CONFIDENTIAL or email within 20 business days of the completion of the examination. Failing candidates will also receive information about their examination performance in major content areas. 10. Eligible candidates who pass the examination will be sent a certificate and information regarding the use and display of the certification logo within six to eight weeks of the completion of their examination. 11. Failing candidates should begin a process of targeted professional development to address the weak performance areas cited in their examination results letter. NOTE: If application is incomplete, i.e., either transcripts or Confirmation Statement on Analytics Soft Skills are not provided, candidates may be eligible to test. If candidate passes the exam, he or she will not be awarded the CAP credential until the application is complete. Candidates will have 90 days from date of exam to complete the application. If candidate is unsuccessful, he or she will have 90 days to complete the application. Applications must be completed in order to re-test. If application is not completed in 90 days, candidates will have to reapply as new applicants. 12. All certificants must renew their certification every three (3) years. See the “Certification Renewal Process” section for more information. Note: Certification is only granted to individuals who meet all eligibility requirements and achieve a passing score on the examination. Grandfathering of certification status is not permitted for any candidates failing to meet these requirements.

7 ®

Application and Payment Submission Process www.informs.org/applyforcertification

To apply for the CAP® certification examination, please follow these steps: 1. Complete and submit to INFORMS the “CAP® Certification Application and Agreement” form online at www.informs.org/applyforcertification 2. Submit a photocopy or electronic version of your official university or college transcript to INFORMS 3. Provide contact information for a previous or current employer who will complete the “Confirmation Statement on Analytics Soft Skills” form 4. Agree to the Code of Ethics 5. Remit application payment fee to INFORMS (waived for INFORMS members) Once accepted, choose test site location and remit exam payment fee to INFORMS NOTE: If application is incomplete, i.e., either transcripts or Confirmation Statement on Analytics Soft Skills are not provided, candidates may be eligible to test. If candidate passes the exam, he or she will not be awarded the CAP credential until the application is complete. Candidates will have 90 days from date of exam to complete the application. If candidate is unsuccessful, he or she will have 90 days to complete the application. Applications must be completed in order to re-test. If application is not completed in 90 days, candidates will have to reapply as new applicants. INFORMS strives to process applications in a timely manner. Process time is seven (7) business days from receipt of application, payment, and accompanying documents sent online or via postal mail. This timeline does not apply to the Confirmation Statement on Analytics Soft Skills form sent from employers. INFORMS will process payment and other materials while awaiting receipt of this form. Applicants can submit payment of the certification fees in any of the following ways: credit card (MasterCard, Visa, American Express, Discover), check made payable to INFORMS in U.S. dollars drawn on a U.S. bank, or wire transfer (contact INFORMS for details).

Fees and Refunds

All INFORMS certification fees must be drawn on U.S. banks and payable in U.S. dollars. Specific fees may change from time to time based on the decisions of INFORMS. INFORMS offers discounts on certification fees to current INFORMS members. INFORMS may offer discounts to other certification partners. If your organization is interested in becoming a certification partner, please contact INFORMS. Refunds may be provided in some circumstances, but not after a candidate has begun an examination. INFORMS reserves the right to impose a processing fee on certification exam refunds. Application fees are nonrefundable.

8 ®

The following fees are currently approved for the CAP® program: Application fee (waived for INFORMS members) $100 Exam fee: INFORMS membersa $495 Exam fee: Nonmembers $595 Annual maintenance fee

$100, payable beginning 4th year of certification

Member reexamination feeb $300 Nonmember reexamination feeb $400 Hand-scoring fee $75 Processing fee on approved refunds

$100

Appeals processing fee $150 Applicants must be a member of INFORMS in good standing at the time of application to receive the member discount and avoid application fee. No refunds will be granted for applicants joining INFORMS after submission of certification application materials. a

Reexamination fees apply to second and third attempts to pass the CAP® examination within the first year of submitting an approved application. See the section titled “Reexamination” for more details.

b

Scheduling an Examination

Once an application is accepted and exam fee paid, applicants may choose an examination site. INFORMS certification staff will send written confirmation of the scheduled examination date, time, and site. The certification website lists updated information on additional examination sites as they become available.

Cancellation of an Examination

Should the need arise to cancel a scheduled CAP® examination, candidates must notify INFORMS certification staff at least ten (10) business days prior to the scheduled examination date. Candidates who contact INFORMS more than ten (10) business days prior to the examination date will have their examination rescheduled at no additional cost. Refunds may be provided in some circumstances less a $100 processing fee. Candidates who cancel their examination less than 10 business days before their examination will not receive a refund and will be required to pay the full examination fee in order to reschedule their examination. INFORMS understands that there may be situations where cancellations are required by circumstances beyond the control of candidates such as the following: •

Natural disasters

•

Medical emergencies

•

Death or illness in the immediate family

•

Travel cancellations or power failure due to inclement weather

INFORMS certification staff will handle these situations on a case-by-case basis. Candidates will be expected to submit written documentation along with possible supporting documentation to avoid a potential cancellation penalty.

9 ®

INFORMS reserves the right to cancel any examination due to inclement weather, power failure, or other unforeseen circumstances that make holding the exam untenable. Affected candidates will have their examinations rescheduled at no additional cost.

Special Examination Arrangements Candidates with Disabilities INFORMS complies with the Americans with Disabilities Act (ADA). INFORMS strives to ensure that no individual with a documented disability is deprived of the opportunity to take the certification examination solely by reason of that disability provided that reasonable special accommodations can be made. To request special accommodations, candidates must complete the INFORMS Certification Examination Special Accommodations form (Appendix B). The form includes a statement of the disability and a history of previous accommodations in education, training, assessment, or the workplace. Candidates must provide all documentation with their application and fees at least 45 days prior to a desired examination date. INFORMS also requires that applicants notify INFORMS of any requests for special accommodations when calling to schedule examinations. Candidates Requesting Other Special Arrangements Religious Obligations. If attendance at a scheduled examination date conflicts with a candidate’s religious obligations, INFORMS will attempt to arrange an alternate examination day. INFORMS must receive these requests at least 45 days prior to the scheduled examination date. Limited English Proficiency. At this time, INFORMS certification examinations are offered in English only. If English is not a candidate’s first language, the candidate may request additional testing time by submitting documentation establishing birth or schooling in a non-English-speaking country. INFORMS must receive this information at least 45 days prior to the scheduled exam date.

10 ®

The CAP® examination is a 100-item written test composed of multiple-choice questions with four options. There is only one correct or best answer for each question. Candidates will be given three (3) hours to complete the examination.

Sample Test Questions

The following sample test questions were developed by subject matter experts in the analytics field. The correct answer key at the end of this list of questions provides the correct answers to each question. These sample questions will never appear in an actual CAP® examination and are provided as an example of the type of question found on the examination. The 24 questions published here are intended to familiarize certification candidates and potential certification candidates with the format of the questions that appear on the CAP® examination. They are also intended to provide a sample of the content (knowledge and skill) assessed by the CAP® examination. These questions are not intended as a self-assessment instrument nor should they be used to predict success or failure on the CAP® exam. Candidates and potential candidates should bear in mind that the CAP® examination is a “pass/ fail” assessment and that passing does not require correct answers to all questions. It should also be kept in mind that examination preparation efforts will likely increase knowledge and sharpen skills. 1. Which of the following BEST describes the data and information flow within an organization? a) Information assurance b) Information strategy c) Information mapping d) Information architecture 2. A multiple linear regression was built to try to predict customer expenditures based on 200 independent variables (behavioral and demographic). 10,000 rows of data were fed into a stepwise regression, each row representing one customer. 1,000 customers were male, and 9,000 customers were female. The final model had an adjusted R-squared of 0.27 and seven independent variables. Increasing the number of rows of data to 100,000 and rerunning the stepwise regression will most likely: a) have no impact upon the adjusted R-squared. b) increase the impact of the male customers. c) change the heteroskedasticity of the residuals in a favorable manner. d) decrease the number of independent variables in the final model. 3. A clothing company wants to use analytics to decide which customers to send a promotional catalogue in order to attain a targeted response rate. Which of the following techniques would be the most appropriate to use for making this decision? a) Integer programming b) Logistic regression c) Analysis of variance d) Linear regression

11 ®

4. Which of the following is an effective optimization method? a) Analysis of variance (ANOVA) b) Generalized linear regression model (GLM) c) Box-Jenkins Method (ARIMA) d) Mixed integer programming (MIP) 5. A box and whisker plot for a dataset will MOST clearly show: a) the difference between the second quartile and the median. b) the 90% confidence interval around the mean. c) where the [actual-predicted] error value is not zero. d) if the data is skewed and, if so, in which direction. 6. In the kickoff meeting with a client for a new project, which of the following is the MOST important information to discuss? a) Timeline and implementation plan b) Analytical model to use c) Business issue and project goal d) Available budget 7. Which of the following statements is true of modeling a multi-server checkout line? a) A queuing model can be used to estimate service rates. b) A queuing model can be used to estimate average arrivals. c) Variability in arrival and service times will tend to play a critical role in congestion. d) Poisson distributions are not relevant. 8. A company is considering designing a new automobile. Their options are a design based on current gasoline engine technology or a government proposed “Green” technology. You are a government official whose job is to encourage automakers to adopt the “Green” technology. You cannot provide funding for development costs, but you can provide a subsidy for every car sold. The development costs and the wholesale price, in thousands of dollars, of the cars are shown in the table below: Gasoline Technology (numbers in $ thousands)

“Green” Technology (numbers in $ thousands)

Wholesale Price/vehicle

25

40

Variable Cost/vehicle

15

35

100,000

200,000

Fixed Cost

How large a subsidy per vehicle sold will be required, assuming there will be enough demand to motivate the switch? a) Greater than $5000 b) Less than $5000 c) Cannot be determined d) Equal to $5000.

12 ®

9. A furniture maker would like to determine the most profitable mix of items to produce. There are well-known budgetary constraints. Each piece of furniture is made of a predetermined amount of material with known costs, and demand is known. Which of the following analytical techniques is the MOST appropriate one to solve this problem? a) Optimization b) Multiple regression c) Data mining d) Forecasting 10. You have simulated the net present value (NPV) of a decision. It ranges between -$10 million and +$10 million. To BEST present the likelihood of possible outcomes, you should: a) present a single NPV estimate to avoid confusion. b) present a histogram to show likelihood of various NPV ranges. c) trim all outliers to present the most balanced diagram. d) relax constraints associated with extreme points in the simulation. 11. A company ships products from a single dock at their warehouse. The time to load shipments depends on the experience of the crew, products being shipped and weather. The company is considering building another dock in order to meet unmet demand. Which is the MOST appropriate modeling approach to determine if the revenue from the additional products sold will cover the cost of the second dock within two years of it becoming operational? a) Optimization because it is a transportation problem. b) Optimization because the company’s objective to maximize profit and capacity at the dock is a limited resource. c) Forecasting because you can determine the throughput at the dock, calculate the net revenue and compare this with the cost of the new dock. d) Discrete event simulation because there are a sequence of discrete random events through time. 12. Two investors who have the same information about the stock market buy an equal number of shares of a stock. Which of the following statements must be true? a) The risks for the two investors are statistically independent. b) Both investors have the same risk profile. c) Both investors are subject to the same uncertainty. d) If the investors are optimistic, they should have borrowed, rather than bought the shares.

13 ®

13. A project seeks to build a predictive data-mining model of customer profitability based upon a series of independent variables including customer transaction history, demographics, and externally purchased credit-scoring information. There are currently 100,000 unique customers available for use in building the predictive model. Which of the following strategies would reflect the BEST allocation of these 100,000 customer data points? a) Use 70,000 randomly selected data points when building the model, and hold the remaining 30,000 as a test dataset. b) Use all 100,000 data points when building the model. c) Build four separate models and randomly partition the data into 4 separate datasets of 25,000 data points each. d) Use 1,000 randomly selected data points when building the model. 14. Conjoint analysis in market research applications can: a) give its best estimates of customer preference structure based on in-depth interviews with a small number of carefully chosen subjects. b) only trade off relative importance to customers of features with similar scales. c) allow calculation of relative importance of varying features and attributes to customers. d) only trade off among a limited number of attributes and levels. 15. One of the main advantages of tree-based models and neural networks is that they: a) are easy to interpret, use, and explain. b) build models with higher R squared than other regression techniques. c) reveal interactions without having to explicitly build them into the model. d) can be modeled even when there is a significant amount of missing data. 16. The monthly profit made by a clothing manufacturer is proportional to the monthly demand, up to a maximum demand of 1000 units, which corresponds to the plant producing at full capacity. (Any excess demand over 1000 units will be satisfied by some other manufacturer, and hence yield no additional profit.) The monthly demand is uncertain, but the average demand is reliably estimated at 1000 units. At this level of demand the monthly profit is $3,000,000. Which of the following statements must be true of the expected monthly profit, P? a) P can have any positive value. b) P is possibly greater than $3,000,000. c) P is equal to $3,000,000. d) P is less than $3,000,000. 17. After building a predictive model and testing it on new data, an under prediction by a forecasting system can be detected by its: a) negative-squared. b) bias. c) mean absolute deviation. d) mean squared error.

14 ®

18. All times in the decision tree below are given in hours. What is the expected travel time (in hours) of the optimal (minimum travel time) decision?

a) 7.8 b) 6.9 c) 7.4 d) 7.0

19. An analytics professional is responsible for maintaining a simulation model that estimates system throughput given different staffing levels required for a specific operational business process. Assuming that the operational team always uses the number of staff determined by the model, which of the following is the MOST important maintenance activity? a) Ensure that all of the model input data items are available when needed. b) Determine if there has been a change in model accuracy over time. c) Ensure that all users are reviewing the model results in a timely fashion. d) Determine that the model’s reports are understood by the users. 20. A segmentation of customers who shop at a retail store may be performed using which of the following methods? a) Monte Carlo Markov Chain and ANOVA b) Clustering, factor and control charts c) Decision tree and recursive function analyses d) Clustering and decision tree 21. In the diagram below, what is true of Strategy B compared to Strategy A? Cumulative Probability Curves NPV - Millions US$

cum. probability 1 0.9 0.8

Strategy A

0.7

Strategy B

0.6 0.5 0.4 0.3 0.2 0.1 0

(400)



(200)

-

200

400

600

NPV, Millions US$

15 ®

800

1,000

1,200

1,400

1,600

a) Strategy B exhibits stochastic (probabilistic) dominance over Strategy A. b) Strategy B has the same downside risk as Strategy A since the curves have the same shape. c) Strategy B must have the same uncertainties impacting it as Strategy A because the curves are so similar in shape. d) Strategy A exhibits stochastic (probabilistic) dominance over strategy B. 22. Each month you generate a list of marketing leads for direct mail campaigns. Which of the following should you do before the list is used? a) Exclude people who were on the list the previous month. b) Retain x% of the leads as control for performance measurement. c) Remove opt-outs. d) Exclude people who were never on the list. 23. When analyzing responses of a survey of why people like a certain restaurant, factor analysis could reduce the dimension in which of the following ways? a) Collapse several survey questions regarding food taste, health value, ingredients and consistency into one general unobserved “food quality” variable. b) Condense similar survey respondent answers into clusters of like-minded customers for market segment analysis. c) Reduce the variability of individual subject ratings by centering each respondent’s ratings around his or her average rating. d) Decrease variability by analyzing inter-rater reliability on the question items before offering the survey to a wide number of respondents. 24. A preferred method or best practice for organizing data in a data warehouse for reporting and analysis is: a) transactional-based modeling. b) multidimensional modeling. c) relation-based modeling. d) tuple-based modeling.

16 ®

Correct answers: 1. d

7. c

13. a

19. b

2. a

8. a

14. c

20. d

3. b

9. a

15. c

21. a

4. d

10. b

16. d

22. c

5. d

11. d

17. b

23. a

6. c

12. c

18. d

24. b

(Note: None of these questions will appear in any CAP® examination. These sample questions are presented to candidates to demonstrate the format of questions that will be included in the actual exams.)

Distribution of sample questions per domain Domain I: Business Problem Framing

Questions 6, 8, 10, 12

Domain II: Analytics Problem Framing

Questions 7, 14, 16, 20

Domain III: Data

Questons 1, 2, 5, 23, 24

Domain IV: Methodology (Approach) Selection

Questions 3, 4, 9, 11

Domain V: Model Building

Questions 13, 15, 18, 21

Domain VI: Deployment

Questions 17, 22

Domain VII: Model Lifecycle Management

Question 19

References

INFORMS subject matter experts compiled the following list of key references that may help you prepare for the CAP® exam. Domain I – Business Problem (Question) Framing Kirkwood CW (1997) Strategic Decision Making: Multiobjective Decision Analysis with Spreadsheets (Duxbury Press, Pacific Grove, CA). Domain II – Analytics Problem Framing Albright SC, Winston W, Zappe C (2011) Data Analysis and Decision Making, 4th ed. (South-Western Cengage Learning, Mason, OH).

17 ®

Domain III – Data Hubbard DW (2010) How to Measure Anything: Finding the Value of “Intangibles” in Business, 2nd ed. (John Wiley & Sons, Hoboken, NJ). Hillier F, Hillier M (2013) Introduction to Management Science: A Modeling and Case Study Approach, 5th ed. (McGraw-Hill Higher Education, New York). Vose D (2008) Risk Analysis: A Quantitative Guide, 3rd ed. (John Wiley & Sons, Chichester, UK). Domain IV – Methodology (Approach) Selection Neter J, Kutner M, Nachtsheim C, Wasserman W (1996) Applied Linear Statistical Models, 4th ed. (McGraw-Hill/Irwin, New York). Domain V – Model Building and Domain VII – Model Life Cycle Management Hillier FS, Lieberman GJ (2010) Introduction to Operations Research, 9th ed. (McGraw-Hill, New York). Ross SM (2010) Introductory Statistics, 3rd ed. (Academic Press, Burlington, MA). Clemen RT (1997) Making Hard Decisions: An Introduction to Decision, 2nd ed. (Duxbury Press, Pacific Grove, CA). Law AM, Kelton DW (2006) Simulation Modeling and Analysis, 4th ed. (McGraw-Hill, New York). Domain VI – Deployment Laursen GHN, Thorlund J (2010) Business Analytics for Managers: Taking Business Intelligence Beyond Reporting (John Wiley & Sons, Hoboken, NJ).

Continuing Education/Training Courses

Along with the above-mentioned references, there are also many analytics-related continuing education and training courses available from many suppliers. INFORMS provides a list of Recognized Analytics Continuing Education Providers on our website for the convenience of candidates, at www.informs.org/RACEP. INFORMS does not endorse any of these courses or ensure the accuracy of the listings. Completion of preparatory courses is not required for eligibility to sit for the CAP® examination.

18 ®

Exam Site Requirements and Instructions Check-in Procedure Candidates should arrive at the test site at least 30 minutes before the scheduled examination time. Candidates must bring a government-issued photo ID (e.g., passport, driver’s license). Absent an official government ID with picture, a secondary ID that includes a photo and/or signature may be used. Examples of acceptable secondary IDs include valid employee IDs, valid credit cards with candidate’s signature, and valid bank/ATM cards. Candidates will not be permitted to take the examination without proper identification. Prohibited Items in the Testing Room Candidates may not bring any of the following items to the test center: If there is no designated secure storage, candidates may bring the items into the testing room but they will be placed in an inaccessible location within the room during the examination.

•

calculators

•

cell phones/smartphones

•

laptops

•

iPads or similar devices

•

tape recorders

•

book bags

•

pagers

•

notes of any kind

•

books

•

newspapers

Testing Aids All candidates will be provided with pencils, scrap paper, and a simple four-function calculator. Disciplinary Policy and Procedures Candidates are expected to conduct themselves in a professional manner in the test center. Any violation may be subject to disciplinary action from the INFORMS Certification Department up to and including dismissal from the examination site. Grounds for termination of the examination and dismissal from the test center include the following:

• Having or attempting to have another individual take the examination • Failing to provide the proper identification • Using any prohibited test aids/materials • Communicating in any manner with other candidates during the administration of the examination • Leaving the test room or center without permission • Engaging in cheating or any other dishonest or unethical conduct • Failing to follow any of the test administration rules as stipulated by INFORMS

19 ®

Examination Score Reports and the Scoring Process

INFORMS uses a criterion-referenced methodology for determining the passing score for its examinations. There is no grading on a “curve,” and candidates are not competing with each other. The specific methodology used is the modified Angoff technique, which relies on the judgments of SMEs to determine an acceptable level of knowledge, skill, and ability in analytics. INFORMS may at times include pretest items in some examination forms. These items are used for developing future examinations and, accordingly, are not scored and have no impact on a candidate’s pass/fail status. Each candidate will receive an examination report in the form of a pass/fail letter including additional information on certification status. This letter will be sent by postal mail and/or email. The letters for failing candidates will include additional information on performance by domain. Pass/fail letters will not include raw scores. Pass/fail letters will be sent to all candidates within 20 business days. Raw examination scores are confidential. INFORMS will not disclose examination scores to anyone unless INFORMS is required by court order or subpoena. INFORMS will publish the names of all individuals who have passed the examination and who maintain current certification status.

Hand Scoring

Candidates who wish to have their examination results hand scored after the initial scoring process may request this service by contacting INFORMS certification staff and paying a $75 fee. Requests for hand scoring of answer sheets must be received no later than 30 business days following the release of examination results. Requests received after 30 business days will not be processed.

Reexamination

Candidates who do not pass their initial examination have the option of retaking this examination up to two (2) additional times during the first year following the approval of their application. Candidates who take the examination a second or third time will be expected to pay the reexamination fee cited in the “Fees and Refunds” section, each time. Candidates who fail the examination three (3) times will be required to wait one year from the date of their last attempt to reapply for certification. Reapplying under these circumstances involves a complete new submission of all application materials. Candidates who fail the examination three times are encouraged to pursue a program of education/training prior to reapplying for certification.

Security and Confidentiality

All test-related materials including the examination form, test questions, worksheets, and graphics included in test items are the exclusive intellectual property of INFORMS. Accordingly, none of these confidential materials is available for review by any persons other than the INFORMS certification staff and INFORMS Analytics Certification Board. All certification candidates sign an application agreement stating that they will not discuss or share the specific content of any INFORMS certification examinations with anyone. Any violation of this provision could result in potential sanctions up to and including revocation of certification status.

20 ®

Appeals Eligibility

Appealable decisions include the following:

• Denial of eligibility for certification • Denial of certification • Denial of renewal • Revocation of certification

Appeals Process

Applicants, candidates, or certificants wishing to appeal a decision must submit written documentation within thirty (30) days of the receipt of the written decision by INFORMS. The written documentation should specify the grounds on which the appeal is based. A nonrefundable fee of $150 drawn on a U.S. bank in U.S. dollars must be submitted with the letter of appeal.

Appeals Panel

INFORMS will appoint an appeals panel consisting of one (1) current member of the INFORMS Board of Directors and one (1) current/former member of INFORMS Analytics Certification Board. None of these individuals shall have had any affiliation (business, professional, or personal) with the individual filing the appeal. The appeals panel members will conduct their work and render a written decision within 60 business days of their appointment.

Appeals Decisions

The appeals panel shall render a decision on any allegations of procedural error or in the making of a decision with insufficient evidence to support it. Appeals regarding required compliance with existing and published testing standards or program requirements are not accepted. The appeals panel may render a decision to uphold the INFORMS decision, grant the appeal requested by the appellant, or refer the matter back to the INFORMS Analytics Certification Board for reconsideration. A written copy of the appeals panel decision shall be sent to the INFORMS Analytics Certification Board and to the appellant.

21 ®

Certificants are required to comply with all existing and future rules, regulations, and administrative ethical standards for certification established by INFORMS. Certificants are responsible for demonstrating compliance, and failure to do so may lead to disciplinary actions, including but not limited to the denial of eligibility, nonrenewal of a certification, revocation of certification, probation or suspension, issuance of a letter of censure, or issuance of a written reprimand. Individuals may report alleged violations of INFORMS rules or regulations in writing to INFORMS. Written documentation should include the identity of the individual involved in the alleged misconduct and the nature of the misconduct described in as much detail as possible, and the signature of the individual filing the complaint. INFORMS has the authority to initiate a disciplinary action without receiving a complaint or notification of inappropriate conduct. INFORMS reserves the right to pursue any and all civil and legal remedies available under the law. Grounds for disciplinary action include, but are not limited to, the following list: 1. Conviction of any felony involving moral turpitude. 2. Conviction of any other criminal offense which reasonably calls into question the certificant’s ability to provide professional analytics services. 3. Engaging in, authorizing, or aiding or abetting fraud, deceit, misrepresentation of materials/ facts, provision of false or forged evidence, or bribery in connection with any application for a certificate or registration. 4. False statements made in any initial or renewal application materials. 5. Obtaining or attempting to obtain certification or renewal by any fraudulent means. 6. Failure to meet renewal requirements. 7. Use of expired credentials or false or unauthorized use of any INFORMS credentials. 8. Unauthorized possession or distribution of INFORMS examination or testing materials. 9. Unauthorized use of any registered trademark of INFORMS. INFORMS shall have the authority to establish procedures for hearings and potential reinstatement upon satisfactory assurance of proper conduct. Individuals who wish to report a possible certification violation may send a written letter of complaint to INFORMS Certification Manager Institute for Operations Research and the Management Sciences 5521 Research Park Drive, Suite 200 Catonsville, Maryland 21228 USA

22 ®

INFORMS has developed the code of ethics/conduct for all Certified Analytics Professionals [see below]. All candidates and certificants participating in the certification process are required to agree to comply with the current and future provisions of this code.

Code of Ethics for Certified Analytics Professionals Prepared by the INFORMS Certification Task Force Background. The Institute for Operations Research and the Management Sciences (INFORMS) does not have an established code of ethics or guidelines for ethical practice that applies to the general membership. However, Article 1, Paragraph 2.v., of the INFORMS constitution states, “The Institute will strive to promote high professional standards and integrity in all work done in the field.” Applicability. This Code of Ethics applies specifically to those seeking (re)-certification as a Certified Analytics Professional (CAP®), but may be useful to other practitioners who use analytical. Clients, employers, researchers, policymakers, journalists, students and the public should expect analytical practice by CAP® certified individuals to be conducted in accordance with these guidelines. Application of these or any other ethical guidelines generally requires good judgment and common sense. Purpose. This code exists to clarify the ethical requirements that are important; to inform the individual regarding rules and standards; to hold the profession accountable; to aid analytics professionals in making and communicating ethical decisions; to help deter unethical behavior and promote self-regulation; and to list possible violations, sanctions, and enforcement procedures. General. Analytics professionals participate in analysis that aids decision makers in business, industry, academia, government, military, i.e. all facets of society; therefore, it is imperative to establish and project an ethical basis to perform their work responsibly. Furthermore, practitioners are encouraged to exercise “good professional citizenship” in order to improve the public climate for, understanding of, and respect for the use of analytics across its range of applications. In general, analytics professionals are obliged to conduct their professional activities responsibly, with particular attention to the values of consistency, respect for individuals, autonomy for all, integrity, justice, utility, and competence. Responsibilities. This Code recognizes that analytics professionals have obligations to a variety of groups, including: society, employers and clients, colleagues, research subjects, INFORMS, and the profession in general. Responsibilities regarding each of these groups are further described below. Society. All professionals have societal obligations to perform their work in a professional, competent, and ethical manner. Professionals should adhere to all applicable laws, regulations, and international covenants. Employers and Clients. In general, it is the practitioner’s responsibility to assure employers and clients that an analytical approach is suitable to their needs and resources, and include presenting the capabilities and limitations of analytical methods in addressing their problem. Analytics professionals should clearly state their qualifications and relevant experience. It is imperative to fulfill all commitments to employers and clients, guard any privileged information they provide unless required to disclose, and accept full responsibility for your performance. Where appropriate, present a client or employer with choices among valid alternative approaches that may vary in scope, cost, or precision. Apply analytical methods and procedures scientifically, without predetermining the outcome. Resist any pressure from employers and clients to produce a particular “result,” regardless of its validity. Colleagues. Analytics professionals have a responsibility to promote the effective and efficient use of analytical methods by all members of research teams and to respect the ethical obligations of members of other disciplines. When possible, professionals share nonproprietary data and methods with others; participate in peer review, focusing on the assessment of methods not individuals. Respect differing

23 ®

professional opinions while acknowledging the contributions and intellectual property of others. Those professionals involved in teaching or training students or junior analysts have a responsibility to instill in them an appreciation for the practical value of the concepts and methods they are learning. Those in leadership and decision-making roles should use professional qualifications with regard to analytic professionals’ hiring, firing, promotion, work assignments, and other professional matters. Avoid harassment of or discrimination based on professionally irrelevant bases such as race, color, ethnicity, gender, sexual orientation, national origin, age, religion, nationality, or disability. Research Subjects. If a project involves research subjects, including census or survey respondents, an analytics professional will know and adhere to the appropriate rules for the protection of those human subjects. Be particularly aware of situations involving vulnerable populations that may be subject to special risks and may not be able to protect their own interests. This responsibility includes protecting the privacy and confidentiality of research subjects and data concerning them. INFORMS and Profession. Analytics professionals will strive for relevance in all analyses. Each study or project should be based on a competent understanding of the subject-matter issues, appropriate analytical methods, and technical criteria to justify both the practical relevance of the study and the data to be used. Guard against the possibility that a predisposition by investigators or data providers might predetermine the analytic result. Remain current in constantly changing analytical methodology, as preferred methods from yesterday may be may be barely acceptable today and totally obsolete tomorrow. Disclose conflicts of interest, financial and otherwise, and resolve them. Provide only such expert testimony as you would be willing to have peer reviewed. Maintain personal responsibility for all work bearing your name; avoid undertaking work or coauthoring publications for which you would not want to acknowledge responsibility. Alleged Misconduct. Certified Analytics Professionals will strive to avoid condoning or appearing to condone careless, incompetent, or unethical practices. Misconduct broadly includes all professional dishonesty, by commission or omission, and, within the realm of professional activities and expression, all harmful disrespect for people, unauthorized or illegal use of their intellectual and physical property, and unjustified detraction from the reputation of others. Recognize that differences of opinion and honest error do not constitute misconduct; they warrant discussion, but not accusation. Questionable scientific practices may or may not constitute misconduct, depending on their nature and the definition of misconduct used. Do not condone retaliation against or damage to the employability of those who responsibly call attention to possible scientific error or misconduct. References. 1. Saul I. Gass, Ethical guidelines and codes in operations research, Omega 37 (2009), 1044-1050. 2. American Statistical Association, Ethical Guidelines for Statistical Practice, August 7, 1999. 3. U.S. federal regulations regarding human subjects protection are contained in Title 45 of the Code of Federal Regulations, Chapter 46 (45 CFR 46).

24 ®

One of the hallmarks of a strong professional certification program is the implementation of a process to help ensure the continuing competence of certificants in the discipline. INFORMS will use a Professional Development Unit (PDU) system similar to those used in other professional certifications. All certificants must participate in an ongoing formal renewal process to maintain their certification status. Certificants must demonstrate compliance with renewal requirements during their three-year certification cycles.

Professional Development Unit (PDU) Requirements

All CAP® certificants will be required to achieve a minimum of 30 PDUs in a three (3)-year renewal period. The table below describes various options for achieving the required PDUs along with any required minimum or maximum PDUs specified in each category. PDU category

Description of policy

PDU points allowed

Participation as a student in formal education/training programs provided on analytics topics

This option includes courses, One (1) PDU per each hour of seminars, and workshops on analytics- instruction. Certificants must achieve at related issues. least 8 PDUs in this category during the three-year renewal period.

Self-directed learning

This category includes reading articles and books or watching instructional videos on analytics issues.

One (1) PDU per each hour of selfdirected learning.

Creating new analytics knowledge or content including serving as faculty at learning events

Examples in this category include authoring articles, books, newsletters, etc. PDUs are also awarded for serving as faculty at various learning events.

One (1) PDU is awarded for each hour of activity spent in these activities.

Volunteer service

Examples in this category include serving as a volunteer for INFORMS or its regional chapters, working on analytics meetings, or assisting the certification process.

One (1) PDU is awarded for each hour of volunteer service. Certificants may earn a maximum of 10 hours in this category in a three-year period.

Analytics professional work experience

Full-time employment as an analytics professional for a minimum of one (1) year.

Five (5) PDUs are awarded for each full year of employment as an analytics professional.

Certificants may earn a maximum of 10 hours in this category in a three-year period.

All claimed PDUs must be submitted/verified to INFORMS prior to the conclusion of a certificant’s three-year renewal cycle. Certificants are reminded that they will not be able to claim more than the maximum PDUs allowed in any specific category.

Recording PDUs and the Audit Process

Certificants are required to keep accurate records of all professional development activities including all certificates/letters confirming attendance and/or participation in approved education/training programs. Certificants must present all required PDU records to INFORMS before certification renewal will be granted. Certificants can input their PDUs as they are earned on the INFORMS website at https://www.informs.org/ Certification-Continuing-Ed/Analytics-Certification/Maintain-Your-Certification. INFORMS will periodically audit a sample of certificants to verify the PDUs claimed in their renewal application.

25 ®

Transfer of Excess PDUs to the Next Renewal Cycle

Certificants earning more than the required 30 PDUs in their three-year renewal cycle may transfer a maximum of 5 PDUs to their next renewal cycle. The transferred PDUs may be from any category.

Changes in the Name and Address of Certificants

All candidates and certificants must notify INFORMS immediately of any change in their name or address used for purposes of commutation regarding certification matters. Notify INFORMS in one of the following methods: 1. Email notification to [email protected]. 2. Call INFORMS certification staff at +1-443-757-3500 or 1-800-446-3676. 3. Fax notification to INFORMS at +1-443-757-3515. 4. Enter changes to your record online. Note: Individuals who are changing their names must contact INFORMS certification staff to review the necessary legal documentation required to verify such changes.

Once certificants receive written confirmation from INFORMS regarding their new certification status, they may use the CAP® mark after their name. Always list the designation in block (capital) letters. The CAP® designation may only be used in conjunction with a certificant’s name. The CAP® designation should not be used in reference to a company or organization. The CAP® logo may also be used on letterhead and personal websites with INFORMS approval. INFORMS will send new certificants a packet of materials including the following:

• Letter conferring certification status • Information on the dates of the certification cycle for renewal purposes • A unique certification number • A certification certificate INFORMS will publish an official list/registry of all current CAP®s on its website at https://www.informs.org/Certification-Continuing-Ed/Analytics-Certification/Certification-Registry.

For more information about the CAP® program and its policies, please contact us: INFORMS ATTN: Certification Manager 5521 Research Park Drive, Suite 200 Catonsville, Maryland 21228 USA Phone: +1-443-757-3500 or 1-800-446-3676 Fax: +1-443-757-3515 Email: [email protected] Web: www.informs.org/Build-Your-Career/Analytics-Certification

26 ®

CAP® Certification Application and Agreement Please PRINT the following information.

1

Personal Information ®

INFORMS Membership ID # (if applicable)

First name

Middle initial

Last name

Job title

Company/organization name

Address

work

City

office

Non-INFORMS mail lists Non-INFORMS email lists INFORMS email

home

State

Phone

Please do not include me in:

Zip/Postal

home

Country

cell

Primary Employment Classification Academic Government Business Consulting Retired, underemployed, & other

E-mail

I prefer to be excluded from the online certification registry.

2

Education Institution: _________________________________ Highest degree attained: (circle one) PhD

Master’s Bachelor’s HS

Year graduated: _________ Field of Study Analytics Operations research Management science Statistics Engineering Logistics Business (and directly related areas such as marketing, finance, etc.)

Reason for Applying

What is your reason for applying for the CAP® certification?

Professional development Employer suggestion Employer requirement Other (please specify) __________________________________________

Applied or theoretical mathematics Information technology Computer science Decision science Marketing science Data Science Other (please specify) ___________________________________________

Note: At least seven (7) years of analytics-related work experience is required for individuals holding a BS/BA degree or higher in an education area unrelated to analytics. All applicants are required to submit either a photo copy or an official version of their final college or university transcript indicating the date of graduation and degree title. The official transcript or photocopy must be submitted to INFORMS with your application form to determine your eligibility for certification. Final certification status will not be granted until all application documents including transcripts, are received.

CAP® Certification Application and Agreement (cont.)

Please PRINT the following information in your certificate application.

3

Professional Analytics-Related Experience

________________________________________________________________________________________ Job Title

Employer

________________________________________________________________________________________ Employer Street Address

City

State/Province/Territory

Zip/Postal

Country

_______________________________________________________________ Employer Phone number

_______________________________________________________________ Years of analytics experience (overall)

Dates of experience

________________________________________________________________________________________ Description of analytics role/responsibilities

Primary contact who is not a close relative for verification purposes on professional analytics experience:

________________________________________________________________________________________ First name

Relationship (circle one) Supervisor Project sponsor

Last name

Project manager

Client

Other (please specify)________________

________________________________________________________________________________________ Email address

Phone number

Note: All applicants must have a previous employer/supervisor who is not a close relative of applicant submit the Confirmation Statement on Analytics Soft Skills to INFORMS before certification status can be granted. A copy of this Statement is provided in the Candidate Handbook. Candidates may apply for certification and take the certification examination prior to the receipt of this statement by INFORMS, but final certification status cannot be granted until all application documents, including an official signed statement are received by INFORMS. Special Accommodations Request _____ Please check here if you are requesting special accommodations for your examination. Please submit the Certification Special Accomodation form found in the Appendix of this handbook. All supporting documentation must be included with your application and submitted within the required time frame in advance of your anticipated examination date.

INDUSTRY CLASSIFICATION Indicate ALL areas of your professional analytics and OR/MS activities.

4

Agriculture, Forestry, Fishing Analytics Arts and Entertainment Chemical/Process Construction Education

eCommerce Environmental Finance/Insurance Govt (non-military) Healthcare Info Systems & Tech

Law Enforcement Manufacturing Marketing Military Mining, Oil/Gas NonProfit

Payment Information

Certification/examination fees (payable in U.S. dollars) ____ INFORMS member $495 ____ Nonmember $695 Note: Member rates apply only to current INFORMS members in good standing as of the date this application has been submitted or certification partners.

Pharmaceuticals Real Estate Retail Telecommunications Transportation/Warehousing Utilities, Water/Power

Please send completed forms to either [email protected], fax to 443.757.3515, or mail to: Certification Manager INFORMS 5521 Research Park Drive, Suite 200 Catonsville, MD 21228

Payment method: _____ Check Enclosed (Make payable to INFORMS and must be drawn on U.S. bank in U.S. dollars. ) _____ MasterCard _____ Visa _____ American Express _____ Discover

_______________________________________________________________________________________ Card # (15 digits)

Expiration date

_______________________________________________________________________________________ Signature

CAP® Certification Application and Agreement

1 2 3

I agree to comply with and conduct myself in accordance with all INFORMS certification program policies and requirements. In addition, I agree not to discuss, debrief, or disclose, in any manner, the specific content of INFORMS examination questions and answers to any individual. I agree to notify INFORMS in a timely manner regarding changes concerning the information provided, including my current name, address, email address, and telephone number. I agree that INFORMS has the right to communicate with any person, government agency, or organization to review or confirm any of the information submitted in conjunction with my application for certification. Furthermore, I agree to and authorize the release of any information requested by INFORMS for such review and confirmation.

4

I agree that all materials that I submit to INFORMS become the property of INFORMS and that INFORMS is not required to return any of these materials to me.

5

I agree that upon achieving certification status, my name may be posted on the INFORMS certification website as part of an online registry to be created and maintained by INFORMS.

6

I agree that all disputes relating to my application or certification status will be resolved solely and exclusively in accordance with INFORMS certification policies, procedures, and appeals processes.

7

INFORMS reserves the right to suspend or revoke the certification of any individual who is determined to have failed to uphold or otherwise breached this agreement.

8

I agree to return my certificate to INFORMS if for any reason I fail to maintain certification status or if my certificate is revoked for cause.

9

I release and indemnify INFORMS from all liability and claims that may arise related to my analytics and related activities.

10

I hereby release, discharge and indemnify INFORMS, its directors, officers, members, staff, and representatives/agents from any actions, suits, obligations, damages, claims, or any other action taken in connection with this application and my examination. _____ I have read and understand all of the policies and procedures described in the Candidate Handbook. _____ I have read and accept the terms and responsibilities outlined in the INFORMS Certification Application and Agreement. _____ I have read, understand, and agree to the policies and procedures described in the CAP® Code of Ethics. _____ I declare that all of the information I have provided is accurate and true. Furthermore, I understand that any misrepresentation or incorrect information provided to INFORMS can result in disciplinary action, up to and including the suspension of my eligibility for certification. Name (please print)

Signature

Date

®

Confirmation Statement on Analytics Soft Skills Certified Analytics Professional (CAP®) Credential

The bottom of this form must be signed (by hand) by a contact who is familiar with your analytics work. The completed and signed form must be returned to INFORMS by the contact in one of the two methods: 1. The preferred method is for your employer to scan the completed signed form and send it as PDF to the INFORMS Certification Manager at [email protected]. 2. A secondary method is to return the completed signed form to INFORMS by postal mail. Please send completed forms to Certification Manager INFORMS 5521 Research Park Drive, Suite 200 Catonsville, Maryland 21228 USA Candidates should request that their contact notify them by email when the completed signed form has been returned to INFORMS.

®

For further information on the CAP® credential, please visit www.INFORMS.org/certification

Please PRINT the following information.

TO BE COMPLETED BY CERTIFICATION APPLICANT Applicant first name

Middle initial

Last name

Applicant job title

Company/organization name

Applicant email

Phone number

SOFT SKILLS CONFIRMATION STATEMENT As a contact for the above applicant for the Certified Analytics Profession (CAP®) credential, it is my opinion that the applicant possesses and demonstrates an acceptable level of analytical “soft skills” noted below to adequately perform the job of an analytics professional. Partnering with Business Clients: Ability to identify, establish professional relationship, and communicate with analytics clients in order to understand business needs. Framing Problems with Stakeholders: Ability to research and construct problem frames in order to understand the analysis context and scope that will provide timely, useful results. Working in Project Teams: Ability to lead and participate in multidisciplinary analytics project teams. Interviewing Subject Matter Experts: Ability to plan and conduct individual interviews with experts to gain valid information and data needed for analysis. Eliciting Information from Groups: Ability to plan and conduct group elicitation sessions with committees or other working groups to develop and assess alternatives, uncertainties, and value and risk preferences. Communicating Results to Decision Makers: Ability to explain the results and conclusions of the analytics process in both written and oral presentation formats.

TO BE COMPLETED BY CONTACT Contact name (please print) ______________________________________________________________________________ Contact signature _______________________________________________________________________________________ Title/position of contact _________________________________________________________________________________ Company/organization of contact _________________________________________________________________________ Date of signature ___________ Business telephone number _________________________ Email address _____________________________________________________________________________________________ Dates of applicant employment Start ____________ Finish ____________ Please return this signed form to INFORMS as soon as possible in one of the methods noted above. Thank you very much for your assistance in helping confirm the eligibility of an applicant for the Certified Analytics Professional (CAP®) credential.

Certification Examination Special Accommodations Form Certified Analytics Professional (CAP®) Credential Please PRINT the following information.

TO BE COMPLETED BY CERTIFICATION APPLICANT

First name

Middle initial

®

Last name

Email Phone number

For further information on the CAP® credential, please visit www.INFORMS.org/certification

Please describe the disability that significantly impairs your ability to complete the CAP® examination.

Please list the specific testing accommodation requested. Feel free to use additional separate sheets if needed.

Note: You must also provide the INFORMS Certification Department with written documentation from a licensed/certified healthcare provider supporting the need for the accommodation requested. This documentation should include a statement describing your disability, diagnosis of your health condition, and a specific recommendation for the type of accommodation requested. INFORMS will not be able to process any request for an accommodation related to compliance with the Americans with Disabilities Act of 1990 until both this Accommodation Request form and the required healthcare provider documentation have been submitted to INFORMS.

Signature

Date

Code of Ethics for Certified Analytics Professionals Prepared by the INFORMS Certification Task Force

®

Background. The Institute for Operations Research and the Management Sciences (INFORMS) does not have an established code of ethics or guidelines for ethical practice that applies to the general membership. However, Article 1, Paragraph 2.v., of the INFORMS constitution states, “The Institute will strive to promote high professional standards and integrity in all work done in the field.” Applicability. This Code of Ethics applies specifically to those seeking (re)-certification as a Certified Analytics Professional (CAP®), but may be useful to other practitioners who use analytics. Clients, employers, researchers, policymakers, journalists, students and the public should expect analytical practice by CAP® certified individuals to be conducted in accordance with these guidelines. Application of these or any other ethical guidelines generally requires good judgment and common sense. Purpose. This code exists to clarify the ethical requirements that are important; to inform the individual regarding rules and standards; to hold the profession accountable; to aid analytics professionals in making and communicating ethical decisions; to help deter unethical behavior and promote self-regulation; and to list possible violations, sanctions, and enforcement procedures. General. Analytics professionals participate in analysis that aids decision makers in business, industry, academia, government, military, i.e. all facets of society; therefore, it is imperative to establish and project an ethical basis to perform their work responsibly. Furthermore, practitioners are encouraged to exercise "good professional citizenship" in order to improve the public climate for, understanding of, and respect for the use of analytics across its range of applications. In general, analytics professionals are obliged to conduct their professional activities responsibly, with particular attention to the values of consistency, respect for individuals, autonomy for all, integrity, justice, utility, and competence. Responsibilities. This Code recognizes that analytics professionals have obligations to a variety of groups, including: society, employers and clients, colleagues, research subjects, INFORMS, and the profession in general. Responsibilities regarding each of these groups are further described below. Society. All professionals have societal obligations to perform their work in a professional, competent, and ethical manner. Professionals should adhere to all applicable laws, regulations, and international covenants. Employers and Clients. In general, it is the practitioner’s responsibility to assure employers and clients that an analytical approach is suitable to their needs and resources, and include presenting the capabilities and limitations of analytical methods in addressing their problem. Analytics professionals should clearly state their qualifications and relevant experience. It is imperative for analytics professionals to fulfill all commitments to employers and clients, guard any privileged information they provide unless required to disclose, and accept full responsibility for their performance. Where appropriate, professionals should present a client or employer with choices among valid alternative approaches that may vary in scope, cost, or precision. Professionals should apply analytical methods and procedures scientifically, without predetermining the outcome. Professionals should resist any pressure from employers and clients to produce a particular "result," regardless of its validity. Colleagues. Analytics professionals have a responsibility to promote the effective and efficient use of analytical methods by all members of research teams and to respect the ethical obligations of members of other disciplines. When possible, professionals share nonproprietary data and methods with others; participate in peer review, focusing on the assessment of methods not individuals. Professionals respect differing professional opinions while acknowledging the contributions and intellectual property of others. Those professionals involved in teaching or training students or junior analysts have a responsibility to instill in them an appreciation for the practical value of the concepts and methods they are learning. Those in leadership and decision-making roles should use professional qualifications with regard to analytic professionals’ hiring, firing, promotion, work assignments, and other professional matters. Professionals should avoid harassment of or discrimination based on professionally irrelevant bases such as race, color, ethnicity, gender, sexual orientation, national origin, age, religion, nationality, or disability.

Code of Ethics for Certified Analytics Professionals (cont.) Prepared by the INFORMS Certification Task Force

®

Research Subjects. If a project involves research subjects, including census or survey respondents, an analytics professional will know and adhere to the appropriate rules for the protection of those human subjects. A professional will be particularly aware of situations involving vulnerable populations that may be subject to special risks and may not be able to protect their own interests. This responsibility includes protecting the privacy and confidentiality of research subjects and data concerning them. INFORMS and Profession. Analytics professionals will strive for relevance in all analyses. Each study or project should be based on a competent understanding of the subject-matter issues, appropriate analytical methods, and technical criteria to justify both the practical relevance of the study and the data to be used. Professionals should guard against the possibility that a predisposition by investigators or data providers might predetermine the analytic result. Professionals should remain current in constantly changing analytical methodology, as preferred methods from yesterday may be barely acceptable today and totally obsolete tomorrow. Professionals should disclose conflicts of interest, financial and otherwise, and resolve them. Professionals should provide only such expert testimony as they would be willing to have peer reviewed. A professional should maintain personal responsibility for all work bearing his or her name; avoid undertaking work or coauthoring publications for which he or she would not want to acknowledge responsibility. Alleged Misconduct. Certified Analytics Professionals will strive to avoid condoning or appearing to condone careless, incompetent, or unethical practices. Misconduct broadly includes all professional dishonesty, by commission or omission, and, within the realm of professional activities and expression, all harmful disrespect for people, unauthorized or illegal use of their intellectual and physical property, and unjustified detraction from the reputation of others. Professionals will recognize that differences of opinion and honest error do not constitute misconduct; they warrant discussion, but not accusation. Questionable scientific practices may or may not constitute misconduct, depending on their nature and the definition of misconduct used. Professionals will not condone retaliation against or damage to the employability of those who responsibly call attention to possible scientific error or misconduct. References. 1. Saul I. Gass, Ethical guidelines and codes in operations research, Omega 37 (2009), 1044-1050. 2. American Statistical Association, Ethical Guidelines for Statistical Practice, August 7, 1999. 3. U.S. federal regulations regarding human subjects protection are contained in Title 45 of the Code of Federal Regulations, Chapter 46 (45 CFR 46).

Recognized Analytics Continuing Education Provider Form To be recognized by INFORMS for training/education in the analytics profession, complete the application below and return it with appropriate fee(s) and items listed below. The completed form must be returned to INFORMS by the contact in one of the following methods: 1. The preferred method is to scan the completed form and send it as PDF to the INFORMS Certification Manager at [email protected]. 2. Return the completed signed form to INFORMS by postal mail. Please form and attachments to: Certification Manager INFORMS 5521 Research Park Drive, Suite 200 Catonsville, Maryland 21228 USA 3. Or you may fax form to Louise Wehrle at 443.757.3515. As part of the application, provider must provide examples of: Certificate granted to student Brochure/website copy/advertising of courses Registration requirements Refund policy Instructor/author qualifications Facilities management policy Org chart of organization

TM

For further information on the CAP™ credential, please visit www.INFORMS.org/certification

Please PRINT the following information.

Name of Organization Address of Organization Point of Contact Name

Point of Contact Phone number Point of Contact Email Number of Courses

Years of Operation

Names of Courses

Description of Courses Instructor(s) Name & Qualifications How did you hear about listing/CAP™/INFORMS? member

colleague

website

contact from INFORMS

Fees: $300 non-refundable application fee per organization $200 first-year approval fee per organization

other_______________________________________

NOTE: if you are an accredited public institution, INFORMS will list your courses free of charge. We are a public institution and hold accreditation from _______________________________________

Payment method: _____ Check Enclosed (Make payable to INFORMS and must be drawn on U.S. bank in U.S. dollars. ) _____ MasterCard _____ Visa _____ American Express _____ Discover

_______________________________________________________________________________________ Card # (15 digits)

Expiration date

_______________________________________________________________________________________ Name on Card

Signature

®

INFORMS 5521 Research Park Drive, Suite 200 Catonsville, Maryland 21228