Keeping Up With Quants - HBR

Keeping Keeping Up with t he Quants: Quants:  Anal  A nal y t i c s f or B us i nes s L eader s Featuring analytics expert Tom Davenport, author of Competing on Analytics, Analytics , Analy  An alyti tics cs at Wor k , and the just-released Keeping Keeping Up wi th the Quants

 AUGUST 14, 2013

Sponsored by

Questions? To ask a qu estion … click on the “ question question icon” icon” in the lower-right corner of your screen.

OCTOBER 17, 2012

1

Follow the Conversation on Twitter  Use #HBRwebinar @HBRExchange

Keeping Up with t he Quants:  Analy tics fo r B usiness Leaders Today’s Speaker  Tom Davenpor t President’s Distinguished Professor, Management & IT, Babson College  Au th or, Keeping Up with the Quants

 AUGUST 14, 2013

2

Keeping Up with the Quants Harvard Business Review Webinar 

Tom Davenport Babson College, IIA, MIT, Deloitte  August 14, 2013

Problem-Solvin g wit h Quantitativ e Analysis • Runs a call center for Cigna customers • Nursing background, with an MBA—not a quant, but appreciates quantitative thinking

Jennifer Joy—Cigna

• Key problem—do telephone interventions make customers more likely to avoid re-hospitalization? • Got lots of reports—figures up, figures down, no meaning • Worked with quant group to do “Smart Innovation”— matched-pair tests • Found that calls only worked for some diseases • Reduced length of calls for other diseases • Redeploying workers to other problems • Now testing no calls at all

6

3

Problem-Solvin g wit h Quantitativ e Analysis • Ran a small (400 person) unit for AIG specializing in credit default swaps • Dramatically accelerated the amount of CDSs sold

Joe Cassano—AIG Financial Products

• Cassano’s predecessor, Tom Savage, was a mathematician who understood models behind CDSs, was conservative in selling them, and debated models with staff  • “Cassano knew a lot less math and had much less interest in debate”—Michael Lewis • “Patient Zero of the global economic meltdown”—Matt Taibbi • Lost approximately $85 billion (to be repaid by US taxpayers)

7

The Planets Are Aligned fo r Analytic s

• • • •

Powerful information technology Data critical mass Skills sufficiency Business need for process optimization, differentiated products and services, better decisions

8

4

 Analyti cal Decisions Are Not a New Idea!

George W. Bush elected

Level of  Accuracy/ Precision

Freud-Decisions are subconscious Descartes-Decisions by reason Decisions by entrails analysis

Prehistory

Bacon-Induction in science

McNamara in Viet Nam McNamara In WW2

Blink

Barack Obama elected

published Global financial crisis

Gauss-Normal distribution

Time

The Present

9

The Modern Histor y of An alytics • WW2 military analysts focus on supply chain and targeting optimization • UPS starts analytics group in 1954 • Michael Scott-Morton, Peter Keen, Tony Gorry and others at MIT and elsewhere describe “decision support systems” in the 70s • Jack Rockart at MIT advances “executive support systems” and “critical success factors” in the 80s • Howard Dresner at Gartner defines “business intelligence” in 1988 • Companies begin to “compete on analytics” after 2000 •  Analytics begins to blend with “decision management” and “big data” in the 2010s 10

5

Decidin g on Analytics versus “ The Gut”

Forty percent said a recent important business decision was based not on data and facts, but on “gut instinct” — 2009 Accenture survey of U.S. and UK managers • Statistical predictions consistently outperform “gut based” predictions • Extensive evidence that having experts is good, but experts using analytics is much better  • Expert intuition is best only when there is little time, limited data, and few variables

11

What are analytics?

Predictive and prescriptive analytics   e   u    l   a   v    d   n   a   e   c   n   e   g    i    l    l   e    t   n    i    f   o   e   e   r   g   e    D

Optimization

What is the best that can happen?

Predictive modeling

What will happen next?

Randomized t esting

What if we try this?

Statistical models

What are the causes and effects?

 Aler ts

What actions are needed now?

Query/drill dow n

Where exactly is the problem?

Scorecards

What information really matters?

Standard reports

What happened?

Descriptive analytics

12

6

Quantit ative Analysis —Three Stages, Six Steps

Fr am ing t he pr ob lem

So lvin g the pr obl em

1. Problem recognition

3. Modeling 4. Data collection

2. Review of previous findings

Communicating/ acting on results

5. Data analysis

6. Results presentation and action

Framing the Problem 1. Problem recogniti on • The least structu red—but most impo rtant—step • Usually starts with a broad hypothesis • “We’re spending too much on marketing” • Must eventually become much more specific and testable • What type of story do you want to tell?

2. Review of pr evious fi nding s • Has anyone solved a pro blem like this b efore? • Internet search • Talk to decision-makers and analysts in your company • Go to meetups of analytical people • Might suggest data, method, specific algorithm • Poisson distribution for bomb clusters in WW2 Lond on

7

Types of A nalytical Stories • CSI: Analyti cs • Solve a tactical problem with analytics • Expedia • Eureka! • Solve a long-term strategic problem with analytics • Zillow • Mad sci entist • Do a rigorous experiment to establish causality • Sears, Food Lion, Red Lobster  • Survey • Observe, codify and analyze something that has already happened • Prediction • Use data about the past to predict the future • What happened? • Straightforward reporting 15

Solving t he Problem 3. Modeling/variable selection •  A pu rp os ely si mp li fi ed repr esen tat io n of reali ty • Variables can be objective or subjective • “Should I get a pet?” • We’re still in intui tive hypothesis territory

4. Data collection • How much precision measurement in your variables? • Binary, categorical, ordinal, numerical • What type of data would be mos t useful? • Primary or secondary • Structured or unstructured

5. Data analysis • Method driven by various factors • Type of story, descriptive vs. inferential analysis, level of measurement, number of variables • Many data analyses involve regression

8

Communicating and Acting on Results 6. Results p resentation and action • Not normally focused on by academics, but beginnin g to change •  A pr ob lem th ro ug ho ut hi st or y

• Florence Nightingale vs. Gregor Mendel • Need to tell a story w ith narrative or pictures • Modern analytical communications in clude: • Interactive graphics • Automated narrative generation • Related videos • Games, simulations, walk-throughs

 A Great Analytic al Comm unic ator—Dr. John Gottm an Measured and modeled key behaviors between 700 couples in conversation Humor: +4  Agreeing: +4 Joy: +4  Affection: +4 Interest: +2  Anger: –1 Domineering: –1

Sadness: –1 Whining: –1 Belligerence: –2 Defensiveness: –2 Stonewalling: –2 Disgust: –3 Contempt: –4

Need 5:1 ration of positive to negative behaviors Model was very effective at predicting relationship failure Published the model with Dr. James Murray Expanded study to 10,000 couples in randomized clinical trial Wrote scholarly and popular books, created videos With Julie Gottman, developed workshop for couples and therapists 18

9

Quantitativ e Analysis at 1. Problem recognition: A general feeling from Transitions’ parent companies (Essilor and PPG) that the company was spending too much on marketing, with no evidence of its success or ROI

2. Review of previous findings: None at Transitions. The CMO knew that this sort of “marketing mix” analysis was possible, so he contacted a consulting firm that specialized in it. 3. Modeling/Variable selection: Typical approach with marketing mix models is to find the weekly or monthly advertising, promotion, and pricing levels that maximize revenue or profit margin. They involve variables of marketing response, marketing costs, and product margins, and control for factors like macroeconomic performance and weather.

Quantitativ e Analysis at 4. Data collection: Most difficult step for Transitions. They went through a multi-year effort to collect customer data from channel partners. The data came in in 30 different formats, and was eventually stored in a customer data warehouse.

5. Data analysis: Consultant did the data analysis, which were based on linear and non-linear programming (optimization) models.

6. Results presentation and action: Transitions hired new employees to interpret the models and lead discussion and action on them. The models led to substantially higher levels of marketing spending and TV advertising.

10

What Non-Quants (Deciders) Should Expect of Quants

• • • •

Learn the business process and problem Communicate results in business terms Seek the truth with no predefined agenda Help to frame and communicate the problem, not  just solve it • Don’t wait to be asked

21

What Quants Should Expect of Non-Quants (Deciders)

• Form a relationship with your quants • Give them access to the business process and problem •  Ask lots of questions, particularly about assumptions • Focus primarily on framing and communicating the problem, not solving it •  Ask for help with the entire decision process • Establish a culture of honesty and self-examination, and admit your decision errors

22

11

Improving Your Analytical Decisions

• Pick a problem that’s important but manageable, and for which data exists • Hire or consult with a quant • Go through the six steps • Work on relationships, not just math • Measure the decision outcome before and after  • Repeat and expand!

23

Questions? To ask a qu estion … click on the “ question icon” in the lower-right corner of your screen.

OCTOBER 17, 2012

12

 AUGUST 14, 2013

Thank you for joining us ! This webinar was made possible by th e generous support of Hitachi. Learn more at www. hitachi.us Sponsored by

13