Research Design

Overview of Research Designs

The Marketing Research Process Step 1: Defining the Problem Step 2: Developing an Approach to the Problem Step 3: Formulating a Research Design Step 4: Doing Field Work or Collecting Data Step 5: Preparing and Analyzing Data

Step 6: Preparing and Presenting the Report

Research Design • A master plan that specifies the methods and procedures for collecting and analyzing needed information.

Tasks Involved In a Research Design Define the Information Needed Design the Exploratory, Descriptive, and/or Causal Phases of the Research Specify the Measurement and Scaling Procedures Construct a Questionnaire Specify the Sampling Process and the Sample Size Develop a Plan of Data Analysis

Today’s Topic

 A Classification of Market Research Designs Research Design Exploratory Research Secondary Data

Experience Surveys

Conclusive Research Pilot Studies

Case Studies

See next slide

 A Classification of Market Research Designs Research Design Exploratory Research

Cross-sectional Study Longitudinal Study

Conclusive Research

Descriptive Design

Causal Design Experiment

Secondary Data Study

Survey

Observation

Types of Research Designs Exploratory research to gain ideas and insights   Newspaper facing decreasing sales to generate possible explanation. Descriptive research to obtain summary measures to address research questions (research objectives are clearly defined).  Trends in lifestyle with respect to age, sex, etc. Causal research for cause-and-effect connection between managerial decisions and market outcome.  –  How people react to a newspaper’s topic selection and space allocation.

Exploratory Research • Usually conducted during the initial stage of the research process • Purposes  –  To narrow the scope of the research topic, and  –  To transform ambiguous problems into well-defined ones

Exploratory Research Techniques •

Secondary Data Analysis

 –  Secondary data are data previously collected & assembled for  some project other than the one at hand •

Pilot Studies

 –  A collective term for any small-scale exploratory research technique that uses sampling but does not apply rigorous standards  –  Includes • Focus Group Interviews  –  Unstructured, free-flowing interview with a small group of people

• Projective Techniques  –  Indirect means of questioning that enables a respondent to project beliefs and feelings onto a third party or an inanimate object  –  Word association tests, sentence completion tests, role playing

Exploratory Research Techniques • Case Studies  –  Intensively investigate one or a few situations similar  to the problem situation

• Experience Surveys  –  Individuals who are knowledge about a particular  research problem are questioned

Conclusive Research • Provide specific information that aids the decision maker in evaluating alternative courses of action • Sound statistical methods & formal research methodologies are used to increase the reliability of the information • Data sought tends to be specific & decisive • Also more structured & formal than exploratory data

Types of Conclusive Research • Descriptive Research  –  Describes attitudes, perceptions, characteristics, activities and situations.  –  Examines who, what, when, where, why, & how questions

• Causal Research  –  Provides evidence that a cause-and-effect relationship exists or does not exist.  –  Premise is that something (and independent variable) directly influences the behavior of something else (the dependent variable).

Common Characteristics Characteristics of  Descriptive Studies • Build on previous information • Show relationships between variables • Representative samples required • Structured research plans • Require substantial resources • Conclusive findings

 Major Types of Descriptive Studies Descriptive Studies

Sales Studies

• Market Potential • Market Share • Sales Analysis

Consumer Perception And Behavior Studies

Market Characteristic Studies

• Image

•Distribution

• Product Usage

•Competitive Analysis

• Advertising • Pricing

Cross Sectional vs. Longitudinal Designs Cross Sectiona l Design

Longitudina l Design

 Time

Sample Surveye d at T1

Sample Surveyed at T1

 T1

Same Sample also Surveyed at T2  T2

Cross-Sectional vs. Longitudinal Designs Cross-Sectional

Longitudinal

Detecting change

Worse

Better  

Amount of data collected

Worse

Better  

Accuracy

Worse

Better

Representativeness

B et t e r

Worse

Response bias

B et t e r

Worse

 

 Some Alternative Research Designs

(a)

Exploratory Research • Secondary Data  Analysis • Focus Groups

(b)

Conclusive Research • Descriptive/Causal 

(c)

Conclusive Research • Descriptive/Causal 

Conclusive Research • Descriptive/Causal 

Exploratory Research • Secondary Data  Analysis • Focus Groups

Common Characteristics of Causal Studies • Logical Time Sequence  –  For causality to exist, the cause must either precede or occur simultaneously with the effect

• Concomitant Variation  –  Extent to which the cause and effect vary together as hypothesized

• Control for Other Possible Causal Factors

How Descriptive & Causal Designs Differ  • Relationship between the variables  –  Descriptive designs determine degree of association  –  Causal designs infer whether one or more variables influence another variable

• Degree of environmental control  –  Descriptive designs enjoy lesser degrees of control

• Order of the variables  –  In descriptive designs, variables are not logically ordered

Comparison of Research Designs Exploratory

Descriptive

Causal

Purpose

ID problems, ga gain insights Describe th things

Determine ca cause-andeffect relationships

Assumed background knowledge

Minimal

Considerable

Considerable

Degree of structure Very little

High

High

Flexibility

High

Some

Little

Sample

Non-representative

Representative

Representative

Research environment

Relaxed

Formal

Highly controlled

Cost

Low

Medium

High

Findings

Preliminary

Conclusive

Conclusive

Which is the “Best” Research Design & Method? • “You cannot put the same shoe on every foot.”  –  Publilius Syrus • It depends on the  –  problem of interest,  –  level of information needed,  –  resources,  –  researcher’s experience, etc.

What is Descriptive Research? • Can involve collecting quantitative information • Can describe categories of qualitative information such as patterns of interaction when using technology in the classroom. • Does not fit neatly into either category

What is Descriptive Research? • Involves gathering data that describe events and then organizes, tabulates, depicts, and describes the data. • Uses description as a tool to organize data into patterns that emerge during analysis. • Often uses visual aids such as graphs and charts to aid the reader 

Descriptive Research takes a “what is” approach • What is the best way to provide access to computer  equipment in schools? • Do teachers hold favorable attitudes toward using computers in schools? • What have been the reactions of school administrators to technological innovations in teaching?

Descriptive Research • Refers to the nature of the research question • The design of the research • The way that data will be analyzed for the topic that will be researched The type of research question will determine whether descriptive research approach is appropriate to use.

Descriptive Research Advantages • Educational research and experiences may contain many variables that cannot  be realistically controlled. • Educational research may require observations of life experiences

• Data collection may be spread over a large number of people over  a large geographic area

Data Collection Methods Surveys

Interviews

Observations

Portfolios

Descriptive Research 1. Surveys • May be used to reveal summary statistics by showing responses to all possible questionnaire items. • Often provide leads in identifying needed changes

• May be used to explore relationships  between 2 or more variables.

Descriptive Research Critical Components Sound Well-designed methodology data collection instruments

Descriptive Research Survey Forms • Written questionnaires • Personal interviews • Telephone interviews

Factors to be considered • Sampling • Type of population • Question Form • Question Content • Response rates • Costs • Available facilities • Length of data collection • Computer assisted techniques for data collection

Descriptive Research 2. Survey Form - Interviews • More time efficient • Allow the researcher to establish a rapport with the respondent • Allow the acquisition of more in-depth information • Allow for interviewer observation • Allow the interviewer to obtain visual cues • May be personal or telephone interviews

Descriptive Research Survey Form – Personal Interviews Disadvantages

• Require more staff time • Require more travel time

Descriptive Research Survey Form – Telephone Interview

Advantages

Disadvantages

 Less expensive

 Limited telephone

 Less time-consuming

access  Lack of interviewer’s

ability to observe the respondent and obtain visual cues

Descriptive Research Survey Form – Mailed Questionnaires

Advantages

Disadvantages

 Ability to reach large

 Lower response rate

number of people across a wide geographic area  Ease and low cost of  distribution  Minimal amount of staff  required  Allows respondents to respond in their time frame

 Need to design a

survey instrument with a simple format

Descriptive Research Survey Form – Mailed Questionnaires

• A letter of transmittal should accompany mailed questionnaires. • Should state purpose and importance of research r esearch • Should state importance of responding • Should give a time frame to respond • Should include a confidentiality statement • Should include an offer to share results • Should include a thank-you note to the respondent

Descriptive Research Characteristics of a Good Survey • Good questioning techniques • Use complete sentences • Offer a limited set of answers • Interesting • Worded so that questions mean the same to all • Provide definitions for confusing terms • Uses the “I don’t know” answer very v ery carefully

Descriptive Research 3. Observational Research Methods

1. Natura Naturall lly y occu occurr rrin ing g beha behavi viors ors obser observed ved in natural contexts 2. Conte Contexts xts that that are are contr contriv ived ed to be reali realisti sticc

Descriptive Research Observational Research Methods • Require direct observation of behavior  • Data gathered without intermediary instruments • Can yield a wealth of invaluable information • Can be a complicated process

Descriptive Research Observational Research Methods • Can be employed productively to support many purposes in educational technology • Can be used to determine how people interact with technology in various stages of  design and implementation

Descriptive Research Observation Research Methods • How do learners interact with a specific s pecific program? • How do learners interact with a new hardware system? Observation makes it possible to explore the implementation of a particular technological innovation and assess the instructional instr uctional outcomes.

Descriptive Research Observational Research Methods • 2 Forms of Observational Research  –  Structured  –  Unstructured

Descriptive Research Observational Research Methods Structured Observations Rigid and controlled Predetermined

methods

Unstructured Observations  Used to determine

unselective, detailed, continuous description of   behavior.  Detects unintended effects  More time consuming  because of time and labor  required to collect and analyze sets of extensive observations

Descriptive Research Observational Research Methods Develop observation form May be paper and pencil or electronic May use a rating scale to evaluate behavior  A 3-point rating scale is sufficient

Descriptive Research Observational Research Methods  Newer Mediated Observation Techniques • Audio • Videotape • Computers provide on-line monitoring (process of  capturing characteristics of the human-computer  interaction automatically)  –  Keystroke records  –  Audit trails  –  Logging data

Descriptive Research 4. Portfolios • Provide a descriptive measure of student work based on actual performance • Consist of learner-created products that reflect the processes of learning and development over time

Impact and Future of  Descriptive Research • Although descriptive studies are important, most educational studies involve questions about cause.  –  What causes underachievement?  –  Will multimedia cause students to be more motivated or lead to high achievement levels?

Impact and Future of  Descriptive Research •  Not always easy to isolate variable that will explain those causes, so descriptive research can  play an important role in providing information from another perspective. • By gathering descriptions of “what is” and comparing them to “what we would like” educators can see the area that needs to be addressed.

Impact and Future of Descriptive Research • Descriptive Research methods have gained acceptance •  Number of descriptive studies published in research journals has increased • Descriptive research leads to prescriptions that instructional designers and educators can heed as they consider future direction

Questions Addressed by Causal   Research * Marketing director director of of local beer company, company, “Will replacing TV commercial A with commercial B lead to increase in consumer preference on our brand?” * Chairman of a charity organization. organization. “Will it be worthwhile worthwhile to mail to previous donors an attractive and expensive brochure to solicit higher contributions this year?” * The sales manager manager of a local life insurance company. “Will training in the use of computers for client management increase agents’ sales?” * Marketing VP of fashion chain, “Can we improve profitability of our fashion clothing line by increasing its price by 10%?”

Causality: Cause-and-Effect  Change-in-X causes change-in-Y  Evidence of Causality • Concomitant variation:  –  If X changes, then Y also changes.  –  If X does not change, then Y does not

change. • Time order: cause (X) occurs before effect (Y). • Elimination of alternative explanation:  –  Could changes in Y be caused by

changes in Z?

Concepts and Terminology Causality Manipulation (experimentation) Independent variable (X, manipulated, computer skill) Dependent variable (Y, reflect the impact of  independent variable, sales) Treatment Group: subject exposed to manipulation (salespeople with training) Control Group: part of sample that sees no change in independent variable (no computer  training)

Validity of Experiment  Internal validity: elimination of other possible causes .  – Sources: Sources: external factors (Asian crisis), maturation of subjects (agents more experienced).  – Lab Lab experiment > Field experiment.

External validity: generalizability of results.  – Artificial Artificial situation; inappropriate sample; inappropriate time.  – Field Field experiment > Lab experiment:

Types of Experiments Laboratory experiment Research investigation in which investigator creates a situation with exact conditions so as to control some, and manipulate other, variables Scientific investigation in which an investigator manipulates and controls one or more independent variables and observes the dependent variable for variation concomitant to the manipulation of the independent variables

Field experiment Research study in a realistic situation in which one or more independent variables are manipulated by the experimenter under as carefully controlled conditions as the situation will permit

Experiments are the STRONGEST Design Logic • The “best” Research • Design to determine CAUSAL RELATIONS

Use Experimental designs whenever possible • Types of  EXPERIMENTAL DESIGNS: • Lab • Field Experiment • Quasi-Experimental