DESCRIPTIVE

RESEARCH
1
MARKETING RESEARCH 103: LECTURE 3
2
OUTLINE
Systematic Observation
Sampling
Standardized measurement
Survey
Sampling
Questionnaire Design (if possible)
Measurement and Scaling (next class)

Clearly-defined research question with the
objective of being descriptive
a structured research design guides the
sampling and ensures control in
standardizing what is being observed and
how it is recorded



3
I. SYSTEMATIC OBSERVATION
Time sampling
Situation sampling
Different locations (e.g., different places beer is
consumed: bar, student center, party etc.)

Sampling determines generalizability
4
SAMPLING TECHNIQUE
STANDARDIZING OBSERVATION SHEET
Date: 22 Aug 2007
Time Gender Type of Packaging P1 P2 P3 P4 P5 P6 P7 P8
1:00 PM
1:05 PM
1:10 PM
1:15 PM

Key
variables 1 2 3 4 5
P1
approximate age
group
Below 12 12 to 18 19 to 25 26 to 35 Above 35
P2
attire uniform business casual sporty Dressy(for
mal)
P3
bag none Small medium large Extra large
P4 no. of companions none 1 to 2 3 to 5 6 to 10 Above 10
P5 quantity 1 2 3 4 Above 4
P6 Other purchases none snacks meal Non food Others
P7
Size of drink Extra small small medium large Extra large
P8
Type of drink Soft/fizzy fruit juice/
fruit-based
Tea/coffee Isotonic
drinks
Milk-based
Observer: George Location: Borders
5
Nominal scale (checklist)
e.g., Brand of shoes, hair color, cellular phone or not,
with accompany or not.

Summarizing nominal data in terms of frequency,
proportion, or percentage of instances in each of
the several categories.
6
MEASUREMENT SCALE
Physical actions
Verbal behavior
Expressive behavior
Spatial relations and locations
Temporal patterns
Physical objects
Verbal and pictorial records
7
WHAT CAN BE OBSERVED?
Cannot observe motives, attitudes or intentions
Inter-observer reliability
Can sometimes be time consuming
Ethical concern: violation of privacy?
8
LIMITATIONS OF OBSERVATIONAL
METHODS
II. SURVEY
Rationale: To enable quantification of opinions, preferences, etc.

Format: Open-ended or scaled questions, scenarios, ongoing
versus one-shot, longitudinal vs. cross-sectional

Tools: mail, telephone, direct intercept, email, internet

Upsides/Downsides: (methodological and implementation)?
10
2.1 SAMPLING
A. Sampling jargons and design process
B. Sampling Methods
C. Sampling Size
D. Sampling errors vs. Non-sampling errors
11
A. SAMPLING JARGON
Population
If the goal is to determine the average number of job
interviews after graduation for the students who have
taken Marketing Research Class, then the entire body of
students who have taken 103 would be the population
Sampling Frame: A list of students from which a sample is
to be chosen (e.g., registration record)
Sampling: the selection of a fraction of the population
Census: all members of the population are measured

Sampling Frame
Element
One student
Sample
100 students
Registrars list of currently enrolled students
Population
All students on SMU campus
SAMPLING DESIGN PROCESS
Define the Population
Determine the Sampling Frame
Select Sampling Technique(s)
Determine the Sample Size
Execute the Sampling Process
14
B. SAMPLING METHODS
Probability sampling
Simple random
sampling
Proportionate
Disproportionate
Cluster
Non-probability
sampling
Convenience sampling
Quota sampling
Judgment sampling
Snowball sampling

SIMPLE RANDOM SAMPLING
Each element in the population has a known and equal
probability of selection, or
Each possible sample of a given size (n) has a known
and equal probability of being the sample actually
selected.
Randomly pick n (e.g., 5) units out of N is statistically
equivalent to enumerating all possible samples of size n and
picking one of them at random
16
SIMPLE RANDOM SAMPLING (SRS)
Procedure
Number everyone in the sampling frame
Pick n elements randomly
Generated by computer or random number table

17
RANDOM NUMBERS GENERATOR IN SPSS
18
ADVANCED PROBABILITY SAMPLING METHODS
Why do I need anything besides SRS?
Suppose our population of interest is all
marketing students, lets say consisting of 700
undergraduates (1/5 freshman, 2/5 sophomore,
1/5 juniors and 1/5 seniors).
If we take a SRS of 140 students then we may
get a sample which looks like this: Seniors (28),
Juniors (28), Sophomores (56) and Freshman (28)
19
PROPORTIONATE STRATIFIED
RANDOM SAMPLING
The population is divided into strata on the basis of an
appropriate variable (related to the outcome of
interest)
Units within each stratum are alike and units across
strata are different
The sample size from each stratum is proportional to
the size of the stratum
E.g., N=6000, 2 strata with sizes 2000 and 4000.
In a sample of 120, we sample 40 and 80 respectively from the
2 strata
20
OTHER ADVANCED METHODS
Disproportionate stratified random sampling
May want to over-sample small stratum or one that is
highly variable
Cluster Sampling
Population is naturally divided into clusters and clusters
are selected at random.
All or some of the units within selected clusters are
measured (e.g., selected blocks or counties)
21
NON-PROBABILISTIC SAMPLING METHODS
Convenience sampling
At the convenience of the experimenter (at the corner, at the
supermarket)
Quota sampling
Population is divided into segments on the basis of certain control
variable
A quota of units to be selected from each segment is determined by the
experimenter
Sampling is done in each segment until the quota is filled
Better than convenience sampling in that you are at least guaranteed
balance in the population with respect to the control variable


22
NON-PROBABILISTIC SAMPLING METHODS
Judgment sampling
Snowball sampling
CLASSIFICATION OF SAMPLING TECHNIQUES
Sampling Techniques
Nonprobability
Sampling Techniques
Probability
Sampling Techniques
Convenience
Sampling
Judgmental
Sampling
Quota
Sampling
Snowball
Sampling
Proportionate
Stratified Sampling
Cluster
Sampling
Simple Random
Sampling
Disproportionate
Stratified Sampling
24
C. DETERMINE SAMPLE SIZE
_
_
Common notation:
95% CONFIDENT
X
L
_
X
U
_
X
_
0.475 0.475
Estimation: Population vs. Sample
CONFIDENCE INTERVAL
Definitions:
Confidence interval: The confidence interval is the range
into which the true population parameter will fall, assuming a
given level of confidence.
Confidence level: The confidence level is the probability that
a confidence interval will include the population parameter.
Precision level: When estimating a population parameter by
using a sample statistic, the precision level is the desired width
of the estimating interval. This is the maximum permissible
difference between the sample statistic and the population
parameter.
27
CONFIDENCE INTERVAL APPROACH TO
ESTIMATE SAMPLE SIZE
1. Need to know how to construct Confidence
Interval (CI)
We will see how it depends on n (your sample size)
using SRS
2. Derive the sample size (n) that is necessary to
construct a confidence interval with a desired
width (H)
Need to know the desired confidence level (?%) and
desired width (H)
E.g. want to be 95% confident that the market share
is +/- 3% around the true value (+/-3% is the width)

28
CONFIDENCE INTERVAL FOR UNKNOWN
PARAMETER O
CI = (Best Guess) +/- (Certain #) SE(Best Guess)
What makes a given statistic
T the best guess? And how
do I compute the SE of it?
The certain # depends on
how confident you want
to be
For the two examples we will
talk about today, i.e. unknown
u is either a mean or an unknown
proportion p, SE = n / o
This is precision level H. Notice, for a given level of confidence
and a given value of o (SD in the population), we can
solve for the value of n that gives you your desired H.
29
Best Guess for Population Mean
x
i
th
i
In general, let be the measurement on the person/household/firm etc.








Then,

m =
1
N
Xi
i=1
N

X =
1
n
X
i
i=1
n

Since we typically do not know we have to estimate it

(best guess for population mean would be sample mean:
30
SE OF BEST GUESS
SE of X =




.


In the CI calculations, simply replace
2
o
with
s
2
.

s
2
=
1
n - 1
X
i
- X
( )
2
i=1
n

n / o
Since population variance is often unknown,
an unbiased estimate of the population variance is the sample variance defined
as

31
Example 1: Confidence Interval for
We can construct a 100(1- q)% confidence interval for population mean
by computing

where
= the sample mean
= the sample standard deviation
n = the sample size
= 1.645, 1.96, 2.575 for a 90%, 95% and 99% confidence
interval
x
s
q z
What we will talk about later is the inverse
problem. What n is necessary so that z
q
*s/ =H? n

X Z
q
s
n
32
APPLIED EXAMPLE 1
We are interested in the mean starting income of all SMU graduates so
we go out and obtain a SRS of 100 SMU graduates. The mean starting
income in our sample is $115,000 with standard deviation equal to
$20,000. Construct a 95% confidence interval (CI) of the true
population mean starting income of all SMU graduates.
Answer:
)
100
000 , 20
96 . 1 000 , 115 ,
100
000 , 20
96 . 1 000 , 115 ( +
= (111080 , 118920)
X
s
q z
n
33
EXAMPLE 2: CONFIDENCE INTERVAL
FOR PROPORTION P
A proportion is a mean where each observation is a 0 or 1, so we can
compute a CI for a population
We can construct a 100(1- q)% CI for population mean p by computing

where
= the sample proportion
n = the sample size
= 1.645, 1.96, 2.575 for a 90%, 95% and 99% confidence
interval
p
q z


p Z
q

p (1-

p )
n
34
APPLIED EXAMPLE 2
We are interested in the proportion of students who have
tried a new soup. We decide to go out and obtain an SRS of
400 students and ask them if they have tried the new soup.
Suppose 20% of the 400 students say they have tried it.
Construct a 90% confidence interval (CI) of the true
population market share.
Answer:
35
THE SIZE OF THE SAMPLE (MEAN)
Question: What sample size do I need so that if I construct
a 100(1-q)% confidence interval it will be an interval with
width H ?

Compute Size of the Sample
Determine q, the desired CI you want, e.g., = 1.645 is a 90% CI
Determine the interval width you want, e.g.,
Compute an estimate of the standard deviation of the population of interest,
denoted by s.
Compute the necessary sample size
0.10
Z
5000 $

n =
Z
q
2
s
2
H
2
36
Where do you get s from?
Historical data

Worst case scenario (largest possible s).

Pre-test
37
EXAMPLE
Suppose you want a 95% CI for mean graduate
starting salaries such that the interval will have
width $5000. Using the results of the previous
example where s = $20,000, how many graduates
do I need to obtain in my SRS to obtain this
desired width?
Answer:
2
2 2
5000
000 , 20 96 . 1
= n
~ 61.5
Note, you need to do this for
every question in your survey
and then take the maximum
38
THE SIZE OF THE SAMPLE (PROPORTION)
Compute Size of the Sample
Determine q the desired CI you want, e.g., = 1.645
is a 90% CI
Determine the interval width you want H, e.g.,
Compute an estimate of population proportion
Compute the necessary sample size:
0.10
Z
% 3
p

n =
Z
q
2

p (1-

p )
H
2
39
An approximate n for proportions (assuming 95% confidence)
in a worst-case scenario is :

n = 1/H
2

WHERE DO YOU GET AN ESTIMATE OF P?



















The largest sampling error occurs when p =


0.5
1
SE
P
0
p(1-p)
40
EXERCISE
Estimate the sample size for the following scenario:
95% CI
Interval width (H) =
Assume 50% of people like the new ad copy
% 3
41
D. SAMPLING VS. NON-SAMPLING ERRORS
Total Survey Error
Non-Sampling
Error
Non-Response
Error
Response
Error
Random
Sampling Error
SOURCES OF ERRORS IN INFORMATION
FROM RESPONDENTS
POPULATION RESPONDENT INTERVIEWER
Inaccuracy in
response
Ambiguity
of answer
Ambiguity of
question
Interviewer
error
sample
Question

Answer
Sampling
error
Non-response
due to refusals
or not-at-home
The method you select usually leads
to trade-offs in these areas.

Mail, Phone, Internet, Personal Interview.
RANDOM SAMPLING ERROR
Sample selected is not a perfect representation of
test population

Can be controlled by:
Random sampling procedure
Quota sampling
Increase sample size

44
NON-RESPONSE ERRORS
A non-response error occurs when units selected as
part of the sampling procedure do not respond in
whole (called unit non-response) or in part (called
item non-response)
If non-respondents are not different from those that did
respond, there is NOT a non-response error
It only happens when the existence of non-respondents
causes respondents to be poor representatives of total
sample

45
RESPONSE ERRORS
A Response or data error is any systematic bias that
occurs during data collection, analysis or
interpretation
Respondent error
Interviewer bias
Recording errors
Poorly designed questionnaires
DILEMMA: LARGE VS. SMALL
Large Sample Size: Reduce random sampling error

Small Sample Size: Better interviewer controls,
higher response rate and higher accuracy




ATT STUDY
Purpose: compare a list of research procedures in terms of the
magnitude of various sampling and non sampling errors.
4 methods of survey administration (same questionnaire):
mail
telephone interviews
personal interviews
questionnaire drop-off
2 methods of prior notification:
mail and telephone
2 methods of follow up
mail and telephone


SPECIAL CHARACTERISTICS OF THIS STUDY
Survey questions:
telephone usage
attitudes in telephone usage
organizational characteristics
Telephone usage for each company were obtained from ATT for
both respondents and non respondents (allowing for assessing non
responses and response error)
Comparisons focused on the following three questions:

Company's average monthly telephone bill over the last 3 months.
Quantity of companys telephone lines.
Quantity of companys telephone sets (stations).

FINDING 1:
Random
Sampling Error
~5%
Non-Sampling
Error
~95%
RANDOM SAMPLING ERROR VS. NON-
SAMPLING ERROR
What does the 95% mean?
Given random sampling in place, the need to get
large representative samples is likely to have been
exaggerated

What should we focus on instead?
Obtain smaller samples so as to better spend
resources on:
Interviewer control
Performing pre-tests of questionnaires
Trying to achieve greater reliability of survey
responses
FINDING 2:
>
>
>
Higher values
suggest that
respondents are
larger companies
than non-
respondents

Respondents
more economically
upscale than non-
respondents
DIRECTION OF RESPONSE & NON-
RESPONSE BIASES
CONCLUSIONS
Given random sampling, focus on minimize non-sampling error:

Response Error
Control interviewer-respondent interface and/or questionnaire
format
Enough time

Non-response error
Ensure respondents are representative of the selected sample
Check whether respondents represent population

54
REVIEW:
Understand SRS and why you might depart from SRS
Understand the confidence approach to estimate sample
size
Understand various errors in sampling



The proof of the pudding is in the eating. By a small
sample we may judge of the whole piece.
Miguel de Cervantes Saveedra

2.2 QUESTIONNAIRE DESIGN
55
Specify the Information Needed
Design the Question to Overcome the Respondents Inability and
Unwillingness to Answer
Determine the Content of Individual Questions
Decide the Question Structure
Determine the Question Wording
Arrange the Questions in Proper Order
Reproduce the Questionnaire
Specify the Survey Administration Method
Identify the Form and Layout
Eliminate Bugs by Pre-testing
Process
STEP 0: MAKE SURE THE DATA YOU GET
WILL BE USEFUL
Population has been defined correctly
Sample is representative of the population
Respondents selected to be interviewed are available
and willing to cooperate
Respondents understand the question and have
knowledge
Motivate them to provide information (incentives)
response rates
56
STEP 1: WHAT INFORMATION TO COLLECT?
Translating research objectives into information
requirements
Question relevancy
Use backward approach- What would you do with the
answer if you knew it?

57
STEP 2: COMPARISON OF SURVEY
ADMINISTRATION METHODS
Criteria Best Second Third
Versatility
Number of Questions
Amount /Var of info
Presentation of stimuli

Personal
Personal
Internet

Mail/Internet
Phone/Internet
Personal

Phone
Mail
Mail/Phone
Response Time Internet/Phone Personal Mail
Cost Internet/Mail
$1-$10
Phone
$15-$30
Personal
$50+
Accuracy
Sampling Control
Supervisory control
Opport. for clarification

Personal
Internet/Mail
Personal

Phone
Phone
Phone

Mail/Internet
Personal
Mail/Internet
Respondent Convenience
Internet/Mail
Telephone Personal


STEP 3: DETERMINE THE SPECIFIC
QUESTIONS
Is this question necessary?

Are several questions needed instead of one?
Do you think Coca-Cola is a tasty and refreshing soft drink?
Why do you shop at Nike Town?
59
STEP 4: OVERCOME INABILITY OR
UNWILLINGNESS TO ANSWER
Inability to answer
Filter/screening questions
Dont know option
Provide context to facilitate the memory
Unwillingness to answer
Reduce the effort (recall vs. recognition task, rating vs. rank order)
Justify purpose (legitimate reason)
Sensitive information
60
STEP 5: CHOOSE QUESTION STRUCTURE
Unstructured: Open-response questions
What do you think of people who patronize discount department
stores?

Structured: Closed-response questions
Multiple choice questions
Dichotomous questions
Scales

61
Response Formats Open-Ended
Advantages:
Respondents can give general (unbiased) reactions to
questions such as Why do you say that brand X is
better?
Response given in real world terminology. i.e.,
consumers own language
Can help interpret closed ended data.
e.g. Why was color the most important product attribute?
May suggest additional alternatives to be used in closed-
ended questions.

62
Response Formats Open-Ended
Drawbacks:
Often not good for self-administered surveys
Answers depend on respondent ability to
articulate
Interviewer bias can be a problem
Post-coding is tedious
63
Response Formats Closed-Ended
The respondent is provided with pre-determined
descriptions and selects one or more of them
Advantages:
easy to use in field
less threatening for respondent
ability to reduce interviewer bias
simple to code and enter data
listing responses may jog respondents memory
cheaper to administer

64
Response Formats Closed-Ended
Disadvantages:
Usually requires pre-testing
Presumes the list of responses is complete

Exploratory research using open-ended questions
can be used to suggest the items and scales in the
quantitative survey


65
STEP 6: DETERMINE THE QUESTION
WORDING
Be specific (who, what, when, where etc.)
Which brand of shampoo do you use?
Use simple, conventional (ordinary) words
Do you think the distribution of soft drinks is adequate?
Avoid ambiguity (usually, normally, often etc.)
In a typical month, how often do you shop in department stores?
(Never, occasionally, sometimes, often, regularly)
Dual Statements: agreement and disagreement, like and
dislike



66
Question wording-contd
Avoid leading and biasing questions
Do you think that we should buy imported goods when that would put
local labor out of work?
Is Colgate your favorite toothpaste?
Avoid implicit alternatives
Do you like to fly when traveling short distances?
Avoid estimates
What is the annual per capita expenditure on groceries in your
households?


67
STEP 7: ARRANGE IN PROPER ORDER
Responses to earlier questions may bias responses to
later questions
Funnel Approach
asking specific questions first may bias responses to general questions
asked later
What considerations are important to you in selecting a department store?
In selecting a department store, how important is convenience of location?
EXAMPLE OF ORDER BIAS:
DOES THE QUESTION CREATE THE ANSWER?
69
1. No question asked 2.8
2. Asked only about advantages 16.7
3. Asked only about disadvantages 0.0
4. Asked about both advantages and
disadvantages
5.7
Percentage of Respondents
Very Much Interested in
Buying New Product
Questions Preceding Buying Interest Question
STEP 8: IDENTIFY THE LAYOUT
Layout Guidelines
Open the survey with an easy and non-threatening question
Smooth and logical flow
From general to specific
Sensitive or difficult questions should not be placed at the
beginning of the questionnaire

Aesthetically pleasing: Font size, paragraph distance, the
number of questions in one page, questions are numbered
clearly
70
STEP 9-10: PRE-TESTING AND
CORRECTING PROBLEMS
Pre-testing specific questions
Variation
Meaning
Task difficulty
Respondent interest and attention
Response latencies
Question saliency
71
REVIEW:
Understand the advantages and disadvantages of each
type of measurement scale
Conduct a mini market research when designing a
questionnaire
Search for and use proven questions
Pretest questionnaire
72
HOMEWORK
Assigned reading from course pack (Harvard School
Survey, Causal Inference)
Textbook reading (chapter 8, 9, 10)
Group Assignment 1 due on Friday night, hardcopy, print,
in my mailbox
73