With

the global leader in sampling and data services

To Catch a Thief:
Figh6ng Respondent Fraud
Jared Schiers
VP Access, Supply & Engagement

SSI Conden*al
2013 Survey Sampling Interna6onal

| 1 |

Taking a holis6c view of data quality

1. Sample representa0on:
How eec*vely does our sample resemble the popula*on under
study?

2. Respondent accuracy:
How eec*vely do the answers resemble the respondents true
feelings?

SSI Conden*al
2013 Survey Sampling Interna6onal

| 2 |

3 Issues with respondent level data quality

1. Respondent is unable to give accurate informa0on (instrument)

Understand the ques6on being posed

Retrieve the relevant experience (i.e. remember it)

Judge it (i.e. relate it to the ques6on)

Be able to respond with the available answers op6ons

2. Respondent isnt mo0vated to give accurate informa0on

(process)

Non deliberate lying (e.g. social desirability bias)

InaOen6on (boredom, fa6gue)

Task diculty and Incen6ve mismatch

3. Respondent is a thief (fraud)

SSI Conden*al
2013 Survey Sampling Interna6onal

| 3 |

Luckily the fraud problem is SMALL

Fraudsters make up a very small % of all respondents
Kantar Opera6ons reported iden6fying 1.3% of their panelists as suspicious at the
ASC Conference, London 2008

Most of these bad accounts are iden0ed before they

par0cipate in research
With focus, fraudsters can be iden0ed because their survey
answers look dierent (dont know the topic) and appear in
behavioral clusters

SSI Conden*al
2013 Survey Sampling Interna6onal

| 4 |

But fraud is a problem we share as an industry

1. How many of us have accepted that there are fraudsters
in our data?
2. How many of us have ques0oned interes0ng results from
our data?
3. How many of us have seen the cost of research rise in
data collec0on and in our own 0me spent ques0oning
data?

SSI Conden*al
2013 Survey Sampling Interna6onal

| 5 |

Why would thieves target market research?

1. We provide incen0ves to par0cipate
2. Our exper0se is understanding marketsnot security
3. Techniques used against one MR company are oWen
eec0ve against others
4. Law enforcement is focused on other high-risk ver0cals
(e.g. online banking)

SSI Conden*al
2013 Survey Sampling Interna6onal

| 6 |

Whats in it for the fraudster?

Market researchers generally provide only a very
small Thank You for par6cipa6on in research






SSI Conden*al
2013 Survey Sampling Interna6onal

| 7 |

The carrot increases with scale and automa6on

The incen6ve to steal
increases with capability
to automate the tasks
required to achieve pay-
out
As the incen6ve to steal
increases so does the
sophis6ca6on of the
fraudster

SSI Conden*al
2013 Survey Sampling Interna6onal

| 8 |

NO Silver Bullets Technology alone cant prevent fraud

Fraud preven6on in MR is following the evolu6on of chess

1. Human phase:
Grandmasters dominate the game of chess
MR fraud specialists control fraud
Experts can adapt and react, but impossible to evaluate every

possible piece of data

Adaptable, but not easily scalable

SSI Conden*al
2013 Survey Sampling Interna6onal

| 9 |

NO Silver Bullets Technology alone cant prevent fraud

Fraud preven6on in MR is following the evolu6on of chess

2. Technology phase:
IBMs Deep Blue defeats GM Garry Kasparov
MR introduces technology solu6ons like TrueSample, RelevantID,

and others
Deep Blue can run through every game ever played, but cant
conceptualize new strategies
Scalable, but not quickly adaptable

SSI Conden*al
2013 Survey Sampling Interna6onal

| 10 |

NO Silver Bullets Technology alone cant prevent fraud

Fraud preven6on in MR is following the evolu6on of chess

3. Hybrid phase:
Games played by teams (GM Garry Kasparov teams with Deep Fritz 14)
Strategy driven by the grandmaster and the poten6al itera6ons of the

move run by the machine

Adaptability and scalability

SSI Conden*al
2013 Survey Sampling Interna6onal

| 11 |

8 Strategies to prevent and manage fraud

1. Anchor respondent iden6ty to the physical world
2. Mine observed behavioral data to iden6fy thieves

Leverage technology to generate data and automate

fraud preven6on
4. Conrm topical understanding
5. Delay respondent pay-out
6. Partner with reward vendors to implement safeguards
7. Oer less abrac0ve rewards (to fraudsters)
8. Reduce the carrot through itera6ve evolu6on
3.

SSI Conden*al
2013 Survey Sampling Interna6onal

| 12 |

1. Anchor respondent to the physical world

Choosing the right anchor is cri0cal
Physical anchors are generally more dicult to create en-masse
than digital anchors

By increasing the diculty to mass replicate accounts, we

decrease the aOrac6veness of the fraud opportunity

More eec6ve if you can validate the anchor through process or

third party

Mailing address
Telephone #

SSI Conden*al
2013 Survey Sampling Interna6onal

Bank account #
Photo ID

| 13 |

Drivers license #

1. Anchor respondent to the physical world

Telephone 2-factor authen0ca0on
Respondent provides a live telephone # for a voice call or SMS message
We deliver a PIN via voice or SMS
Respondent enters that valid PIN to complete reward redemp6on

SSI Conden*al
2013 Survey Sampling Interna6onal

| 14 |

1. Anchor respondent to the physical world

Postal redemp0on restric0on
A physical check is the only reward available for the rst redemp6on
Respondent provides the physical address and name for the check to
be sent
The respondent cashes the check valida6ng their name and address

SSI Conden*al
2013 Survey Sampling Interna6onal

| 15 |

2. Mining observa6onal data

Trust is gained in inches and lost in miles

Respondents constantly exhibit behavior while they are in our systems

Good behavior over 6me conrms legi6macy
Anomalous behavior is a trigger to suspect fraud
A good respondent can trigger 1 or 2, but not 3, 4, or 5 or more
A few of the 1,000s of characteris0cs to watch for

Over-qualica6on
Geographic inconsistency
Survey overlapping
Over-par6cipa6on in high
incen6ve projects

SSI Conden*al
2013 Survey Sampling Interna6onal

| 16 |

Over-ac6ve
PaOerned email format
IPs or ISP of known fraudsters
Consistently odd survey length
Par6cipa6on at odd 6mes

2. Fraudsters can be iden6ed in clusters

Behavior considered good from a single respondent
illustrates fraud when exhibited by 100 respondents.
1 respondent from Michigan signs
1 respondent qualies for 5 out of 5
up Oct 15th at 3:15 PM, completes 5
surveys they aOempt.
surveys within 72 hours, and
100 respondents from the same IP
th
redeems on Oct 18 .
address take the same 5 surveys
100 respondents exhibit the same
with the same qualica6on rate
behavior
1 respondent signs up with the email address
j1sch2@hotmail.com
100 respondents exhibit the same behavior
j3sch4@hotmail.com
j12sch34@hotmail.com
SSI Conden*al
2013 Survey Sampling Interna6onal

| 17 |

3. Leverage technology to automate

Technology allows us to act on a broader scale to behavioral
paOerns and vulnerabili6es found during data mining.
Every technology has its own vulnerabili6es, combining many of them
allows us to mi6gate the risk.
Most technologies are rules based and require the user to tune their
sensi6vity over 6me.

SSI Conden*al
2013 Survey Sampling Interna6onal

| 18 |

4. Conrming topical understanding

Fraudsters can be iden6ed by tes6ng their understanding of the
topic during the survey instrument
They dont understand the topic under study
They generally dont read the ques6ons they are answering

Open-Ended Trap Ques0ons

Close-Ended Trap Ques0ons

Dicult to create many unique and

relevant answers

Fraudsters are good at automa6ng

processes

Good at iden6fying fraud

Rela0vely simple to iden0fy and

script the survey path

Requires post-hoc evalua6on

BeOer at iden6fying inaOen6on from

good respondents

SSI Conden*al
2013 Survey Sampling Interna6onal

| 19 |

4. Important on low incidence projects

How would you describe yourself when it comes to health, diet,
and exercise?
Professional athlete - I compete professionally in sports or races
2. Compe66ve athlete - I compete in sports or races, but not professionally
3. Ac6ve lifestyle - I engage in regular exercise, ac6vity and play
4. Health conscious - I manage my health through diet and exercise
5. None of the above
1.

1 in
20,000

An honest respondent has a 0.00524% chance of qualifying

4,000 in
20,000

A fraudster has a 20%+ chance of qualifying

315 million people in the US (Census)

16,500 professional athletes in the US (Bureau of Labor
Sta6s6cs)
1 in 5 chance of qualica6on if chosen at random
Or more if the qualifying criteria is easily iden6able

Low incidence and high incen0ve projects are vulnerable to fraud abacks
SSI Conden*al
2013 Survey Sampling Interna6onal

| 20 |

5. Delay respondent pay-out

Fraudsters prefer in-and-out fraud opportuni6es

The longer the delay between account registra6on and payout, the
less aOrac6ve the fraud opportunity
The more 6me a fraudster spends in the system the more data they
generate
Increasing the likelihood a fraudster gets caught and exposes their
signature

New account block

Claim delay

Claim process is unavailable

un6l a new account has
reached a pre-dened age

SSI Conden*al
2013 Survey Sampling Interna6onal

Rewards can be claimed,

but delivery is delayed for a
pre-dened interval, while
the account is scru6nized

| 21 |

6. Safeguards built into the reward

Reward redemp6on is our last chance to stop a fraudster

Reward vendors have safeguards against fraud
Leveraging those to stop the fraudster from receiving their
reward
No reward = no incen6ve to con6nue the aOack
PayPal veried: physical anchor, nancial sector
iden6ty control, and a uniqueness check

SSI Conden*al
2013 Survey Sampling Interna6onal

Respondent provides PayPal with a valid bank

account or credit card and address

PayPal validates that account is ac6ve, owned by the

respondent, and is unique to this PayPal user

Respondent must provide us with the email address

aOached to the veried account

| 22 |

7. Oer less aOrac6ve rewards (to fraudsters)

Informa0on rewards (e.g. a copy of the survey

results) are much less appealing to fraudsters:
they arent invested in the research
Charitable rewards (e.g. Red Cross dona6ons) are
much less appealing to fraudsters: we havent run
across Robin Hood yet

SSI Conden*al
2013 Survey Sampling Interna6onal

| 23 |

8. Itera6ve improvements to reduce the carrot

1.Diverse use of strategies increases the eort required for fraud

2.Con6nuous evolu6on and itera6on keeps the barriers high
3.We only need to out run other fraud opportuni6es to be
protected

SSI Conden*al
2013 Survey Sampling Interna6onal

| 24 |

Apply these strategies to prevent and manage fraud

1.
2.
3.
4.
5.
6.
7.
8.

SSI Conden*al
2013 Survey Sampling Interna6onal

Anchor respondent iden6ty

Mine behavioral data
Leverage technology
Conrm understanding
Delay pay-out
Partner with reward vendors
Less abrac0ve rewards
Reduce the carrot

| 25 |

But apply them holis6cally

Applying fraud preven6on strategies across every stage in
a respondents lifecycle increases the eec6veness

SSI Conden*al
2013 Survey Sampling Interna6onal

| 26 |

And balance respondent strain and fraud preven6on

Its good to remind ourselves that the good guys (99%)
experience the same security measures we use to deter the
fraudsters (1%)
Too much respondent strain and we deter good respondents from
par6cipa6ng in research, adversely impac6ng our sample frame
Too lax in fraud preven6on and we allow fraudsters to par6cipate, adversely
impac6ng our data

Respondent
strain

Fraud
preven0on
SSI Conden*al
2013 Survey Sampling Interna6onal

| 27 |

For more informa6on

Email us at info@surveysampling.com:

If you would like to request addi6onal informa6on, please put
webinar in the subject line and request one or both of the
following in the body of your email:

Todays presenta6on
Point of view (POV) on Impact of Fraud
Please provide your name, 6tle, company.

SSI Conden*al
| 22013
8 | Survey Sampling Interna6onal

| 28 |

Ques6ons

SSI Conden*al
2013 Survey Sampling Interna6onal

To Con6nue the
Conversa6on
Visit our Facebook page:
facebook.com/surveysampling

or Tweet #SSIWebinar

SSI Conden*al
| 23013
0 | Survey Sampling Interna6onal

surveysampling.com
info@surveysampling.com
Jared.Schiers@surveysampling.com

SSI Conden*al
2013 Survey Sampling Interna6onal

| 31 |
| 31 |