EFL Psychometric Credit Scores

Psychometric testing in credit risk
assessment
A digital readiness primer and case study
@eflglobal // Amie.Vaccaro@EFLglobal.com
EFL Intro
Where EFL works best
Digital readiness primer

Agenda HCD: Designing for the feature
phone
Equity Bank case study
Q&A
Traditional credit scoring is blind to billions of people
3 billion are unscorable, another 3 billion will enter the middle class in the next 20 years
Covered by
70%
credit Not covered
bureau,
30%
•  Billions of good applicants rejected
Not
covered, •  Enormous missed opportunity
70%
Source: http://www.ey.com/gl/en/issues/driving-growth/middle-class-growth-in-emerging-markets; http://data.worldbank.org/indicator/IC.CRD.PRVT.ZS
3 Copyright © 2017. All Rights Reserved.

We are entering a new era of lending
And credit scoring needs to adapt to serve more people with end to end digital processes
Loan officers Loan officers Loan officers Digital approvals

with deep + + without loan officers
relationships Traditional Traditional and +
credit scoring alternative scoring Traditional and
alternative scoring
1990’s 2000’s 2010’s Today

Predictive power and availability vary by data source
Psychometric is only data source with 100% availability – everyone has a personality
More Available
(Hit rate on sample of low-income applicants in emerging markets)
100% PSYCHOMETRIC DATA

90%
80%
MOBILE DATA
70%
Availability
60%
50%
40%
30%
ONLINE DATA BUREAU DATA
Less Available
20%
10%
0%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Less Predictive More Predictive
Predictive Power
(Out of sample gini coefficient with comparable model & sample)

EFL is the proven universal score for today’s lenders
Empowering digital and in-person approvals across $1.5B+ and counting in disbursed loans
Alternative scoring leader Universal coverage Third party vetted
•  4 years R&D at Harvard on •  Score anyone, anywhere •  Published 3rd party

psychometric, demographic, with our psychometric and validations by World Bank
social & CDR scoring behavioral science based and others Link
assessment
•  6 years implementing •  Won multiple awards
scores with lenders across •  No client history or 3rd including G-20 SME
Africa, Latin America & Asia party database needed Finance Challenge, African
Business Award for
Innovation, and Mahindra
Finance Challenge

EFL assesses risk with a short online loan application
How we score anyone The character traits our
assessment measures
EFL Algorithms
Analyze the answers and
Unscorable Applicant determine risk level Score Generation
Takes EFL assessment on Create 3-digit EFL Score and
the web, mobile or SMS
2 Score Confidence rating Attitude
1
1 3 Control
Entrepreneurial potential
Interaction metadata
4
Continuous Learning Social Behavior
EFL receives monthly loan
repayment used to constantly
customize and improve
models

EFL Intro

phone
Q&A
EFL works best when there’s a clear need for an
alternative credit score, and the FI is challenged by:
1. 2. 3.
Low Acceptance Rates High Levels Costly & Lengthy Loan

(due to lack of data) of Default Application Processes
High rejection rates due Default rates over target; Loan application process
to lack of data – too hesitant to dramatically takes time and is resource-
many grey zone or thin decrease acceptance intensive for lender
file clients rate

Use the EFL score to…
Our partners typically use our behavioral psychometric score in one of three ways:
1. 2. 3.
Increase Your Decrease Your Increase Your

Acceptance Rate Default Rate Operational Efficiency;
Lower Application Cost
Approve more people, Decrease and control Fast track: Cut pre-filters
without taking on more your default rate, with for high-scoring
risk limited volume impact applications

‘Institutional Readiness’ plays a large role in success
We’ve noticed certain traits within organizations that allow the EFL digital credit assessment to thrive
High Volume,
Credit Score Ready Driven to Push Innovation Mindset
Ability to receive a 3-digit, Strong desire across Willingness to try new solutions
numerical, credit score and use it in management & teams to and consider a departure from
their existing credit decision making implement and scale traditional processes.
process. solution quickly.
Receiving thousands of credit Motivated team, with Willingness to take calculated

applications each month. willingness and motivation to risks and move in digital direction.
change existing process
Extra Plus? Ability to combine EFL
score with other credit scores via
scorecard or ensemble model

EFL Intro

phone
Q&A
Financial institutions are going digital globally
Digital technologies can cut the cost of providing financial services by 80 to 90% and improve scalability
Annual cost to serve one customer in emerging economies,

2014 ($)
Accounts Cash-in, cash-out Transactions Total Traditional

bank branch
Digital
Cost savings
due to digitization
Source: McKinsey Global Payments Map; “Fighting poverty, profitably: Transforming the economics of payments to build sustainable, inclusive financial
systems,” Bill and Melinda Gates Foundation, September 2013; McKinsey Global Institute analysis

Going digital? It’s more than building an app
What does it mean for a bank to be “digital”?
Being digital means being scalable. If you have to add staff to increase lending, you aren’t digital.
Acquisition Affordability Risk KYC / ID Servicing

Online acquisition is Web scrappers are Multiple companies, Proving ID online is Few companies are
possible, though emerging to support including EFL are an emerging field, focused on are looking
usually pricey. digital affordability, proving that risk however, in EM to outsource
however, limited profiles can be built limited databases collections.
coverage and via digital coverage is a factor.
inconsistent data interactions.
remain an issue

But there are challenges to going digital
Moving the credit application process from in person with a loan officer to online comes with its own risks
Increased Risk
Digital applications can come with increased
risk and fraud. Effective credit scoring is key EFL is your digital
to keeping a healthy portfolio.
loan officer.
Efficiently assess risk

Lose Personal Touch on anyone, anywhere,
No human interaction can dampen the bank- online. Build trust with
to-customer relationship. Tools are needed applicants.
to increase human designed interaction.

Going digital is a process
We work with partners at each stage
100%
Fully digital process;

pushing for scale
No existing digital
processes Integrate EFL into a
digital application:
% DIGITAL
Use EFL as a stand alone •  Control risk while
tool, focused on: Hybrid model – maintaining
•  Customer Acquisition some parts digitized efficiency.
•  Reject Recovery •  Increase
Add EFL to partially digital acceptance rates.
process flow, see results in:
•  Operational efficiency
0%

EFL Intro

phone
Q&A
Product Design: Understand Your “Bones”
What framework are you working within?
•  SMS = a necessarily text-heavy product

•  Question length limited to 160 chars –containing both question and answer key
•  Need to battle potential mistrust of the product – concerns of SMS scams
•  Open answer responses – need to validate and send appropriate responses
Understand your customers:

•  Design for the mass-market, and try to incorporate the low-common denominator as
much as possible. People will struggle with:
Literacy Understanding Technology Connectivity

Know What You NEED
Hypothesize:
Why does someone
default?
•  Relatively easy to understand (both
the question and desired answer
Intelligence General Behavior
format)
•  Predictive even within question

number and character limits Honesty Stability
•  Forgiving if users make mistakes

and need to correct answers
Fraud Attitudes
•  Trust building – Reliable product
that is connected to bank and
reward believed
Product design PROCESS
3. Data analysis of
flows and field 1. Field observation
interviews: ensure & interviews: on
which key metrics both question
are improving and framing and content
which aren’t
2. Experimental design:
validate/quantify hypotheses
from field via more real-life
channel using flow split-testing
and within-flow experiments
1 Get OUT! START
Watch! How and when are people

interacting with their phones?
Get to know your competition – what mobile services are people

using? What do they like / dislike about them?
Build: Fast and easy prototypes

Paper print outs of phones with text questions
“Hey! Can I ask your opinion?”
Talk with people- get them to use your product, explain as little as
possible.
“What do you think it means?”

2 Create the RIGHT data
Test many versions of your product, over wide audiences, through various
channels
CHANNEL EXPERIMENTAL DESIGN
Parallel Flows
Test two similar products at the same time, with different
versions of the same content you need to test – isolates
the “treatment effects”
In Person
Single Flows
Intersperse new iterations of product parts within your

current product iteration to see if behavior changes –
controls for individual differences
Distance Testing
Output: Well-structured quantitative and qualitative data to analyze

3 Explore Your DATA
Define metrics for what makes the product “good” – explore the data to see what's
working…
•  Answer distribution (extreme values vs. well-distributed
•  Answer timing (confusing questions take longer)
•  Drop off rates (do certain questions cause people to disengage?)
•  % of unusable answers
…Do we see an improvement in performance
metrics as a result of a new iteration?
Percent of responses with extreme

answers or unusable answers
BEFORE
10 Point Scale
Reply with a NUMBER ONLY on a scale from 0 - 35%
10 (0=never 5=sometimes 10=always). Do you 30%

show up late or miss appointments? 25%
20%
15%
AFTER
Percentages 10%
What PERCENT (%) of the time do you show up 5%
late or miss appointments? 0%

10-point scale Percent scale

Final SMS Content: Three Targeted Core Traits
Objective: Identify a few core traits that are most relevant to the population. Focus a majority of
psychometric content around these to max predictive power.
Understanding the
user’s concept of Understanding the
honesty and fairness user’s sense of
in the context of her personal responsibility
financial life for and control over
her life
Locus of
Integrity Control
% of responses with
no valid answer
25%
Opportunism/
20%
Progress
mindset Understanding the
15%
user’s perspective on
10%
opportunities and
Locus of Confidence, motivations to take risks
Control, Optimism, to improve her life
Integrity, GSE
Opportunism

Final SMS Product
Maximizing predictive power Reply with a

NUMBER in KSH. If
1)  3 core traits focused on
you found 100,000
2)  Granular answer potential (%s) KSH today, how
much would you
… through ensuring understandability spend on leisure?
1)  Answer in percentages
2)  Non-hypotheticals – concrete questions / “mini stories”
3)  Data quality check – smartphone? Help on survey?
… and building trust
1)  Human approach – make people feel comfortable
2)  Start with easy, but non-invasive questions (back load more intrusive
demographic questions)
3)  Trust this product – start with the person’s name

EFL Intro

phone
Q&A
Overview - Scoring on the feature phone
Equity extends credit through chat, proves ability to boost acceptance rates
How does it work?

The Challenge
High volume, feature phone mobile lending comes with Tine, an Equity
significant risk. Equity needed an innovative solution to accept customer, applies
more of their customers without taking on additional risk. for a loan via her
Equitel SIM card.
The Engagement
EFL designed a new product – taking the learnings of our
traditional psychometric content, and transforming them into a 22
She’s new to the
SMS question assessment. Instead of being flat-out rejected, low bank, so she
data customers could take the assessment remotely on their receives the first
feature phones, and gain an opportunity to be approved.
EFL SMS asking
her to start the test.
The Results
Collecting 24k completed SMS tests, EFL built a custom SMS
model, and saw significant predictive power in the data Tine’s survey is
(psychometric & meta). The strong model results indicate analyzed, and her
potential SMS uses across markets / channels. credit score sent back
to Equity in real time.
Tine is disbursed!
28 Copyright © 2017 EFL. All Rights Reserved.

Designing the SMS Pilot – Process & Criteria
FIVE STEPS UNDERTAKEN TO
Equity & EFL partnered in 2013 to develop a custom
COMPLETE THE SMS PILOT…
feature phone credit scoring tool
1 Design SMS
Test
…TO VALIDATE TWO MAIN HYPOTHESIS
2 Build & Integrate

SMS Platform
HYPOTHESIS DESIGN MEASUREMENT
1. The SMS channel Test
3 Run Pilot;
Gather Results
will facilitate a high
conversion rate.
UX of SMS
Test
Completion
Rate (%s in
funnel)
4 Build Custom
Model
2. SMS psychometric
5 Out-of-Sample Predictive
testing will enable Equity Psychometric
Model Power of SMS
Validation to better segment and Assessment Model
predict risk

Assess willingness to repay through a conversation
Through EFL’s chatbot platform, we converse with your clients, one SMS at a
time. Through an automated conversation, we ask our traditional psychometric
questions – straight to their phone. Predictive, real-time, credit scores available
in minutes.
What percent (%)
of businesses
Derives predictive power from: cheat their
clients?
1.  Psychometric answers
2.  Timing data SMS – USSD

3.  Free-text input data ~20 psychometric questions
4.  Self-reported demographic data 4 test sections
Any phone, any location
Only need to score once

Seamlessly assess an applicant’s character
PRODUCT FLOW
CREDIT
SCORE
337

Benson, a new Instead, he immidietely Benson engages, Credit Score

customer, applies for receives an inia9on conversa9on style, in the returned to lender
loan. Normally, he SMS masked from the SMS test.
would be rejected. lender
Behavioral – Text Analysis; Timer Analysis Psychometrics

What time of day are they responding?
What are response answer patterns?
Do they understand the question?
How is their spelling? Grammar?
What words or phrases do they use?
© 2016, EFL Global. All rights reserved. 31
EFL SMS test seamlessly fits in with existing UX mobile
lending product flow
Before rejecting low-data and new applicants, Equity can digitally score and asses risk on borrowers.
CREDIT
SCORE
337

Benson, a customer Immidietely Benson engages,conversa9on Credit Score

of Equity Bank, receives inia9on style, in the SMS test. -
applies for loan via SMS from - Returned in real-
Equitel SIM SDK/ dedicated Equity Answers test in 4 secPons. SMS Pme through API to
USSD menu short code reminders sent to keep Benson Equity’s CU
- - focused.
Benson replies to
the SMS (zero- Total test )me <15 minutes,
rated) and begins taken at once or over hours/
the test days

Results: Predictive Model; Strong Usability
Customized Model Out of

High Sample
With 24k tests completed, EFL built a Default
AUC:
customized model for SMS – achieving a 63.7
predictive power of 63.7% AUC
Default Rate
Default Rate
Usability validated through low drop off
rates.
Average completion rate of 78%, with an
average test time of 33 minutes.
Low EFL Score Quartiles
80% of drop-off occurred within first 4 questions. Default
1 Worst 12 23 34 54Best
56% of tests were completed within 15 minutes. Worst Scoring Best Scoring
bad45_m3 bad30_m3 bad60_m3

Ability to increase acceptance rates by 20% when
combined with internal behavioral score
High
Cumulated Default Rates
Default
Default Rate
EFL & Equity’s Internal
Behavioral Score
Combining the EFL score with an
internal behavioral (transactional) score:
Low
Default
boost acceptance rate by 20% High 1.2x 1x Low
without taking on additional risk Acceptance Rate
20%
Ensemble Experian
Behavioral EFL

EFL finds different bads, and different goods, than
traditional credit bureau scores
Cumulated Default Rates
High
Default
Default Rate
EFL & Traditional Bureau
Score
Combining the EFL score with a
traditional bureau score:
Low
boost acceptance rate by 53% Default
without taking on additional risk

High Low
Acceptance Rate
Acceptance
53% Acceptance
Ensemble Bureu
Bureau EFL

EFL Intro

phone
Q&A

EFL Psychometric Credit Scores

Cargado por

Información del documento

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

EFL Psychometric Credit Scores

Cargado por

Copyright:

Formatos disponibles

Psychometric testing in credit risk

Where EFL works best

Digital readiness primer

Equity Bank case study

Source: http://www.ey.com/gl/en/issues/driving-growth/middle-class-growth-in-emerging-markets; http://data.worldbank.org/indicator/IC.CRD.PRVT.ZS

3 Copyright © 2017. All Rights Reserved.

Loan officers Loan officers Loan officers Digital approvals

1990’s 2000’s 2010’s Today

100% PSYCHOMETRIC DATA

6 Copyright © 2017. All Rights Reserved.

Alternative scoring leader Universal coverage Third party vetted

• 4 years R&D at Harvard on • Score anyone, anywhere • Published 3rd party

7 Copyright © 2017. All Rights Reserved.

8 Copyright © 2017. All Rights Reserved.

Where EFL works best

Digital readiness primer

Equity Bank case study

Low Acceptance Rates High Levels Costly & Lengthy Loan

10 Copyright © 2017. All Rights Reserved.

Increase Your Decrease Your Increase Your

11 Copyright © 2017. All Rights Reserved.

Receiving thousands of credit Motivated team, with Willingness to take calculated

12 Copyright © 2017. All Rights Reserved.

Where EFL works best

Digital readiness primer

Equity Bank case study

Annual cost to serve one customer in emerging economies,

Accounts Cash-in, cash-out Transactions Total Traditional

14 Copyright © 2017. All Rights Reserved.

Acquisition Affordability Risk KYC / ID Servicing

15 Copyright © 2017. All Rights Reserved.

Efficiently assess risk

16 Copyright © 2017. All Rights Reserved.

Fully digital process;

17 Copyright © 2017. All Rights Reserved.

Where EFL works best

Digital readiness primer

Equity Bank case study

• SMS = a necessarily text-heavy product

Understand your customers:

Literacy Understanding Technology Connectivity

19 Copyright © 2017. All Rights Reserved.

• Predictive even within question

• Forgiving if users make mistakes

Watch! How and when are people

Get to know your competition – what mobile services are people

Build: Fast and easy prototypes

“Hey! Can I ask your opinion?”

“What do you think it means?”

22 Copyright © 2017. All Rights Reserved.

Intersperse new iterations of product parts within your

Output: Well-structured quantitative and qualitative data to analyze

23 Copyright © 2017. All Rights Reserved.

Percent of responses with extreme

10 (0=never 5=sometimes 10=always). Do you 30%

What PERCENT (%) of the time do you show up 5%

late or miss appointments? 0%

24 Copyright © 2017. All Rights Reserved.

25 Copyright © 2017. All Rights Reserved.

Maximizing predictive power Reply with a

26 Copyright © 2017. All Rights Reserved.

Where EFL works best

Digital readiness primer

Equity Bank case study

•  4 years R&D at Harvard on •  Score anyone, anywhere •  Published 3rd party

•  SMS = a necessarily text-heavy product

•  Predictive even within question

•  Forgiving if users make mistakes

2.  Timing data SMS – USSD