Modeling Uncertainty

 Fun with decisions and uncertainty

 Sensitivity analysis

 Preview Monte-Carlo simulation

 Brief review of probability distributions and

statistics using Excel
 Warning! Decision and risk
analysis is a radical concept
 People, in general, are not comfortable with
probabilistic reasoning
 Most people commonly use point estimates for
uncertain quantities and then may carry out a limited
1 or 2 variable sensitivity analysis
 Everyone will say, “too much thinking and planning
required, don’t have time in the real world”
 but somehow, people have time to revisit the messes they
make with “seat of the pants” decision making
 Importance and Difficulty of
Uncertainty Modeling
 The world is uncertain
 Replacing random quantities with averages or single
“guesstimates” can be dangerous
 The Flaw of Averages
 Allows prediction of distribution of results
 Not just one predicted number or outcome
 Sensitivity analysis of outputs to inputs
 Which inputs really affect the outputs?
 Fun with Uncertainty
 Sensitivity Analysis
 Sensitivity analysis (SA) a big part of modeling
and analysis
 SA = “What matters in this decision?”
 which variables might I want to explicit model as
uncertain and which ones might I just as well fix to my
best guess of their value?
 On which variables should we focus our attention on
either changing their value or predicting their value?
 No “optimal” SA procedure exists
 SA can help identify Type III errors - solving the
wrong problem

4
 Some SA Techniques
 Scenarios – base, pessimistic, optimistic
 How did we do with “scenario planning”?
 1-way and 2-way data tables and associated graphs
 as in the Break Even spreadsheet
 Tornado diagrams
 a one variable at a time technique
 Top Rank –Excel add-in for simple “What if?”
 Risk Analysis or Spreadsheet Simulation
 direct modeling of uncertainty through probability distributions
 @Risk , CrystalBall – sophisticated Excel add-ins

5
 Tornado Diagrams

 Graphical sensitivity analysis technique

 Create base, low and high value scenarios for
each input variable
 Set all variables at base value
 “Wiggle” each variable to its low and high
values, one at a time.
 A one-way sensitivity analysis technique
 Calculate total profit for each scenario
 Create “tornado diagram” - Excel
GreatThreads-Tornado.xls

From “Making Hard Decisions” by Clemen

6
 Using Top Rank (see p714-721 in PMS)
 Quit Excel (Top Rank acting flaky)
 Start | Programs | Palisade Decision Tools | Top Rank 1.5
 it will launch Excel and start
 Open up your file
 GreatThreads–Tornado.xls
 Select output cell and click “Add output cells” on TopRank
toolbar
 Click “Step through input cells” on TopRank toolbar and specify
which inputs are to be varied and by how much
 Click “Run what-if analysis” on toolbar
 When results come up, click the tornado graph toolbar button
and select tornado

7
Sensitivity Analysis with TopRank

Big bars means

high impact

8
Preview of Monte-Carlo Simulation
 Simple Excel Simulation example
 Revenue for 3 products with fixed prices and
random demands
 What is simulation
 When do you use simulation
 Real applications of simulation
 Prob distributions – the building blocks of
simulation

9
 Monte-Carlo Simulation
 A modeling approach that allows explicit incorporation of uncertainty
in spreadsheet models
 Got its start during the Manhattan project in WWII for modeling
nuclear devices
 One or more random elements modeled with probability distributions
 Sample from the input distributions many, many times
 Keep track of the values of the outputs for each sampling of the inputs
 Analyze the outputs
 Often called Risk Analysis
 Uncertainty is about values of unknown variables
 Risk is about consequences of uncertainty
 @RISK - Palisade Software
 Spreadsheets provide good environment for simulation
 Goes beyond expected values and point estimates
 Doing simulation involves more than just building models with
software
 must be probability/stats literate to do proper input and output analysis
 Our First Simple Simulation Model
 3ProductSimulation-template.xls
Three Product Revenue Simulation Model

Uniform Demand Distribution Revenue

Product Price Min Max Mean Mean
A $80 20 80
B $150 0 40
C $400 0 10

Total

 What is expected revenue?

 Deterministic: prices of products A,B,C
 Stochastic: demand for products A,B,C
 What is the variability in revenue?
11
A few simulation applications
 T Rowe Price 529 Simulator
 NFL play calling
 http://www.sciencedaily.com/releases/2006/04/06
0420232621.htm
 http://www.pigskinrevolution.com/index.html
 @Risk case studies

12
 Building a Spreadsheet Based Simulation Model

(1) Build
deterministic Inputs Formulas Outputs
model

Deterministic Inputs
(2) Choose
inputs to model Inputs
as random Stochastic or Uncertain or Random Inputs

(3) Model uncertain inputs with probability distributions

Uniform Exponential Empirical Normal Poisson
Discrete Probability Distribution of Demand

0.35

0.30

0.25
Probability

0.20

0.15

0.10

0.05

0.00
100 150 200 250 300
Demand

Many more…
 Building a Spreadsheet Based Simulation Model
“Running the model”
(4.1) “Manually”, through formulas and
(4) Recalculate either many rows or VBA (Ex 11.1)
spreadsheet many times 2 options
(4.2) Use spreadsheet simulation add-in
 @Risk such as @Risk or Crystal Ball (Ex 11.2)
 www.palisade.com  Crystal Ball
 www.decisioneering.com
 Probability: The Language of Uncertainty
Distributions: Building Blocks of Simulation
 Random variables
 Discrete probability distributions
 Expected value of a discrete random
variable
 Continuous probability distributions
 Using Excel’s probability and statistics
functions
 Using the RiskView add-in
You learned about most of the above and more in your
Statistics course. I’ll just do a quick refresher as needed on
some concepts we’ll need for this course.
 Random variables (RV) and
probability distributions
 RV is a variable whose value depends on the outcome of an uncertain
event(s)
 Low bid by competing firms, project completion date
 Demand for some product or service next year
 Number of patients requiring open heart surgery next month at Hospital H
 Cost of Drug X in December, 2004
 Probability of various outcomes determined by probability distribution
associated with the RV
 Probability distributions are the “shapes of RV’s”
 As modelers, we select appropriate distributions
 Probability distributions
 mathematical functions
 Assign numeric probabilities to uncertain events modeled by the
distribution

See “Distributions, Simulation and Excel Functions” handout that Prof. Doane
created and that I’ve posted on Web.
 Using Distributions for
Simulation
 We will model uncertain inputs with probability
distributions
 Need to be able to generate random numbers from various
probability distributions
 We may “fit” probability distributions to raw data to
serve as a convenient model of the data
 Simulation model outputs will be distributions
 Need to know how to compute various measures from
distributions
 Simulating different scenarios - Need to know how to
compare features of distributions with each other

17
Two Types of Distributions
Discrete Distributions
• Integer, countable X
• EX: # of warranty claims in a day
• P(X) is the probability at each point
• P(X) may be summed over X values

Continuous Distributions
• X defined over an interval
• EX: Length of stay for open heart surgery patients
• Points have no area
• Calculus gives area under curve
 P.D.F. vs. C.D.F
Probability Density Function
• X axis shows values of X
• Y axis shows probability
•  P(X) = 1 if discrete

•  f(x) = 1 if continuous
• Histogram is pdf for data

Cumulative Distribution Function

• X axis shows values of X
• Y axis shows cumulative probability
• 0  F(X)  1 and is non-decreasing
 Discrete RVs and Probability Distributions
Discrete Probability Distribution of Demand

DistributionReview.xls 0.35

0.30

0.25

 Countable # of outcome values

Probability
0.20

0.15

 Each possible outcome has an 0.10

0.05

associated probability 0.00

100 150 200 250 300
Demand

x Prob[X=x] Prob[X<=x]
Cumulative
Expected Value of Discrete RV
Demand Probability Probability n
100
150
0.30
0.20
0.30
0.50 E[ X ]   xi P[ X xi ]
200 0.30 0.80 i 1
250 0.15 0.95
300 0.05 1.00  A few discrete distributions
172.5 1.00
 Empirical
 Binomial – BINOMDIST()
 Poisson – POISSON()
Expected Demand Total Probability
Cumulative Distribution Function (CDF)
for a Discrete Random Variable

The probability a random variable X

takes on a value less than or equal to x.

F ( x)  Pr( X  x)   p( xi )
xi  x

Properties of the CDF

0  F ( x)  1 F(x) is nondecreasing in x

21
 Distribution Review
 Download DistributionReview.xls
 Let’s answer questions on sheet Discrete
 We’ll do Continuous sheet momentarily
 Excel has many probability and statistical
related functions
 Remember, probability distributions are a
type of model for some uncertain quantity
 Think of histograms as empirical probability
distribution functions
 Continuous RVs and Probability
Distributions
 Infinite # of outcome values b
 Has a probability distribution P[ X  b]   f ( x)dx

(density) function (pdf), f(x),
 We calculate probabilities over
Area under the f(x) curve
intervals using the cumulative
from –infinity to b
distribution function (cdf), F(x),
which is P[X<=b] 
E[ X ]   xf ( x)dx


Uniform f(x) Exponential f(x) Normal f(x)

 RiskView

 Excel Add-In, Part of Palisade Decision Tools Suite

 “Live” distribution viewing, Huge number of distributions
 Online Help has background info on distributions
 Start | Palisade Decision Tools | RiskView 4.5
 Can also launch from within Excel from the Palisade Decision
Tools toolbar (which is visible if any of the Palisade tools are
running, e.g. @Risk)
 A few useful distributions
D istribution Illustration C haracteristics
N orm al Th e fam iliar “bell-sh aped curve.”
S ym m etric, w ith a peak in th e
m iddle and gradually taperin g tails.

Pro: Fam iliar, well-kn own .

C on: E xtrem e outcom es possible.

Trun cated n orm al Sam e as n orm al but w ith lim its to

p reven t extrem e cases from arisin g.

Pro: N o wild outcom es.

C on: M ore com plicated.

Trian gular Has a central peak an d clearly-

d efin ed end poin ts (lowest, m ost
likely, h igh est). Can be skew ed.

Pro: Easy to un derstand.

C on: N o ex trem es can occur.

G en eral R e v e n u e f r o m A s s e t S a le D efin e an y k categories an d m ake

0 .4 0
sure th e probabilities sum to 1.
Pr o b ability

0 .3 0
0 .2 0
0 .1 0
Pro: Easy to un derstand.
0 .0 0 C on: N eed to create categories.
50 100 200 500 1000

Re v e n u e
 The Normal Distribution
 Two parameters: Mean, standard deviation
 Symmetric
 Standard normal distribution has mean=0, std dev=1
 Normally distributed data with any mean and standard
deviation can be converted to a N(0,1) by
standardizing

X 
X~N(,) Z Z~N(0,1)


Excel has a number of functions related to the normal distribution:

NORMDIST(), NORMINV()
NORMSDIST(), NORMSINV()
Let’s review handout “Excel Functions for Working with Normal
Distributions” and do the Continuous tab in DistributionReview.xls
 Descriptive Statistics in Excel
 Data Analysis Tool-Pak  AVERAGE(), STDEV(),
MEDIAN()
 FREQUENCY()
 PERCENTILE()
 RANK(),
PERCENTRANK()
 MIN(), MAX()
2 ways to create histograms
 Data Analysis Tool-Pak
 Default bins
StatReview.xls
 User specified bins

 FREQUENCY() array function

 The very special Uniform
Random Variable (r.v.)
r.v. has a distribution of type Uniform with a min=0 and max=1

 IfX~Uniform(0,1) Then E[X]=1/2

(expected value)
 X is equally likely to take any
value between 0 and 1
 Probability[X<=x] = x

 Excel’s RAND() function

Uniform Random Numbers for
Simulation
 Building blocks of simulation
 Modeling randomness
 Basis for generating random variables
 Normal, exponential, Poisson, triangular, etc.
 Need reliable stream of Uniform(0,1) RVs
 Excel’s RAND() function
 How do computers generate “random”
numbers?
 All examples in RandNum_Isken.xls. Let’s
open it.
Uniform Distribution Function
Implication of shape of
distribution?

Question How could you use a U(0,1) number to

create a random number between a and
b? Let’s do it in RandNum_Isken.xls
 Using U(0,1)’s to generate
other random variables
Walton example
“Find” U(0,1) random D E
number in cumulative 3 Demand distribution
4 Cumulative Prob Demand
distribution of random 5 0.00 100
variable you want to 6
7
0.30
0.50
150
200
generate. 8 0.80 250
9 0.95 300

A B C is the key. VLOOKUP syntax?

This
17 Simulation
18 Replication Random # Demand
19 1 0.1747 100
Return value of random
variable.
 Simulation NORMINV(.1747,160,40)=122.57
Replication Random # Demand
1 0.1747 122.57 NORMDIST(122.57,160,40,TRUE)=.1747

Using U(0,1)’s to
generate Normal
random CDF for N(160,40)
variables

Random # =122.57
 Generating Random Numbers
 Excel’s Data Analysis Tool-Pak
 Excel RAND() along with transformations
 Not possible for all distributions
 @Risk functions
 @Risk has myriad of functions for generating random
numbers from a wide variety of distributions
 The file ProbabilityDistributions.xls (Downloads
section of course web) illustrates generating various
random variables
 www.random.org

33
Some of the broadly applicable
insights...
 Explicit incorporation and quantification of risks and uncertainties is often
important
 Be wary of clairvoyant analysts!
 Several methods for trying to incorporate uncertainty in analysis
 Quantification of risk is difficult and subject to common human decision
biases
 Humans have hard time with uncertainty
 It’s important to guard against decision biases
 Awareness is half the battle
 It’s OK to say “I DON’T KNOW”
 Not all information is worth the cost or equally valid
 Obtaining data for some of these modeling approaches can be difficult
 probability estimation can be tough
 historical data may or may not exist
 What is Simulation?
• A simulation is a computer model that
attempts to imitate the behavior of a real
system or activity.
• Simulations helps to quantify relationships
among variables that are to complex to
analyze mathematically.
• If the simulation’s predictions differ from
what really happens, refine the model in a
systematic way until its predictions are in
close enough agreement with reality.

35
 What is Simulation?
 When Do We Simulate?
• In general, consider simulation when
- The system is complex
- Uncertainty exists in the variables
- Real experiments are impossible or costly
- The processes are repetitive
- Stakeholders can’t agree on policy

36
 What is Simulation?
 When Do We Simulate?
• Conversely, we are less inclined to simulate
when
- The system is simple
- Variables are stable or nonstochastic
- Real experiments are cheap and
nondisruptive
- The event will only happen once
- Stakeholders agree on policy 37
 What is Simulation?
 Advantages of Simulation
• In a deterministic model, variables can’t
vary.
• Simulation lets key variables change in
random but specified ways.
• Simulation helps us understand the range of
possible outcomes and their probabilities.
• Simulation allows sensitivity analysis.

38
 Advantages of Simulation
• Simulation is useful because it
- Is less disruptive than real experiments
- Forces us to state our assumptions clearly
- Helps us visualize the implications of our
assumptions
- Reveals system interdependencies
- Quantifies risk by showing probabilities of events
- Helps us see a range of possible outcomes
- Promotes constructive dialogue among
stakeholders
39
 Risk Assessment
• Risk assessment means thinking about a
range of outcomes and their probabilities.
• Variation is inevitable.
• Knowing the 95% range of possible values for
the decision variable as well as the “most
likely” value , is the point of risk assessment.
• Risk assessment is useful when the model is
complex.

40
 What is Simulation?
 Components of a Simulation Model

41
 What is Simulation?
 Components of a Simulation Model

42