QBM 101 Business Statistics

Dr. Lai Kee Huong

Department of Business Studies
Faculty of Business, Economics & Accounting
keehuong.lai@help.edu.my

SUBJECT OUTLINE:
Module

1: Introduction; organizing
and graphing data; numerical
descriptive measures

Module

2: Probability, discrete random

variables; continuous random variables
and the normal distribution

Module

3: Sampling distributions;
estimation; hypothesis testing

Module

4: Simple linear regression

CHAPTER 10:
SIMPLE LINEAR REGRESSION
10.1 Simple linear regression
10.2
Standard deviation of errors
and coefficient of determination
10.3 Inferences about B
10.4 Linear correlation
10.5 Regression analysis: A complete
example
10.6 Interpretation of Excel output

A regression model is a mathematical equation

that describes the relationship between two or
more variables. A simple regression model
includes only two variables: one independent and
one dependent. The dependent variable is the one
being explained, and the independent variable is
the one used to explain the variation in the
dependent variable.
A (simple) regression model that gives a straightline relationship between two variables is called a
linear regression model.

Regression: describing the nature of relationship

between variables positive, negative, linear, or
nonlinear.
Correlation: determining whether a relationship
between variables exists
Questions: Are the two variables related? If so,
what is the strength? What kind of relationship?
What prediction can be made?
Examples: Height and weight of human, number
of cigarettes smoked vs weights of infants;
time spent on studying and exam marks.

Dependent variable (DV) (y, the one being

explained) vs. independent variable (IV) (x, used
to explain the variation).
Simple (only 1 IV) vs. multiple (> 1 IV)
regression
Linear (straight-line relationship) vs. nonlinear
regression

SIMPLE LINEAR REGRESSION ANALYSIS

In the regression model y = A + Bx + , A
is called the y-intercept or constant term, B
is the slope, and is the random error term.
The dependent and independent variables
are y and x, respectively.
In the model = a + bx, a and b, which are
calculated using sample data, are called the
estimates of A and B, respectively.

SCATTER PLOT/DIAGRAM

ERROR SUM OF SQUARE (SSE)

The error sum of squares, denoted SSE, is

SSE e 2 (y y )2
The values of a and b that give the minimum
SSE are called the least square estimates of A
and B, and the regression line obtained with
these estimates is called the least squares line.

Least square/best-fit line:

y a bx
SS xx x

SS yy y

SS xy xy
b

SS xy
SS xx

a y bx

n
x y
n

FORMULAS

Source: http://mathworld.wolfram.com/LeastSquaresFitting.html

Source: http://mathworld.wolfram.com/LeastSquaresFitting.html

Source: http://mathworld.wolfram.com/LeastSquaresFitting.html

Source: http://mathworld.wolfram.com/LeastSquaresFitting.html

Least square/best-fit line:

x 386 55.1429, y y 108 15.4286

n

SS xy xy
SS xx x
2

x y 6403 386 108 447.5714

n

(386) 2
23058
1772.8571
7

n
SS xy 447.5714
b

0.2525
SS xx 1772.8571

a y bx 15.4286 (0.2525)(55.1429) 1.5050

y a bx 1.5050 0.2525x

Least square/best-fit line (estimation and its

reliability):
b

SS xy
SS xx

447.5714
0.2525
1772.8571

a y bx 15.4286 (0.2525)(55.1429) 1.5050

y a bx 1.5050 0.2525 x
Estimate the amount of food expenditures when the income is $6100.
y a bx 1.5050 0.2525(61) $16.9075 hundred $1690.75
Error, e y y 16 16.9075 $0.9075 hundred $90.75
Estimate the amount of food expenditures when the income is $6000.
y a bx 1.5050 0.2525(60) $16.655 hundred $1665.50
The estimation is reliable because 60 (33,83)
Estimate the amount of food expenditures when the income is $2000.
y a bx 1.5050 0.2525(20) $6.555 hundred $655.50
The estimation is not reliable because 20 (33,83) *Extrapolation

ERROR OF PREDICTION

Least square/best-fit line (interpretation of

regression coefficients):
y a bx 1.5050 0.2525 x
y intercept, a 1.5050
A family with RM 0 income will
spend RM1.5050 hundred
=RM150.50 on food.
Slope coefficient, b 0.2525
For every one unit (RM100) of increment
in income, the expenditure on food will
increase by RM0.2525 hundred = RM25.25.

Least square/best-fit line (assumptions of

regression models):
1. Error has a mean of zero.
2. The errors are independent.
3. The distribution of error is normal.
4. The distribution of population errors has
the same (constant) standard deviation.

~ N ( 0, )
2

Degrees of Freedom for a Simple Linear

Regression Model
The degrees of freedom for a simple linear
regression model are
df = n 2

Standard deviation of errors:

is estimated by se
SSE
2
se
, where SSE ( y y )
n2
df n 2
se

SS yy bSS xy
n2

Standard deviation of errors:

SS xy
SS xx

447.5714
0.2525
1772.8571

SS xy xy
SS yy y
se

x y 6403 386 108 447.5714

n

SS yy bSS xy
n2

(108) 2
1792
125.1743
7
125.1743 (0.2525)(447.5714)
1.5939
72

Coefficient of determination (COD)

r
2

bSS xy
SS yy

,0 r 1
2

b 0.2525, SS xy 447.5714, SS yy 125.7143

r
2

bSS xy
SS yy

0.2525(447.5714)

0.899 89.9%
125.7143

Interpretation: 89.9% of the total variation in food expenditures

of household can be explained by the variation in incomes, and
the remaining 10.1% is due to randomness and other variables.

Coefficient of correlation (COC)

SS xy
SS xx SS yy

, 1 r 1

SS xx 1772.8571, SS xy 447.5714, SS yy 125.7143

r

SS xy
SS xx SS yy

447.5714

0.9481
1772.8571125.7143

Interpretation: Positive or negative sign/correlated.

Very weak, average/moderate, strong, very strong
r 0.9481: very strong and positively correlated
Other example:
r 0.1111: very weak and negatively correlated

Test statistic: tcalc

bB

, df n 2
sb

H0 : B 0
H1 : B 0 (two-tailed test)
B 0 (positive),B 0 (negative) (one-tailed test)
is unknown, use the t distribution.

HT about the slope coefficient, B

Test at the 1% significance level whether the
slope of the regression line is positive.
H 0 : B 0, H1 : B 0 (one-tailed test)

0.01
df n 2 7 2 5
tcalc

b B 0.2525 0

6.662
sb
0.0379

tcritical t ,n 2 t0.01,5 3.365

tcritical 3.365 tcalc 6.662
Reject H 0 . There is sufficient evidence to conclude
that the slope is positive, or, income determines
food expenditure positively.

A random sample of eight drivers selected from a small city

insured with a company and having similar minimum
required auto insurance policies was selected. The following
table lists their driving experiences (in years) and monthly
auto insurance premiums (in dollars).

Regression Analysis: A Complete Example

(a) IV and DV. Do you expect a positive or negative relationship?

(b) Compute SS xx , SS yy , and SS xy .
(c) Find the least square regression line.
(d) Interpret the regression coefficients in (c).
(e) Calculate the COC and COD. Interpret their meanings.
(f) Predict the monthly premium for a driver with 10 years of experience.
Comment on the reliability of the estimation.
(g) Compute the standard deviation of errors.
(h) Test at a 5% significance level whether B is negative.

Regression Analysis: A Complete Example

(a) IV: Driving experience, DV: Monthly auto insurance premium
A negative linear relationship.

Regression Analysis: A Complete Example

x 90
y 474

(b) x

11.25, y

59.25
n

SS xy

x y

(90)(474)

xy
4739
593.5

SS xx x

SS xy
SS xx

SS yy y 2

(c) b

y
n

(90) 2
1396
383.5
8
(474) 2
29, 642
1557.5
8

593.5
1.5476
383.5

a y bx 59.25 (1.5476)(11.25) 76.6605

y a bx 76.6605 1.5476 x

Regression Analysis: A Complete Example

(d) y a bx 76.6605 1.5476 x
y intercept, a 76.6605
A driver with 0 years of driving experience will need to pay
a monthly premium of $76.66.
Slope coefficient, b 1.5476
For every one extra year of driving experience, the monthyly
premium will decrease by $1.55.
(e) COC, r

SS xy
SS xx SS yy

593.5
0.7679
(383.5)(1557.5)

A moderately strong and negatively correlation.

r
2

bSS xy
SS yy

(1.5476)(593.5)
0.5897
1557.5

Alternative: COD,r 2 0.7679 0.5897

2

58.97% of the variation in monthly premium can be explained by

driving experience, whereas the remaining 41.03% is due to
randomness and other unaccounted factors.

Regression Analysis: A Complete Example

(f) y (10) 76.6605 1.5476(10) $61.18

The estimstion is reliable because 10 (2,25).

(g) se

SS yy bSS xy
n2

1557.5 (1.5476)(593.5)

10.3199
82

Regression Analysis: A Complete Example

(h) H 0 : B 0, H1 : B 0

0.05, df n 2 8 2 6
tcalc

b B 1.5476 0

2.937
sb
0.5270

tcritical t ,df t0.05,6 1.943

tcalc 2.937 tcritical 1.943
Reject H 0 . There is sufficient evidence to conclude that the slope is negative.

The hypothesis test on B can be

performed using the p-value approach,
using the output obtained from
statistical software.

EXCEL OUTPUT

Source: http://www.excel-easy.com/examples/regression.html

EXCEL

EXCEL

EXCEL

SUMMARY
Identify IV (x) and DV (y)
Calculate SS of xx, yy, and xy
Determine the best fit line
Calculate and interpret regression coefficients
Calculate and interpret COC and COD
Estimate and comment on its reliability
Hypothesis test on B (critical value approach
using manual calculation, or p-value
approach from the Excel output)
Finding missing values from the given Excel
output