1

Multiple Regression (Part 1) (DR SEE KIN HAI)

1. Standard Multiple Regresion 2. Stepwise Multiple Regression 3. Hierarchical Multiple Regression

1. This is an extension from the bivariate correlation ( 2 variables correlation) 2. Regression analysis is used when several independent variables (continuous or categorical) are correlated with themselves and with the dependent variable (continuous scale). If dept var is not continuous use Discriminant Function Analysis). 3. Three Regression Models: (a) Standard/ Simultaneous Regression = all indep vars enter at once to see the relationship btw whole set of predictors and dep var. (b) Hierarchical Multiple Regression = you determine which indep vars to enter first based on your theoretical knowledge. (c) Stepwise Multiple Regression = No of indep vars entered and the order of entry (forward= enter predictor one at a time accepted if F ratio > Critical value (FIN) and critical alpha level (PIN) determined by statistical criteria. Backward selection = with all variables in equation and poor performers deleted if F value < critical value (FOUT). Stepwise uses combination of forward and backward procedures). 4. You will practice on the use of the 3 types of regressions as stated above.
Independent Variables

1. Standard (Simultaneous) Multiple Regression Analysis (Practice)

A school uses data from 15 students below to examine the relationship between the English Language scores, Motivation of students and the Attitude towards English Language. Using the data shown, answer the research questions below.
Dependent Variable

1. What contribution do both motivation of students and the attitude towards English language make to the prediction of the academic score in English? (Use Standard Multiple Regression) 2. Which is the best predictor of the academic score in English? (Use Stepwise Multiple Regression) 3. Previous research has suggested that attitude towards English is the salient (most important) predictor of the academic score in English. Is this hypothesis correct? (Use Hierarchical Multiple Regression) How to carry out this Standard (Simultaneous) Multiple Regression Analysis? 1. Enter the data as shown below into SPSS 20. 2. Click on [Analyze] then [Regression] then [Linear] to open the dialogue box

FIG 1

Select [English] and enter into the [Dependent] Box

Select [Motivation] and [Attitude] and enter into the [Independent] box

For [Method] select [Enter] Select [Statistics] to open [Linear Regression] subdialogue Box In the previous screen, click on [Plots..] to open [Linear Regression Plots] sub-dialogue box

Select [Estimates] , [R sq change] and [Model fit] then click on [Continue]

3
Select [ZRESID] and click to [Y] and [ZPRED] and enter into [X] box In the [Std Residual Plot] select [Normal Prob plot] then [Produce all partial plots] and [Continue]. Click [Save] from FIG 1 to open [Linear Regression Save New Variables] sub-dialogue box

In the [Distances] box select [Mahalanobis] then [Continue] and [OK]

Both independent variables as predictors together explain 87.8% of the variance in [English] and is highly sig as indicated by F value (43.1) at sig p = 0.000

An examination of T values indicates that [motivation] adds to the predictor power in the equation p = 0.000 and [Attitude] is not sig at p= 0.126

This[ MAH_1] is added to your data file with [Mahal Distance] small values and there is no multivariate outliers among the ind variables as no values are greater or equal to critical

2 of 13.8 at

p = 0.001 level

From scatterplot of [Residuals] Vs [Predicted values] it does not show any clear relationship. The [Normal plot] of Regression standardized residuals for Dep variable indicates a relatively Normal distribution

Highly correlated

Even we partial out the other Independent variables (Motivation or Attitude) we can see that [Motivation] is quite highly correlated with the Dependent variable [English scores] but not [Attitude] Conclusion for Research Question 1:
No correlation

We can say that [Motivation] significantly predicts the [English language scores] of the students. However the number of Attitude towards English is not a significant predictor.

2. Stepwise Multiple Regression Analysis

1. [Analyze] then [Regression] then [Linear .] to open the dialogue box 2. Enter [English score] into the [Dependent] box [Motivation] and [Attitude] into the [Independent] box. 3. For the [Method] select [Stepwise] then click on [Statistics] to open [Linear Reg Stat] subdialogue box. 4. Select[Estimates], [Model fit] and [R sq change] then [Continue] and [OK].

7 To Plot Charts (You will get the same charts as Standard Regression) 1. Click on [Plots..] to open the [Linear Regression Plots] sub-dialogue box.

Interpreting the Output Here each new step is called a Model. Model 1 uses highest correlated predictor [Motivation] and Model 2 uses [Motivation] and lesser correlated second predictor [Attitude]. Generally you concentrate on the Model that has the highest correlation. Table below is badly shown with highest correlation entered first then follow by second highest correlated predictor and the rest of smaller correlated predictor if any.

No value added below 0.850 for [Attitude]

This Table shows predictor [Motivation] was entered first in stepwise analysis (Model 1) having the highest correlation with Dep variable [English] and is significant at p = 0.000. Multiple R= correlation between [Motivation] and [English] = 0.922. R square = square of multiple correlation coefficient = 0.85 indicating 85% of the variance in the criterion is shared/ explained by this predictor [Motivation]. Adjusted R-square =R-square adjusted for the size of the sample and the number of predictors in the equation. { =1.977 + 1.057

y a b1 x1 b2 x2
and

x1

+ 0 x 2 where y=estimated English scores,

x1 =Motivation, b1 =Regression coefficient for x1

x2 = Attitude, b2 =

Regression coeff for

contribute to significant proportion of Dep variable [English] so no Model 2. R-square change = increase in the proportion of the variance in [English] by predictor 2 if any (not in this case for [Attitude]) Beta for Model 1 = 0.922= Muliple R = standardized regression coefficient. To get output fast ignore [constant] see the next row/s here [Motivation] sig predictor of [English] at p=0.000

x2 } Effect is to reduce size of R square to Interpreting the output

0.839 from 0.850. Here analysis stop as the second predictor does not

B = +ve means positive high correlation of [Motivation] and [English] = 0.922

T value for [Motivation] is sig at p = 0.000 and [Attitude] is not sig

9 Reporting the output In this stepwise multiple regression analysis, [Motivation] was entered first and explained for 85% of the variance in [Academic Score in English ] for the students with F1,13 73.84, p < 0.001 . Table included as below: (Optional) Table 1.0 Stepwise multiple regression of predictors of academic scores in English (only significant predictors are included) Variable Motivation Multiple R 0.922 B 1.057 Standard error b 0.123 Beta t Significance of t 0.000

0.922 8.563

3. Hierarchical Multiple Regression and Coursework for the 3 types of Regressions (See Next Lecture)