1

PSYC 2101

Assessment & Personality

Course Co-ordinator: Suzanne Czech (sczech@psy.unsw.edu.au) Corse broken into 2 sections: - Assessment - %ersona#ity ( ks 1-! "u#ie $enry) ( ks !-11 &om 'enson( ks 11-1) Suzanne czech)

Assessment References
*ap#an + Saccuzzo Chapters 1( 2( ) + , -. /re0ory Chapters 1( 1( )

Overview of Assessment Section of Course

$istory( norms( re#iabi#ity( 2a#idity 3ethods o4 assessment &est bias Assessment o4 a0e-re#ated co0niti2e chan0e Assessment o4 emotion Assessment o4 chi#dren 5orensic app#ications 6europsycho#o0ica# assessment

The History of Psycholo ical Assessment

G The !irth of the "# Test
7 .udimentary 4orms o4 testin0 occurred as 4ar back as 2200 !$C$ (Chinese ci2i# ser2ice testin0) 7 1%&'( )un*t 5ounded 4irst psycho#o0ica# #aboratory in /ermany 89rass instruments era:: Sensory thresho#ds;.eaction &imes 7 1%'0( Cattell 5irst to introduce the term 8menta# test: 1< 8menta# tests: (a## o4 the 8brass instruments: approach) but there was #itt#e corre#ation b;w the menta# test scores and academic inte##i0ence.

G History of +sycholo ical testin

Prior to the mi* 1%th Century: -8'e2iations: attributed to demonic possession( mora# 4ai#in0s or crimina#ity. = -9ruta# treatment 7 As the 1'th century +ro resse*( - 3enta# i##ness re0arded as a 8disease o4 the mind: -'istinction made between menta# retardation (idiocy) and menta# i##ness (dementia) -'e2e#opmenta# 2s. abrupt onset ->ncurab#e 2s. curab#e ->ncreased humanism: 8Specia# education pro0ram: = ->mportant to identi4y 83enta# .etardation: as ear#y as possib#e 7 !inet ,1'0-. - 5irst 8modern: psycho#o0ica# test de2e#oped which predominant#y 4ocused on academic potentia# and know#ed0e 7 "ntelli ence tests in the /S 1'10( Terman re#eased the Stan4ord-9inet test (1) 5irst to use >?: IQ @ Chronological age (divided by) Mental Age A1<< (2) 6umber o4 items increased (1) C#ear( or0anised instructions ()) .epresentati2e standardisation samp#e (,) 3ean @ 1<<( S' @ 1, Standard o4 inte##i0ence testin0 4or decades B#timate#y superseded by the echs#er sca#es The emer ence of rou+ testin Yer1es or#d ar 1: 1.!, mi##ion army recruits in BSA 'e2e#oped Army a#pha and Army beta.= (1) Se0re0ate;e#iminate menta##y incompetent (2) C#assi4y accordin0 to menta# abi#ity (1) Assist in p#acement o4 competent men Serious prob#ems with 2a#idity (standardisation) $i0h C BS men c#assi4ied as 84eeb#e-minded:

= 7

Stan*ar*isation2 relia3ility an* vali*ity

G 7 7

Stan*ar*isation hen a test procedure is uni4orm 4rom one eAaminer and settin0 to another 'irections 4or administration in test manua#: %ro2ide comparab#e stimu#us materia#s to a## testers %recise#y speci4y the instructions 4or each component Ad2ise eAaminer how to hand#e wide ran0e o4 Dueries How *o we inter+ret test scores4

A psycho#o0ica# test must possess norms or standards 56orms -Summary o4 test resu#ts 4or a #ar0e( representati2e 0roup ->nterpretation is possib#e throu0h comparison with other:s scores -&hese other scores constitute 6-.3S -3ust be representati2e o4 the popu#ation 4or whom the measure is intended -Estab#ish a2era0e per4ormance as we## as the 4reDuency with which hi0h and #ow scores obtained -Can be outdated 7 Transformation of raw scores

.aw score data is o4ten 8trans4ormed: into a di44erent metric 3ost common trans4ormations: - %ercenti#es - Standard scores 7 Percentiles

-Score be#ow which a certain percenta0e o4 the members in the comparison 0roup come -,<th %ercenti#e @ median;mid-point -A#ways who#e numbers (1-1<<) -Can see eAact#y how 80ood: or 8bad: a particu#ar score is re#ati2e to norm 0roup 7 A*vanta es an* *isa*vanta es of +ercentiles

A*vanta es (1) Easi#y to compute (2) Easi#y to understand 7isa*vanta es (1) %ub#ic o4ten con4use with percenta0es FF G<th percenti#e 2s. G< percent (2) 6ot eDua# units o4 measurement 7 Stan*ar* Scores -9ased on mean and S'

-EDua# units o4 measurement -H-scores -$ow many S's a score is F or I 3 -3ean @ < (S' @ 1) - e can interpret H scores re#ati2e to the norma# distribution cur2e

Relia3ility

-&he de0ree o4 consistency in measurement. 3easured between < + 1. ->mportant in situations such as: 'ia0nostic decisions Courtroom testimonyJ 7 7 /nrelia3ility8s affect on test scores C#assica# test theory:

-Each person has a true score (&) that wou#d be obtained i4 there were no errors in measurement -Any obtained score (-) re4#ects in4#uence o4: (1) 5actors that contribute to consistency (&) (2) Error (Kunk( noise): (E) -@&LE -3ost measurement errors are random MF= reduces re#iabi#ity 7 9liminatin measurement error:

->t:s impossib#e to e#iminate it a## in psycho#o0y - e can e#iminate some by: i) &est Administration- try to reduce en2ironmenta# a44ects (4ati0ue etc) + eAaminer e44ects (subconscious appro2a# si0ns e0. 6od) ii) &est scorin0- make it so that the scorin0 o4 a test is as unsubKecti2e as possib#e. iii) Systematic measurement error -&est administration and test scorin0 may introduce unsystematic error ->t is when a test consistent#y measures somethin0 other than it was supposed to e.0. Some depression sca#es a#so measure somatic comp#aint (chan0es in #ibido( wei0ht( s#eep patterns) = -$ospita# AnAiety 'epression Sca#e( /eriatric 'epression Sca#e -9y de4inition initia##y undetected

Ty+es of relia3ility

,1. Test5retest relia3ility

-$i0her r @ better re#iabi#ity -EApect hi0her scores on re-testin0N -%ractice e44ects (abi#ity( achie2ement( manua# deAterity) -So#ution M #on0er time inter2a#N --ther 4actors may inter2ene e.0. chi#dren tested at a0es ) and , may chan0e on characteristic ,2. Alternate forms relia3ility -(%ara##e#;eDui2a#ent 4orms) -Simi#ar content: Same ran0e;#e2e# o4 di44icu#ty in items Simi#arities: -&wo test administrations to the same participants -.e#iabi#ity reduced by inter2enin0 chan0es 'i44erences ->tem-samp#in0 di44erences: additiona# source o4 error 2ariance -EDua# di44icu#ty on a2era0e( but para##e# items may not be eDua##y 4ami#iar to e2ery person e.0. 2ocabu#ary test -A#so di44er in eApense ,:. ;easures of internal consistency -Sp#it-ha#4 re#iabi#ity -Cronbach:s coe44icient a#pha ->nterscorer re#iabi#ity 7 S+lit5half relia3ility

Ad2anta0es: -6ot a#ways possib#e to obtain a second set o4 test scores -&est-retest data potentia##y mis#eadin0: -%ractice e44ects #ar0e and 2ariab#e -&rait 4#uctuates rapid#y( e.0. positi2e a44ect 'isad2anta0es: -8EDui2a#ent: ha#2esN -So#utionN -Cronbach (1G,1): Coe44icient a#pha -3ean o4 a## possib#e sp#it-ha#4 coe44icients 7 "nterscorer relia3ility -Some tests reDuire Kud0ment in scorin0 e.0. Assessment o4 theses -Scores 4or pairs o4 eAaminers are corre#ated

How relia3le is relia3le4

-'ependent on purpose ->mportant indi2idua# decisions: .G, -Combined with other in4ormation: .O, -5or basic research: .P< minimum 7 "m+rovin relia3ility

(a) >ncrease number o4 items e.0. medica# dia0nosis -Can use 8prophecy: 4ormu#a to 84orecast: re#iabi#ity (b) 'iscriminabi#ity ana#ysis EAc#ude items that corre#ate weak#y with tota# score 3i0ht be: 3easurin0 somethin0 di44erent 4rom the other items SubKect to 4#oor;cei#in0 e44ects 7 Test <ali*ity

-Qa#idity de4ines the meanin0 o4 test scores -A%A workin0 party de4inition: RA test is 2a#id to the eAtent that in4erences made 4rom it are appropriate( meanin04u# and use4u#.S ->n2a#idity: >n4erences drawn are not Kusti4ied 7 7ifferent ty+es of vali*ity(

(1) Criterion (2) Content (1) Construct 1$ Criterion vali*ity: -'o scores predict re#e2ant outcome measureN e.0. Academic achie2ementN &rainin0;Kob per4ormanceN &eacher ratin0sN -ther standardised testsN -utcome measure @ criterion

2 Ty+es of Criterion <ali*ity( ,1a. 7ia nostic=concurrent vali*ity -EAtent to which test scores accurate#y estimate present position on re#e2ant criterion e.0.( c#inician-rated depression as 2a#idation 4or se#4-rated depression sca#e

,13. Pro nostic=+re*ictive vali*ity -EAtent to which test scores accurate#y estimate 4uture position on re#e2ant criterion e.0. schoo# scores and Kob achie2ement ->n2o#2es deri2in0 a re0ression eDuation 2$ Content <ali*ity ->tems represent a 4air samp#e o4 the tota# potentia# content -'oes test samp#e correct domainN -*ap#an and Saccuzzo: R$ow many times ha2e you studied 4or an eAamination and known a#most e2erythin0( on#y to 4ind that the pro4essor has come up with some stran0e items that do not represent the content o4 the courseN >4 this has happened( you may ha2e encountered a test with poor content-re#ated e2idence 4or 2a#idity.S (p. 111). -Content 2a#idity e2idenceN :$ Construct <ali*ity - hat does a test actua##y measureN -Teadership abi#ity( hosti#ity( depression( and inte##i0ence ->n4erred 4rom beha2ior but more than the beha2ior itse#4

Assessment of construct vali*ity

(a) Con2er0ent 2a#idity e.0. measures o4 2erba# inte##i0ence shou#d corre#ate hi0h#y with other measures o4 2erba# inte##i0ence (b) 'iscriminant 2a#idity e.0. 2erba# inte##i0ence and cookin0 ski##s (c) 5actor ana#ysis Assess pattern o4 intercorre#ations amon0 2ariab#es

PSYC2101 Psycholo ical Assessment Test 3ias Overview >s there bias in co0niti2e assessmentN

Ar0uments 4or and a0ainst 5airness 2s. bias Can bias be obKecti2e#y measuredN M Content 2a#idity M Construct 2a#idity M %redicti2e 2a#idity !ias in co nitive assessment &est bias contro2ersy ori0ins: Consistent di44erence between minority;maKority 0roups M (o2er#ap is #ar0er than the di44erences) >nte##i0ence tests are used 4or se#ection purposes Consistent underper4ormance wi## restrict access 'eser2ed#y a topic o4 intense scrutiny )hat is meant 3y test 3ias4 Qa#idity: RA test is 2a#id to the eAtent that in4erences made 4rom it are appropriate( meanin04u# and use4u#S (A%A( 1GGG) &est bias re4ers to di44erentia# 2a#idity Can be eAamined obKecti2e#y 'oes an absence o4 bias imp#y a test is 4airN &est 4airness re4#ects socia# 2a#ues in test usa0e 'oes it seem 4air that a test is used in a certain wayN )hy test 3ias is controversial /roup di44erences in >? score: 7o rou+ *ifferences +rovi*e evi*ence of 3ias4 TaypersonsN CTA>3S -5 9>AS M &ests re4#ect the dominant cu#ture and there4ore members o4 that cu#ture do better CTA>3S A/A>6S& 9>AS M &he tests can be shown to be eDua##y 2a#id 4or minority and maKority 0roups Three main criteria for 3ias (1) Content 2a#idity (2) Criterion-re#ated 2a#idity (1) Construct 2a#idity ,1. 9vi*ence for content vali*ity 3ias hen a test is re#ati2e#y more di44icu#t 4or

members o4 one 0roup than another (a4ter 0enera# abi#ity is he#d constant) (a) >n4ormation not eDua##y a2ai#ab#eN (b) Scorin0 biasedN (c) ordin0 o4 Duestion un4ami#iarN 3ost inte##i0ence tests: $i0h#y #an0ua0e dependent Assess cu#tura##y-acDuired know#ed0e ;easurin !ias in Content <ali*ity A>S-. ho was Sir 'ona#d 9radmanN ho wrote 5austN >SC-. hat shou#d you do i4 a youn0er chi#d hits youN Are "# tests >culture s+ecific8 tests4 R9#ack >nte##i0ence &est o4 Cu#tura# $omo0eneityS .unnin0 a 0ame means: A) writin0 a bad cheDue 9) #ookin0 at somethin0 C) 'irectin0 a contest ') /ettin0 what one wants An RoreoS is: A) an inte##ectua# 9) an Bnc#e &om C) a cookie ') a white #ibera# .e#iabi#ity and 2a#idityN So intelli ence tests are 3iase* in content vali*ity4 3c/urk (1G,1) C#assi4ied standardised >? test items as: Teast cu#tura#;6eutra#;3ost cu#tura# Rmost cu#tura#S .1< R#east cu#tura#S .,O Attempts to Rpuri4yS testsN Conc#usionN (>4 4ami#iar with #an0ua0e and education system) ,2. Criterion5relate* vali*ity 'oes the test #ead to biased decisionsN e.0. schoo# scores and Kob achie2ement Bnbiased test: %redict per4ormance eDua##y we## 4or indi2idua#s 4rom di44erent subpopu#ations

hat do we mean by predictionN Bsin0 scores on one measure to predict scores on another hen tests used 4or predictionN 'eri2e re0ression eDuation ,:. Construct <ali*ity -ra# arithmetic test: M >tem 1: R>4 you had P Qe0emite sandwiches- and took two bites 4rom each( how many bites did you take a#to0etherNS 5or an Austra#ian chi#d M test o4 arithmetic 5or Qietnemese chi#d M test o4 En0#ish ays to measure bias in Construct Qa#idity (a) Construct EAp#ication (b) .ank orderin0 item di44icu#ty (c) 5actor ana#ysis ,a. Construct 9?+lication Took at re#ationships with other measures ,3. Ran1 or*erin item *ifficulty Are the items that are most (or #east di44icu#t) 4or one sub0roup a#so the most (or #east) di44icu#t 4or the other sub0roupN 'o they need to obtain eDui2a#ent passin0 ratesN Compute C passin0 4or each item in each subpopu#ation to 0au0e task di44icu#ty "tem 3y rou+ interactions hat is the capita# o4 &hai#andN hat is the popu#ation o4 Austra#iaN here is estminster AbbeyN ,c. @actor analysis 6eed to demonstrate 4actoria# in2ariance

Conclusions4 (1) Content 2a#idity Attempts to 8puri4y: measures not success4u# (2) Criterion-re#ated 2a#idity Separate( para##e#( re0ression #ines needed (1) Construct 2a#idity >tem di44icu#ties preser2ed in di44erent 0roups. 5actor ana#ysis resu#ts -2era##( e2idenceN

9?+lainin the *ifferences &est biasN 6ot supported 3inor ro#e( i4 any /eneticsN En2ironmenta# in4#uencesN Role of environmental factors un*erestimate* in early research( 9rooks-/unn( *#ebano2 and 'uncan (1GGP) Compared A4rican American and white ,-yr-o#ds on %%S> 5o##owed 4rom birth: 6ei0hborhood and 4ami#y po2erty( 4ami#y structure( 4ami#y resources( materna# characteristics( home en2ironment A4rican American chi#dren:s >? scores I 1 S' RAdKustments 4or economic and socia# di44erences . . . a## but e#iminate di44erences in the >? scores between these two 0roups.S