JOURNAL OF PERSONALITY ASSESSMENT, 76(1), 150168

Copyright 2001, Lawrence Erlbaum Associates, Inc.

Predicting DSMIV Cluster B

Personality Disorder Criteria From
MMPI2 and Rorschach Data: A
Test of Incremental Validity
Mark A. Blais
Inpatient Psychiatry Service
Massachusetts General Hospital and
Harvard Medical School

Mark J. Hilsenroth
Department of Psychology
University of Arkansas

Frank Castlebury
Raleigh Psychiatric Associates

J. Christopher Fowler
Erik H. Erikson Institute for Training and Research
Austen Riggs Center
Stockbridge, Massachusetts

Matthew R. Baity
Department of Psychology
University of Arkansas

Despite their frequent conjoint clinical use, the incremental validity of Rorschach
(Rorschach, 1921/1942) and MMPI (Hathaway & McKinley, 1943) data has not been
adequately established, nor has any study to date explored the incremental validity of
these tests for predicting Diagnostic and Statistical Manual of Mental Disorders (4th
ed. [DSMIV]; American Psychiatric Association, 1994) personality disorders (PDs).
In a reanalysis of existing data, we used select Rorschach variables and the MMPI PD
scales to predict DSMIV antisocial, borderline, histrionic, and narcissistic PD criteria in a sample of treatment-seeking outpatients. The correlational findings revealed a

INCREMENTAL VALIDITY

151

limited relation between Rorschach and MMPI2 (Butcher, Dahlstrom, Graham,

Tellegen, & Kaemmer, 1989) variables, with only 5 of 30 correlations reaching significance (p < .05). Hierarchical regression analyses showed that both the MMPI and
Rorschach data add incrementally in the prediction of DSMIV borderline and narcissistic PD total criteria scores. The findings were less clear for the incremental value of
Rorschach and MMPI2 data in predicting the total number of DSMIV histrionic PD
criteria, which were best predicted by Rorschach data, and antisocial PD criteria,
which were best predicted by MMPI2 data. In addition to providing evidence of the
incremental validity of Rorschach data, these findings also shed light on the psychological characteristics of the DSMIV Cluster B PDs.

The versions of the Minnesota Multiphasic Personality Inventory (MMPI,

Hathaway & McKinley, 1943; MMPI2, Butcher, Dahlstrom, Graham, Tellegen,
& Kaemmer, 1989) and the Rorschach (Rorschach, 1921/1942) continue to be
among the most popular psychological instruments for both clinicians and researchers interested in measuring personality (Lubin, Larsen, & Matarazzo, 1984;
Lubin, Wallis, & Paine, 1971; Piotrowski & Keller, 1984, 1989; Piotrowski,
Sherry, & Keller, 1985; Watkins, Campbell, Nieberding, & Hallmark, 1995). Despite their ongoing clinical popularity and the prodigious number of published
studies using these instruments (estimated to be more than 16,000; Ganellen,
1996a, 1996b), important questions remain regarding the interrelation between the
Rorschach and the MMPI2 (Archer & Krishnamurthy, 1997; Ganellen, 1996a,
1996b; Meyer, 1993, 1999). Furthermore, the utility of combining data from these
tests to predict relevant non-test-based criterion variables, such as treatment outcome or diagnoses, remains unclear.
Archer and his colleagues (Archer, 1996; Archer & Gordon, 1988; Archer &
Krishnamurthy, 1993a, 1993b, 1997; Krishnamurthy, Archer, & House, 1996)
have both reviewed the existing literature and conducted a series of studies exploring the interrelation between the Rorschach and the MMPI and MMPI2. In their
review of the 37 Rorschach and MMPI studies that existed at that time, Archer and
Krishnamurthy (1993b) found that 51% of the studies reported nonsignificant
findings and an additional 22% yielded only weak associations across the two
tests. For the remaining studies (27%) that reported significant findings, the reported correlations tended to be modest (rs = .24.34), and the positive findings
were typically limited to a small subgroup of the total variables studied. The results of this review and his own empirical studies have persuaded Archer to
conclude that conceptually similar MMPI scales and Rorschach variables are essentially unrelated (Archer & Krishnamurthy, 1997).
Meyer (1993, 1996, 1997, 1999) added to our understanding of Archers findings by correctly pointing out that correlating Rorschach and MMPI data represents a form of cross-method correlation and that such correlations are typically
modest in nature. In addition, Meyer developed a more sophisticated methodology
for investigating Rorschach and MMPI relations. Using this new methodology,

152

BLAIS ET AL.

Meyer (1993, 1997) demonstrated that when Rorschach and MMPI data are
grouped according to response style (so that only respondents with similar response styles on both tests are studied), strong cross-test associations are found.
Meyer (1999) recently replicated his earlier work, again showing strong cross-test
association but this time using new response-style grouping criteria. The results
from this series of studies suggests that the convergence of MMPI and Rorschach
constructs is tied to broad stylistic qualities related to how patients interact with
both assessment methods (Meyer, 1999, p. 32). Although Meyers work represents a promising evolution in this area of research, his findings require independent replication (Archer, 1996; Archer & Krishnamurthy, 1999), and their clinical
application also needs to be explored. The absence of clearly demonstrated relations between Rorschach and MMPI data thought to be measuring conceptually
similar constructs does not mean that these data cannot be combined to increase
their predictive validity. It is well established that uncorrelated but valid measures
combine to provide the best predictors of a criterion variable (Meehl & Hathaway,
1946).
Incremental validity has long been recognized as a property necessary for a
clinical diagnostic instrument to possess (Sechrest, 1963). However, the question of whether Rorschach and MMPI data can be combined to improve the prediction of a desired outcome variable has been infrequently studied (Archer &
Krishnamurthy, 1997; Ganellen, 1996a, 1996b; Meyer, 1996). This is a puzzling
omission in the research literature, one that leaves open questions about the true
clinical validity of these commonly conjointly used psychological tests (Acklin,
1993). Two incremental validity studies conducted by Archer and Gordon
(1988) and Archer and Krishnamurthy (1997) found little evidence of incremental validity for combined Rorschach and MMPI scales in accurately classifying
adolescents as either depressed or schizophrenic. For example, Archer and
Gordon found that both the Rorschach Schizophrenia Index and MMPI Scale 8
were somewhat successful in identifying adolescents with a clinical diagnosis of
schizophrenia (hit rates of .80 and .76, respectively); however, the combined use
of these two indexes did not significantly improve the overall classification rate.
In a follow-up study, Archer and Krishnamurthy (1997) obtained highly similar
findings showing limited incremental validity for the combined use of MMPI
and Rorschach data in classifying either conduct-disordered or depressed adolescents. The limited empirical data supporting the combined use of the Rorschach
and MMPI were highlighted by Archer (1996) and Wood, Nezworski, and
Stejskal (1997) as a significant weakness in the scientific foundation of the
Rorschach.
The purpose of this study was to add to the existing incremental validity literature
by exploring the ability of Rorschach and the MMPI to predict the criteria for the Diagnostic and Statistical Manual of Mental Disorders (4th ed. [DSMIV]; American
Psychiatric Association, 1994) personality disorders (PDs). Specifically, we used

INCREMENTAL VALIDITY

153

multiple-regression analyses to explore the ability of select Rorschach variables and

the MMPI2 PD scales (Colligan, Morey, & Offord, 1994) to predict ratings of
DSMIV Cluster B PD (antisocial personality disorder [ANPD], borderline personality disorder [BPD], histrionic personality disorder [HPD], and narcissistic personality disorder [NPD]) criteria in a sample of treatment-seeking outpatients with PDs.
The Rorschach variables were selected for use in this study on the basis of prior research demonstrating their conceptual or empirical relation to one of the Cluster B
PDs. Portions of the data presented in this article were previously reported in a series
of Rorschach and MMPI studies (see Baity & Hilsenroth, 1999; Blais, Hilsenroth, &
Fowler, 1998; Blais, Hilsenroth, Fowler, & Conboy, 1999; Castlebury, Hilsenroth,
Handler, & Durham, 1997; Hilsenroth, Fowler, Padawer, & Handler, 1997); however, all of the multiple-regression analyses performed for this study and the majority of the correlations reported in this article are original to this study.1 This study was
distinctive in that it represents the first effort to empirically establish the incremental
validity of Rorschach and MMPI2 data for predicting the DSMIV Cluster B PD
criteria.

METHOD
Patients
The data used in this study were drawn from an archival search of patients files at a
university-based outpatient psychology clinic.2 This review included approximately 800 case files covering a 7-year period. A number of previous studies have
successfully used chart information to retrospectively rate patients for Axis II diagnostic criteria (Fyer, Frances, Sullivan, Hurt, & Clarkin, 1988; McCann, 1991;
Morey, 1985). The selection of cases proceeded in three phases. In the first phase,
217 patients were identified as having been diagnosed with a PD. A team consisting
of an advanced clinical psychology doctoral student and a supervising licensed
clinical psychologist rendered the original clinical diagnoses on completion of the
intake assessment procedure of the clinic.
In the second phase of selection, the chart material for these 217 patients was
rated for the presence or absence of a DSMIV PD diagnosis. The presence or absence of a PD was determined through a retrospective review of the patient records, which included an evaluation report, session notes from the first 12 weeks
of therapy, and 3-month treatment reviews when available. Information regarding
1Correlations between HPDOverlapping (O), HPDNonoverlapping (N), DEN, and T, for N = 62
were reported in Blais et al. (1998). Also, correlations between NPDO, NPDNO, REF, and IDEAL,
for N = 62 were reported in Hilsenroth et al. (1997).
2 For a more complete review of the sample selection methodology, see Hilsenroth et al. (1997).

154

BLAIS ET AL.

patient identity, diagnosis, and test data (including all Rorschach and MMPI2
data) were appropriately masked from the reviewers. Raters in this phase of the
study were four advanced doctoral students in an American Psychological Association-approved clinical psychology program. The raters had received special training in the diagnosis of DSMIV Axis II disorders. Interrater reliability was
established through independent ratings of a randomly selected pool of 31 patients; the obtained kappa value was .90 for the presence or absence of a DSMIV
PD. Of the 217 patient case files reviewed in this manner, 57 were found to meet
DSMIV criteria for an Axis II disorder as well as to contain a completed Rorschach and MMPI2 protocol. The Axis II diagnoses of these 57 patients had the
following distribution: ANPD = 11, BPD = 18, HPD = 5, NPD = 10, Cluster A PD
= 6, and Cluster C PD = 7.
The sample (N = 57) was 53% female and had a mean age of 28 years (SD = 8)
at the time of their clinic intake. They averaged 14 years of education (SD = 2.5),
and their mean Wechsler (1981) full-scale IQ was 106 (SD = 11). Thirty-three patients were single, 8 were married, 15 had been divorced, and 1 was widowed. In
the third phase, the records of these 57 patients were again independently rated on
all of the DSMIV Cluster B PD symptom criteria (ANPD, BPD, HPD, and NPD)
with the same case material and methodology outlined earlier. Again, interrater reliability was established by independent ratings of a randomly selected pool of 25
patients. Interrater agreement for the presence or absence of each individual
DSMIV Cluster B symptom criterion were as follows: .86 (ANPD), .80 (BPD),
.90 (HPD), and .90 (NPD).

Procedure
The Rorschach was originally administered and scored according to the procedures of Exner (1986, 1993). All Rorschach protocols were rescored by Mark J.
Hilsenroth, who was blind to previous scores and patient diagnoses. Interrater
reliability (Weiner, 1991) was obtained by having J. Christopher Fowler, who
was blind to the first coders scores and patients diagnoses, score 20 randomly
selected protocols. The interrater agreement for the Structural Summary Rorschach variables and the Rorschach content scales (described later) were above
80%. All Rorschach protocols used in this study were reviewed for validity;
none were found to have fewer than 14 responses and a Lambda above 1.0.
The MMPI2 (Butcher et al., 1989) was administered according to standard instructions. All MMPI2 protocols were reviewed for validity (see Castlebury et
al., 1997, for details). The MMPI2 Cluster B PD (ANPD, BPD, HPD, NPD)
scales were scored and included as variables in this study (Colligan et al., 1994).
Originally developed by Morey, Waugh, and Blashfield (1985) to assess the
DSMIII PDs, the MMPI PD scales have frequently been used in personality re-

INCREMENTAL VALIDITY

155

search (Castlebury et al., 1998; Dubro, Wetzler, & Kahn, 1988; McCann, 1989,
1991; Morey, 1986; OMaille & Fine, 1995; Trull, 1993; Wise, 1996) and represent the only PD scales currently available for the MMPI. Two versions of the
MMPI PD scales, one containing O (overlapping) items and the other having no O
items, were developed by Morey et al. The O version of the PD scales allows
MMPI items that are relevant to multiple DSM PDs to be scored on multiple PD
scales (the scales have O items). The NO version of the MMPI PD scales focus on
the core or defining features of the DSM PDs and restricts item scoring to a single
scale (these scales have no O items). Across a number of research studies, the
MMPI PD scales have demonstrated adequate reliability and validity (see
OMaille & Fine, 1995, for a review of these data). The impact of the MMPI
restandardization and item revision on the PD scales was explored by Colligan et
al. While also presenting updated norms for the PD scales, Colligan et al. found the
MMPI2 version of the PD scales to be comparable to the original scales. Wise
(1996) also found the MMPI2 PD scales to be comparable to the original scales.
Both the O and NO versions of the MMPI2 PD scales were included in the
analyses.

Rorschach Variables
All Rorschach variables used in this study were selected, a priori, on the basis of
either prior theoretical or empirical linkage to DSMIV Cluster B PD criteria. Although the Rorschach variables used in this study are briefly reviewed next, all relevant citations are provided, allowing the interested reader to explore the rationale
for their selection in greater detail.

ANPD. The Rorschach Comprehensive System (CS; Exner, 1986, 1993)

variables T, PER, MOR, and AG, along with Holts (1977) primary process aggression variable (A1) and Gacono and Meloys (1991) measure of Aggressive Content
(AgC) have all been conceptually and empirically associated with DSMIV ANPD.
(See Baity & Hilsenroth, 1999, and Gacono & Meloy, 1994, for interrater reliability
and a more complete explanation of the variables, including their scoring.)

BPD. Rorschach measures of aggression, including Holts (1977) A1 variable, the Rorschach Oral Dependency (ROD) scale (Masling, Rabie, & Blondheim,
1967), quality of object relations using the Mutuality of Autonomy Scale (MOAS;
Urist, 1977), and the Lerner Defense Scale (LDS; Lerner, 1991) defenses of splitting (SPLIT), and devaluation (DEVAL) have all been conceptually and empirically
linked to the DSMIV BPD. In this study, the MOASHighest score (MOASH) was

156

BLAIS ET AL.

used, which is the single highest or most disturbed MOAS score from a given protocol. (See Blais et al., 1999, for interrater reliability and a more complete explanation of the variables, including their scoring.)

HPD. The Rorschach CS variables (Exner, 1986, 1993) FC + CF + C, T, and

the ROD scale (Masling et al., 1967), have all been conceptually and empirically
linked to DSMIV HPD. (See Blais et al., 1998, for interrater reliabilities and a
fuller explanation of the rationale underlying the selection of these variables.)

NPD. The Rorschach CS (Exner, 1986, 1993) REF variable and the LDS
(Lerner, 1991) variable of Idealization (IDEAL) have been conceptually and empirically associated with DSMIV NPD. (See Hilsenroth et al., 1997, for interrater reliability and a more complete explanation of the variables, including their scoring.)

Statistical Analyses
Before conducting our analyses, we correlated the Rorschach variables with the total number of Rorschach responses (R). For any Rorschach variable that showed a
trend toward significant (p = .10) correlation (T, ROD, REF, and AgC), the effect or
R was partialed out in all reported analyses. Although multivariate analyses have
been shown to be robust to violations normality (Stevens, 1996), following
Vigliones (1997) recommendation we reviewed the descriptive data for each of
the Rorschach variables to determine the nature of their distributions. Curran,
West, and Finch (1996) suggested that distributions with a skew of 2.0 or more or
kurtosis of 7.0 or more should be considered moderately nonnormal in shape. Applying these criteria, we found three Rorschach variables, SPLIT, REF, and MOR,
to be sufficiently nonnormal in their distribution to potentially effect the
multivariate analyses. We subjected these three variables to a log transformation in
an attempt to improve their distributions. The distributions for two of the variables
(MOR and REF) were successfully transformed, although the distribution for one
variable (SPLIT) was not be sufficiently improved by this process and was dropped
from the study. Table 1 presents the descriptive data for all retained Rorschach variables. Following these transformations, correlations (Pearson rs) were obtained for
the Rorschach variables and the appropriate MMPI2 O and NO PD scales. Next,
we summed the Cluster B (ANPD, BPD, HPD, NPD) symptom ratings to produce
total scores (equaling the total number of criteria met) for each of the Cluster B PDs.
This procedure produced a dimensional representation of each of the four Cluster B
PDs. Substantial empirical (see Blais & Norman, 1997) and theoretical (see
Widiger, 1991) support exists for the use of dimensional PD scores.

157

INCREMENTAL VALIDITY
TABLE 1
Descriptive Data for the Rorschach Variables Used in the Multivariate Analyses
Rorschach Variable
Ta
RODa
REF a,b
A1
AG
MORb
PER
AgC a
IDEAL
FC + CF + C
MOASH
DEVAL

SD

MIN

MAX

Skew

Kurtosis

0.05
0.16
0.12
0.51
0.91
0.35
1.80
2.75
3.50
4.03
4.84
7.17

0.06
0.11
0.21
1.04
1.15
0.28
2.10
1.81
3.73
2.90
1.91
6.47

0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.0
0.00

0.25
0.42
0.78
3.0
4.0
1.08
9.0
8.0
14.0
12.0
7.0
23.0

1.28
0.31
1.60
1.14
0.97
0.27
1.41
0.66
0.87
1.20
0.68
0.87

1.45
0.54
1.36
0.69
0.30
0.79
1.74
0.31
0.25
1.03
1.04
0.06

Note. N = 57. MIN = minimum; MAX = maximum; Rorschach variables: T = Texture (Comprehensive
System [CS]; Exner, 1986, 1993); ROD = Rorschach Oral Dependency scale (Masling et al., 1967); REF
= Reflections (CS); A1 = Primary Process Aggression (Holt, 1977); AG = Aggressive Movement (CS);
MOR = Morbid (CS); PER = Personalized (CS); AgC = Aggressive Content (Gacono & Meloy, 1994);
IDEAL = Idealization (Lerner Defense Scale [LDS]; Lerner, 1991); FC + CF + C = Form Color + Color
Form + Color (CS); MOASH = Mutuality of Autonomy scale single highest or most disturbed score
(Urist, 1977); DEVAL = Devaluation (LDS).
aThe effect of R was partialed out of this variable. bLog-transformed Rorschach variables.

Multivariate analyses. Using the Cluster B PD total symptom scores as

target or criterion variables, we sought to explore the incremental validity of
Rorschach and MMPI2 test data using two forms of multiple regression: stepwise multiple regression and hierarchical regression. First, we ran a series of
four separate stepwise multiple-regression analyses in which all predictor variables (MMPI2 scales and Rorschach variables) were available for entry into the
equation in a single block. In this form of multiple-regression analysis, the decision concerning which variables are added or subtracted at a given step is determined on the basis of the empirical relation among the variables (Cohen &
Cohen, 1983; Licht, 1995). Second, we conducted two series of hierarchical regression analyses to determine the incremental validity offered by each test in
the prediction of the Cluster B PD criteria totals. Hierarchical regression analysis has been endorsed as the appropriate statistical method for determining incremental validity (Cohen & Cohen, 1983). Hierarchical regression analysis uses a
series of simultaneous multiple-regression analyses in which one or more new
predictors are added to those used in the previous analysis; the decision concerning which variables to add at each point in the series is made by the investigator
(Cohen & Cohen, 1983; Licht, 1995). The analyses used in this study were modeled after those presented by Ben-Porath, Butcher, and Graham (1991). The hi-

158

BLAIS ET AL.

erarchical regression analyses were performed in the following manner. First, a

series of four hierarchical regressions were conducted (one for each DSMIV
Cluster B PD) with the appropriate MMPI2 PD scales (both the O and NO versions of the target disorder) and the a priori selected Rorschach variables as predictors. The predictor variables were entered into the regression analyses as two
separate blocks, with the MMPI2 scales entered as Block 1, followed by the Rorschach variables, which were entered as Block 2. Thus, all scales that were entered into the regression equation in the second block contributed incrementally to
the prediction of the DSMIV PDs (criteria totals). We then ran these four additional analyses with the order of the blocks switched: The Rorschach variables
were entered stepwise in Block 1, and the MMPI2 scales were entered stepwise
in Block 2. In all the regression analyses, we entered the independent predictors
(Rorschach variables and MMPI2 scales) into the regression equation using a
forward stepwise method until the point at which they no longer contributed significantly to the predictive power (p < .05) of the equation. Two measures of incremental validity were available from the regression analyses: the coefficient of
determination (R2) and the adjusted R2. Both of these coefficients reflect the
amount of variance in the dependent variable that can be accounted for by the
independent variable. However, the formula for R2 is not designed to include the
number of variables, so when variables are added to a regression equation, the
value of R2 will invariably increase. Adjusted R2 is a more conservative estimate
than R2 because it takes into account the addition of further variables when estimating the variability explained by the independent predictors. Given that the
adjusted R2 is a more conservative statistic, we chose to report it in place of R2
in this article.

RESULTS
Table 2 shows that five significant correlations (p < .05) were obtained between the
matched Rorschach and MMPI2 PD variables. From the correlations examined
between the Rorschach and MMPI2 variables selected to measure the DSMIV
ANPD criteria totals, AgC was the only significant correlation with both the
MMPI2 ANPDO and ANPDNO scales (rs = .31 and .30, respectively). Among
the test scores selected to measure the DSMIV BPD criteria totals, no significant
correlations were obtained. One significant correlation was obtained among the test
variables selected to tap the DSMIV HPD criteria ROD with MMPI2 HPDNO (r
= .26). Of the variables selected to measure the DSMIV NPD criteria, the Rorschach REF and IDEAL scores were significantly correlated with the MMPI2
NPDNO scale (rs = .31 and .27, respectively).
Table 3 provides the results of the four separate stepwise multiple-regression
analyses in which all predictors were included simultaneously. Table 3 shows that

INCREMENTAL VALIDITY

159

TABLE 2
Correlations for Select Rorschach Variables and the MMPI2 PD Scales
MMPI2 Cluster B PD Scale
ANPD
Rorschach Variable
Ta
PER
MORb
AG
A1
AgC a
RODa
MOASH
DEVAL
FC + CF + C
REF a,b
IDEAL

BPD

NO

.02
.24
.15
.01
.04
.31*

.02
.23
.13
.03
.02
.30*

HPD
NO

.16

.04

.17
.10
.02

.02
.04
.00

NPD

NO

.20

.21

.22

.26*

.09

.12

NO

.23
.22

.31*
.27*

Note. N = 57. MMPI2 = Minnesota Multiphasic Personality Inventory2; PD = personality

disorder; ANPD = antisocial personality disorder; BPD = borderline personality disorder; HPD =
histrionic personality disorder; NPD = narcissistic personality disorder; O = overlapping version; NO=
nonoverlapping version; Rorschach variables: T = Texture (Comprehensive System [CS]; Exner, 1986,
1993); PER = Personalized (CS); MOR = Morbid (CS); AG = Aggressive Movement (CS); A1 = Primary
Process Aggression (Holt, 1971); AgC = Aggressive Content (Gacono & Meloy, 1994); ROD =
Rorschach Oral Dependency scale (Masling et al., 1967); MOASH = Mutuality of Autonomy scale
single highest or most disturbed score (Urist, 1977); DEVAL = Devaluation (Lerner Defense Scale
[LDS]; Lerner, 1991); FC + CF + C = Form Color + Color Form + Color (CS); REF = Reflections (CS);
IDEAL = Idealization (LDS).
aThe effect of R was partialed out of analyses that employed this variable. bLog-transformed
Rorschach variable.
*p < .05.

the MMPI2 NPDO scale (Step 1 adjusted R2 = .25) and Rorschach REF scores
(Step 2 adjusted R2 = .33) were both nonredundant significant predictors of the
DSMIV NPD total criteria. The MMPI2 BPNO scales (Step 1 adjusted R2 =
.18), the Rorschach variables ROD (Step 2 adjusted R2 = .30) and MOASH (Step 3
adjusted R2 = .43) and DEVAL (Step 4 adjusted R2 = .48) were all nonredundant
significant predictors of the DSMIV BPD total criteria. The Rorschach variable
ROD was the only negative predictor of the DSMIV BPD total criteria (standardized = .41). The Rorschach variables FC + FC + C (Step 1 adjusted R2 = .16)
and T (Step 2 adjusted R2 = .24) were all nonredundant significant predictors of the
DSMIV HPD total criteria. In predicting the DSMIV ANPD total criteria, the
MMPI2 ANPDO scale (Step 1 adjusted R2 = .29) and the Rorschach MOR
scores (Step 2 adjusted R2 = .33) were both nonredundant significant predictors.

160

BLAIS ET AL.

TABLE 3
Summary of the Stepwise Multiple-Regression Analyses With the MMPI2 PD Scales and
Rorschach Variables Used to Predict DSMIV Cluster B PD Criteria Totals
Dependent Variable
(DSMIV PD)

Predictor Variables
(Rorschach and MMPI2)

ANPD
ANPD
BPD
BPD
BPD
BPD
HPD
HPD
NPD
NPD

ANPDO
MORa
BPDO
RODb
MOASH
DEVAL
FC + CF + C
Tb
NPDO
REF a,b

Step

Adjusted R2

Standardized

1
2
1
2
3
4
1
2
1
2

.55
.59
.44
.57
.67
.72
.42
.52
.51
.60

.29
.33
.18
.30
.43
.48
.16
.24
.25
.33

.58
.23
.34
.41
.30
.24
.38
.30
.44
.31

Note. N = 57. All multiple-regression analyses were significant at p < .02 or greater. MMPI2 =
Minnesota Multiphasic Personality Inventory2; PD = personality disorder; DSMIV = Diagnostic and
Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association, 1994); ANPD =
antisocial personality disorder; O = overlapping version; BPD = borderline personality disorder; HPD =
histrionic personality disorder; NPD = narcissistic personality disorder; Rorschach variables: MOR =
Morbid (Comprehensive System [CS]; Exnter, 1986, 1993); ROD = Rorschach Oral Dependency scale
(Masling et al., 1967); MOASH = Mutuality of Autonomy scale single highest or most distributed score
(Urist, 1977); DEVAL = Devaluation (Lerner Defense Scale; Lerner, 1991); FC + CF + C = Form Color
+ Color Form + Color (CS); T = Texture (CS); REF = Reflections (CS).
aLog-transformed Rorschach variables. bThe effect of R was partialed out of analyses that used this
variable.

However, the Rorschach MOR score was negatively associated with the DSMIV
ANPD total criteria (standardized = .23).
Table 4 provides the results of the first series of four hierarchical regression
analyses. In these analyses, the MMPI2 scales were entered first in Block 1, and
the Rorschach variables were entered second, in Block 2. Table 4 shows that for
predicting the DSM ANPD criteria totals, the MMPI2 ANPDO scale entered
from Block 1, and the Rorschach variable MOR were entered from Block 2, adding
incrementally to the regression equation. For predicting the DSMIV BPD criteria
totals, the MMPI2 BPDO scale entered from Block 1, whereas the Rorschach
variables ROD (Step 1), MOASH (Step 2) and DEVAL (Step 3) all entered from
Block 2. Together these three Rorschach variables accounted for an additional
30% of the variance in the BPD criteria total score. For predicting the DSMIV
HPD criteria, the MMPI2 HPDO scale was entered from Block 1, whereas the
Rorschach variables FC + F + C (Step 1) and T (Step 2) were entered from Block
2. The Rorschach variables FC + CF + C and T combined to account for an additional 19% of the variance in the HPD criteria totals beyond that accounted for by
the MMPI2 HPDO scale. In the last analyses, the MMPI2 NPDO scale was

161

INCREMENTAL VALIDITY

entered from Block 1, whereas the Rorschach variable REF was entered from
Block 2.
Table 5 contains data from the second series of hierarchical regression analyses. In these analyses, the order of the variables was switched, with the Rorschach
variables entered at Block 1 and the MMPI2 scales entered at Block 2. Table 5
shows that, for the ANPD criteria, no Rorschach variable was entered from Block
1, whereas the MMPI2 ANPDO scale was entered from Block 2 and accounted
for 28% of the variance in the ANPD criteria total. For the DSMIV BPD criteria,
three Rorschach variables were entered from Block 1 (ROD, MOASH, and
DEVAL), whereas the MMPI2 BPDO scale was entered from Block 2. The
MMPI2 BPDO scale accounted for additional 11% of the variance in the BPD
TABLE 4
Hierarchical Regression Analyses With MMPI2 PD Scales Entered Stepwise in Block 1
and the Rorschach Variables Entered Stepwise in Block 2 to Predict DSMIV PD Criteria

Criterion Variable
(DSMIV PD)
ANPD
Block 1
Block 2
BPD
Block 1
Block 2

HPD
Block 1
Block 2
NPD
Block 1
Block 2

Predictor Variable
(MMPI2 Cluster B PD
Scale and Rorschach)

Step

Adjusted R2

Standardized

ANPDO
MORa

1
1

.28
.05b

.55
.23

BPDO
RODc
MOASH
DEVAL

1
1
2
3

.18
.12b
.25
.30

.44
.41
.30
.25

HPDO
FC + CF + C
Tc

1
1
2

.08
.14b
.19

.22
.33
.26

NPDO
REF b,c

1
1

.25
.09b

.44
.31

Note. N = 57 for all analyses. All multiple-regression analyses were significant at p < .02 or greater.
MMPI2 = Minnesota Multiphasic Personality Inventory2; PD = personality disorder; DSMIV =
Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association,
1994); ANPD = antisocial personality disorder; O = overlapping version; BPD = borderline personality
disorder; HPD = histrionic personality disorder; NPD = narcissistic personality disorder; Rorschach
variables: MOR = Morbid (Comprehensive System [CS]; Exnter, 1986, 1993); ROD = Rorschach Oral
Dependency scale (Masling et al., 1967); MOASH = Mutuality of Autonomy scale single highest or
most distributed score (Urist, 1977); DEVAL = Devaluation (Lerner Defense Scale; Lerner, 1991); FC +
CF + C = Form Color + Color Form + Color (CS); T = Texture (CS); REF = Reflections (CS).
aLog-transformed Rorschach variables. bThe adjusted R2 for Block 2 variables is reported as the
incremental improvement over the Block 1 values. cThe effect of R was partialed out of analyses that
employed this variable.

162

BLAIS ET AL.

TABLE 5
Hierarchical Regression Analyses With Rorschach Variables Entered Stepwise in Block 1
and the MMPI2 PD Scales Entered Stepwise in Block 2 to Predict DSMIV PD Criteria

Criterion Variable
(DSMIV PD)
ANPD
Block 1
Block 2
BPD
Block 1

Block 2
HPD
Block 1
Block 2
NPD
Block 1
Block 2

Predictor Variable
(MMPI2 Cluster B PD
Scale and Rorschach)

Step

Adjusted R2

Standardized

No variable entered
ANPDO

.28

.55

RODa
MOASH
DEVAL
BPDO

1
2
3
1

.16
.33
.37
.11b

.41
.30
.24
.34

FC + CF + C
Ta
No variable entered

1
2

.16
.24

.38
.30

REF a,c
NPDO

1
1

.16
.18b

.31
.44

Note. N = 57 for all analyses. All multiple-regression analyses were significant at p < .02 or greater.
MMPI2 = Minnesota Multiphasic Personality Inventory2; PD = personality disorder; DSMIV =
Diagnostic and Statistical Manual of Mental Disorders (4th ed.; American Psychiatric Association,
1994); ANPD = antisocial personality disorder; O = overlapping version; BPD = borderline personality
disorder; HPD = histrionic personality disorder; NPD = narcissistic personality disorder; Rorschach
variables: ROD = Rorschach Oral Dependency scale (Masling et al., 1967); MOASH = Mutuality of
Autonomy scale single highest or most distributed score (Urist, 1977); DEVAL = Devaluation (Lerner
Defense Scale; Lerner, 1991); FC + CF + C = Form Color + Color Form + Color (Comprehensive
System [CS]; Exner, 1986, 1993); T = Texture (CS); REF = Reflections (CS).
aThe effect of R was partialed out of analyses that employed this variable. bThe adjusted R2 for Block 2
variables is reported as the incremental improvement over the Block 1 values. cLog-transformed
Rorschach variables.

criteria score beyond the Rorschach variables. With the DSMIV HPD criteria, the
Rorschach variables FC + CF + C and T were entered from Block 1, and no
MMPI2 scales were entered from Block 2. For the DSMIV NPD criteria, the
Rorschach variable REF was entered from Block 1, whereas the MMPI2 NPDO
scale was entered from Block 2. The MMPI2 NPDO scale accounted for an additional 18% of the variance in the DSMIV NPD criteria score beyond the Rorschach variable.
DISCUSSION
In this study, we sought to explore two related issues: (a) the interrelation between
select MMPI2 scales and Rorschach variables and (b) the incremental validity

INCREMENTAL VALIDITY

163

achieved by combining data from these two assessment instruments for predicting
the total number of DSMIV Cluster B PD criteria (ANPD, BPD, HPD, and NPD).
These are two important areas that have not been extensively studied in the assessment literature. The intertest correlational data obtained in this study are quite interesting because they reveal a fairly low degree of association between the matched
Rorschach variables and the MMPI2 PD scales. Only 5 of the 30 correlations were
significant (17%), and 1 of these was unexpectedly in the negative direction (ROD
with MMPI2 HPDNO, r = .26, p < .04). Our correlational findings are highly
consistent with Archer and Krishnamurthys (1997) review of the literature, both in
terms of the number of significant correlations and their magnitude. Still, the modest magnitude of the correlations is within the range expected, given that they represent cross-method correlations (see Meyer, 1996). With the exception of Meyers
(1996, 1997, 1999) recent and, in some ways, novel work, our findings are consistent with past research and support the conclusion that MMPI and Rorschach data
are generally unrelated or at best modestly related (Archer & Krishnamurthy,
1997). However, the results from the two series of hierarchical regression analyses
provide additional important information regarding the diagnostic utility of these
two tests.
Despite the weak interrelation observed between the two instruments, the multiple- and hierarchical regression analyses clearly indicate that both the Rorschach
and MMPI2 PD test variables are meaningfully related to the DSMIV Cluster B
PD criteria total scores. The multiple-regression analyses (Table 3) showed that,
for three of the four target variables (NPD, BPD, and ANPD total criteria scores),
when the MMPI2 and Rorschach variables were presented together (as a single
block), a variable from each test was entered into the equation on either the first or
second step. Together the combined MMPI2 and Rorschach data accounted for
one third (33% for NPD) to just under one half (48% BPD) of the variance in these
two criterion variables. This impressive finding can be seen as supporting the clinical tradition of combining Rorschach and MMPI data in the assessment process.
Still, one should not make too much of these findings, because the hierarchical regression analyses provide a more rigorous evaluation of the incremental validity
offered by these two instruments.
In the first series of hierarchical regression analyses (Table 4), Rorschach variables (entered in Block 2) added significant predictive power to all the DSMIV
Cluster B PD criteria scores beyond that provided by the MMPI2 PD scales. For
example, in the prediction of the DSMIV BPD total criteria score, three Rorschach variables combined to explain an additional 30% of the variance beyond
the MMPI2 scales. The results from the second series of regression analyses were
less clear. When the Rorschach variables were entered first, in Block 1, there was
evidence of incremental validity for the MMPI2 NPDO and BPDO scales,
which both entered their respective regression equations from Block 2. However,
in this series of analyses, no Rorschach variable entered from Block 1 into the
equation for predicting the DSMIV ANPD criteria score, and no MMPI2 scale

164

BLAIS ET AL.

(MMPI HPDO and HPDNO) was entered from Block 2 into the equation for
predicting the DSMIV HPD criteria score. These findings raise questions regarding the strength of the relation of both the Rorschach MOR score and the MMPI2
HPD scales to their respective target variables.
Taken together, the results of these two series of regression analyses provide
strong support for the criterion-related validity of three MMPI2 PD scales
(ANPDO, BPDO, and NPDO) and six of the Rorschach variables (ROD,
MOASH, DEVAL, FC + CF + C, T, and REF) used in this study. Strong support
was also obtained for the incremental validity achieved by combining Rorschach
and MMPI2 data in predicting DSMIV BPD and NPD criteria. Our findings are
consistent with the observation that the best data from which to predict a criterion
variable are data that are valid but not highly intercorrelated (Meehl & Hathaway,
1946).
It is interesting to speculate a little further on the meaning of our findings. For
example, both the Rorschach REF response and the MMPI2 NPDO scale independently predicted the total number of DSMIV NPD criteria assessed to patients.
The final adjusted R2 for this regression equation was .33, indicating that together
these scales accounted nearly one third of the total variance in the DSMIV NPD
criteria assignment. However, as the data in Table 1 show, these scales were not
themselves significantly correlated (r = .23). Together these findings indicate that
to a substantial degree, material present in a patients self-report (the MMPI2
data) and material that is provided outside of a patients awareness (Rorschach responses) reflect important but different components of the DSMIV conception of
narcissism and NPD. The findings for the DSMIV BPD also show that MMPI2
and Rorschach data can be combined to improve the prediction of the disorder (or
its criteria). These data provide further support for the importance of using
multimethod assessment procedures, particularly in the evaluation of personality
functioning (Hilsenroth, Handler, & Blais, 1996).
One potential limitation to the generalizability of our results is the nature of the
sample we used. We used a retrospective chart review to identify potential patients
and a comprehensive review of chart material to make the DSMIV PD diagnoses
used as target variables in the study. Although chart review methodology has been
successfully used in previous studies of the DSM PDs (Fyer et al., 1988; McCann,
1991; Morey, 1985), such a methodology tends to identify a prototypic example
of PDs and therefore establishes a purer or better defined criterion variable
(Zimmerman, 1995). The use of prototypic patients might have enhanced the predictive value and functioning of our test data, making our findings more pronounced than those that might be obtained with a less prototypic sample. However,
any enhancement effect would be expected to affect both tests equally.
Compared to previous Rorschach and MMPI studies, this study has many
unique features. We established a stringent level of reliability both for the Rorschach variables used in the study and the criterion variable (DSMIV Cluster B

INCREMENTAL VALIDITY

165

PD criteria). This basic requirement has not always been meet by past studies exploring relations between Rorschach and MMPI data (Ganellen, 1996b). Furthermore, the Rorschach variables selected for this study were all either conceptually
or empirically associated with the criterion variables. We used MMPI2 scales
that were developed through a conceptual and empirical process to specifically
measure the criterion variables, the DSM PDs (unlike the traditional MMPI2 clinical scales, which have a less clear relation to current psychiatric constructs). Last,
our criterion variables, the DSMIV Cluster B PD criteria total scores, were subject
to a rigorous and reliable diagnostic process and therefore may possess greater criterion validity than the more typically used self-report measures. Together these
features served to maximize the reliability and validity of all the assessment methods used, increasing our statistical power and allowing underlying relations to reveal themselves. This study provides the first clear support for the incremental
validity achievable by combining Rorschach and MMPI2 data in predicting clinically relevant, nontest, real-world behaviors (the DSMIV PD criteria). Future
work in this area should seek to incorporate the methodological advantages of using reliable and conceptually relevant test data when exploring the incremental validity of these two popular psychological assessment instruments.

ACKNOWLEDGMENTS
An earlier version of this article was presented at the annual midwinter meeting of
the Society of Personality Assessment, Boston, February 1998.
We thank Myra Christensen, Susan Church, Sheila OKeefe, Karen Toman,
and Greta Smith for their various contributions to this study. We also thank the two
anonymous reviewers for their constructive and helpful comments.

REFERENCES
Acklin, M. W. (1993). Integrating the Rorschach and MMPI in clinical assessment: Conceptual and
methodological issues. Journal of Personality Assessment, 60, 125131.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th
ed.). Washington, DC: Author.
Archer, R. P. (1996). MMPIRorschach interrelationships: Proposed criteria for evaluating explanatory
models. Journal of Personality Assessment, 67, 504515.
Archer, R. P., & Gordon, R. A. (1988). MMPI and Rorschach indices of schizophrenic and depressive
diagnoses among adolescent inpatients. Journal of Personality Assessment, 52, 276287.
Archer, R. P., & Krishnamurthy, R. (1993a). Combining the Rorschach and MMPI in the assessment of
adolescents. Journal of Personality Assessment, 60, 132140.
Archer, R. P., & Krishnamurthy, R. (1993b). A review of MMPI and Rorschach interrelationships in
adult samples. Journal of Personality Assessment, 61, 277293.

166

BLAIS ET AL.

Archer, R. P., & Krishnamurthy, R. (1997). MMPIA and Rorschach indices related to depression and
conduct disorder: An evaluation of the incremental validity hypothesis. Journal of Personality Assessment, 69, 517533.
Archer, R. P., & Krishnamurthy, R. (1999). Reply to Meyer on the convergent validity of the MMPI and
Rorschach. Journal of Personality Assessment, 73, 319321.
Baity, M. R., & Hilsenroth, M. J. (1999). Rorschach aggression variables: A study of reliability and validity. Journal of Personality Assessment, 72, 93110.
Ben-Porath, Y., Butcher, J., & Graham, J. (1991). Contribution of the MMPI2 content scales to the differential diagnosis of schizophrenia and major depression. Psychological Assessment: A Journal of
Consulting and Clinical Psychology, 3, 634640.
Blais, M. A., Hilsenroth, M. J., & Fowler, J. C. (1998). Rorschach correlates of the DSMIV histrionic
personality disorder. Journal of Personality Assessment, 70, 355364.
Blais, M. A., Hilsenroth, M. J., Fowler, J. C., & Conboy, C. A. (1999). A Rorschach exploration of the
DSMIV borderline personality disorder. Journal of Clinical Psychology, 55, 110.
Blais, M., & Norman, D. (1997). A psychometric evaluation of the DSMIV personality disorder criteria. Journal of Personality Disorders, 11, 168176.
Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. (1989). MMPI2: Minnesota Multiphasic Personality Inventory: Manual for administration and scoring. Minneapolis: University of Minnesota Press.
Castlebury, F., Hilsenroth, M., Handler, L., & Durham, T. (1997). Use of the MMPI2 personality disorder scales in the assessment of DSMIV antisocial, borderline, and narcissistic personality disorders.
Assessment, 4, 155168.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Colligan, R., Morey, L., & Offord, K. (1994). The MMPI/MMPI2 personality disorder scales: Contemporary norms for adults and adolescents. Journal of Clinical Psychology, 50, 168200.
Curran, P., West, S., & Finch, J. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis, Psychological Methods, 1, 1629.
Dubro, A., Wetzler, S., & Kahn, M. (1988). A comparison of three self-report questionnaires for the diagnosis of DSMIII personality disorders. Journal of Personality Disorders, 2, 256266.
Exner, J. (1986). The Rorschach: A comprehensive system: Vol. 1. Basic foundations (2nd ed.). New
York: Wiley.
Exner, J. (1993). The Rorschach: A comprehensive system: Vol. 1. Basic foundations (3rd ed.). New
York: Wiley.
Fyer, M., Frances, A., Sullivan, T., Hurt, S., & Clarkin, J. (1988). Comorbidity of borderline personality
disorders. Archives of General Psychiatry, 45, 348352.
Gacono, C., & Meloy, J. (1991). A Rorschach investigation of attachment and anxiety in antisocial personality disorder. Journal of Nervous and Mental Disease, 179, 546552.
Ganellen, R. J. (1996a). Exploring the MMPIRorschach relationships. Journal of Personality Assessment, 67, 529542.
Ganellen, R. J. (1996b). Integrating the Rorschach and the MMPI2 in personality assessment.
Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Hathaway, S. R., & McKinley, J. C. (1943). The Minnesota Multiphasic Personality Inventory.. Minneapolis: University of Minnesota Press.
Hilsenroth, M. J., Fowler, J. C., Padawer, J. R., & Handler, L. (1997). Narcissism in the Rorschach revisited: Some reflections on empirical data. Psychological Assessment, 9, 113121.
Hilsenroth, M., Handler, L., & Blais, M. (1996). Assessment of narcissistic personality disorder: A
multimethod review. Clinical Psychology Review, 16, 655683.
Holt, R. R. (1977). A method for assessing primary process manifestations and their control in Rorschach responses. In M. A. Rickers-Ovsiankina (Ed.), Rorschach psychology (2nd ed., pp.
375420). Huntington, NY: Krieger.

INCREMENTAL VALIDITY

167

Krishnamurthy, R., Archer, R. P., & House, J. J. (1996). The MMPIA and Rorschach: A failure to establish convergent validity. Assessment, 3, 179191.
Lerner, P. (1991). Psychoanalytic theory and the Rorschach. Hillsdale, NJ: The Analytic Press.
Licht, R. (1995). Multiple regression and correlation. In L. Grimm & P. Yarnold (Eds.), Reading and
understanding multivariate statistics (pp. 1965). Washington, DC: American Psychological
Association.
Lubin, B., Larsen, R. M., & Matarazzo, J. D. (1984). Patterns of psychological test usage in the United
States: 19351982. American Psychologist, 39, 179191.
Lubin, B., Wallis, R., & Paine, C. (1971). Patterns of psychological test usage in the United States. Professional Psychology, 2, 7074.
Marsh, A., & Viglione, D. (1992). A conceptual validation study of the texture response on the Rorschach. Journal of Personality Assessment, 58, 571579.
Masling, J., Rabie, L., & Blondheim, S. (1967). Obesity, level of aspiration, and Rorschach and TAT
measures of oral dependence. Journal of Consulting Psychology, 31, 233239.
McCann, J. (1989). MMPI personality disorder scales and the MCMI: Concurrent validity. Journal of
Clinical Psychology, 45, 365369.
McCann, J. (1991). Convergent and discriminant validity of the MCMIII and the MMPI personality
disorders. Psychological Assessment, 3, 918.
Meehl, P. E., & Hathaway, S. R. (1946). The K factor as a suppressor variable in the MMPI. Journal of
Applied Psychology, 30, 525564.
Meyer, G. J. (1993). The impact of response frequency on Rorschach constellation indices and on their
validity with diagnostic and MMPI2 criteria. Journal of Personality Assessment, 60, 153180.
Meyer, G. J. (1996). The Rorschach and MMPI: Toward a more scientifically differentiated understanding of cross-method assessment. Journal of Personality Assessment, 67, 558578.
Meyer, G. J. (1997). On the integration of personality assessment methods: The Rorschach and
MMPI2. Journal of Personality Assessment, 68, 290330.
Meyer, G. J. (1999). The convergent validity of MMPI and Rorschach scales: An extension using profile
scores to define responses and character styles on both methods and a reexamination of simple Rorschach response frequency. Journal of Personality Assessment, 72, 135.
Morey, L. (1985). A psychometric analysis of five DSMIII categories. Personality and Individual Differences, 6, 323329.
Morey, L. (1986). A comparison of three personality disorder assessment approaches. Journal of
Psychopathology and Behavioral Assessment, 8, 2530.
Morey, L., Waugh, M., & Blashfield, R. (1985). MMPI scales for the DSMIII personality disorders:
Their derivation and correlations. Journal of Personality Assessment, 49, 245251.
OMaille, P., & Fine, M. (1995). Personality disorder scales for the MMPI2: An assessment of
psychometric properties in a correctional population. Journal of Personality Disorders, 9,
235246.
Piotrowski, C., & Keller, J. (1984). Psychodiagnostic testing in APA-approved clinical psychology programs. Professional Psychology: Research and Practice, 15, 450456.
Piotrowski, C., & Keller, J. (1989). Psychological testing in outpatient mental health facilities: A national study. Professional Psychology, 20, 423425.
Piotrowski, C., Sherry, D., & Keller, J. (1985). Psychodiagnostic test usage: A survey of the Society for
Personality Assessment. Journal of Personality Assessment, 49, 115119.
Rorschach, H. (1942). Psychodiagnostics. Berne, Switzerland: Hans. (Original work published 1921)
Sechrest, L. (1963). Incremental validity: A recommendation. Educational and Psychological Measurement, 23, 153158.
Stevens, J. (1996). Applied multivariate statistics for the social sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Trull, T. (1993). Temporal stability and validity of two personality disorder inventories. Psychological
Assessment, 5, 1118.

168

BLAIS ET AL.

Urist, J. (1977). The Rorschach test and the assessment of object relations. Journal of Personality Assessment, 41, 39.
Watkins, C., Campbell, V., Nieberding, R., & Hallmark, R. (1995). Contemporary practice of psychological assessment by clinical psychologists. Professional Psychology, 26, 5460.
Wechsler, D. (1981). Wechsler Adult Intelligence ScaleRevised. New York: Psychological Corporation.
Weiner, I. (1991). Editors note: Interscorer agreement in Rorschach research. Journal of Personality
Assessment, 56, 1.
Widiger, T. (1991). Personality disorder dimensional model proposed for DSMIV. Journal of Personality Disorders, 5, 386398.
Wise, E. A. (1996). Comparative validity of the MMPI2 and MCMIII personality disorder classification. Journal of Personality Assessment, 66, 569582.
Wood, J., Nezworski, M., & Stejskal, W. (1997). The reliability of the Comprehensive System for the
Rorschach: A comment on Meyer (1997). Psychological Assessment, 9, 490494.
Viglione, D. (1997). Problems in Rorschach research and what to do about them. Journal of Personality
Assessment, 68, 590599.
Zimmerman, M. (1995). Diagnosing personality disorders: A review of issues and research methods.
Archives of General Psychiatry, 51, 225245.

Mark A. Blais
Inpatient Psychiatry Service
Massachusetts General Hospital
Blake-11
55 Fruit Street
Boston, MA 02114
E-mail: mblais@partners.org
Received May 3, 2000
Revised June 19, 2000