Measuring Quality of Care With Explicit

Process Criteria Before and After

Implementation of the DRG-Based
Prospective Payment System
Katherine L. Kahn, MD; William H. Rogers, PhD; Lisa V. Rubenstein, MD, MSPH; Marjorie J. Sherwood, MD; Ellen J. Reinisch, MS;
Emmett B. Keeler, PhD; David Draper, PhD; Jacqueline Kosecoff, PhD; Robert H. Brook, MD, ScD

We developed explicit process criteria and scales for Medicare patients hospital- Developing Process Criteria
ized with congestive heart failure, myocardial infarction, pneumonia, cerebro- We used literature review and con¬
vascular accident, and hip fracture. We applied the process scales to a nationally sultation with experts to develop a set of
representative sample of 14 012 patients hospitalized before and after the process measures for which better pro¬
implementation of the diagnosis related group\p=m-\basedprospective payment cess was likely to make a difference in

system. For the four medical diseases, a better process of care resulted in lower patient outcome. These measures were
mortality rates 30 days after admission. Patients in the upper quartile of process then presented to disease-specific pan¬
scores had a 30-day mortality rate 5% lower than that of patients in the lower
els consisting of five to 12 physicians,
who were selected by our collaborators,
quartile. The process of care improved after the introduction of the prospective the professional review organizations.
payment system; eg, better nursing care after the introduction of the prospective Each panel reviewed the suggested cri¬
payment system was associated with an expected decrease in 30-day mortality teria to decide whether they believed
rates in pneumonia patients of 0.8 percentage points, and better physician that data to assess these criteria were
cognitive performance was associated with an expected decrease in 30-day reliably recorded in the medical record
mortality rates of 0.4 percentage points. Overall, process improvements across and whether the criteria made clinical
all four medical conditions were associated with a 1 percentage point reduction in sense. Process criteria based on data

30-day mortality rates after the introduction of the prospective payment system.
PROCESSES of care-what we do to
patients—have been considered an es¬
sential component of quality of care
measurement for over 50 years.1'6 Even
if outcomes of care—what happens to
patients—are the most meaningful
measures of quality to the patient, we
will be unable to develop clinical meth¬
ods to improve outcomes unless we un¬
derstand how processes and outcomes
are related. Assessing quality of care by
process also provides some measure-
From the Health Program of the RAND Corp, Santa
Monica, Calif (Drs Kahn, Rogers, Rubenstein, Sher-
wood, Keeler, Draper, and Brook and Ms Reinisch); the
Departments of Medicine (Drs Kahn, Rubenstein, Ko-
secoff, and Brook) and Health Services (Drs Kosecoff
and Brook), UCLA; and Value Health Sciences Inc,
Santa Monica (Dr Kosecoff).
The opinions, conclusions, and proposals in the text
are those of the authors alone and do not necessarily
represent the views of the RAND Corp or UCLA.
Reprint requests to the RAND Corp, 1700 Main St, PO
Box 2138 Santa Monica, CA 90406-2138 (Dr Kahn).
(JAMA. 1990;264:1969-1973)
ment advantages over studying out¬
comes, because not all patients who ex¬
perience a poor process of care suffer a
poor outcome.
The purpose of this article is twofold.
First, we report on the development of a
set of validated process criteria for el¬
derly patients admitted to the hospital
with one of five conditions. By validated
we mean that process predicts outcome.
Second, we apply the validated process
criteria to patients treated before and
after the implementation of the pro¬
spective payment system (PPS) to de¬
termine whether the PPS has been asso¬
ciated with changes in the processes of
care.
METHODS
We based our analysis on the sample
described in more detail elsewhere in
this series.7
whose recording was likely to vary by
year, state, or hospital type were ex¬
cluded. We developed disease-specific
abstraction forms8"12 to collect data on
approximately 100 process criteria for
each disease.
Scoring Process Criteria
In scoring process criteria, we first
applied the criteria only to patients who
were likely to benefit from their use.
Using this kind of conditional logic,
many criteria were applicable to all pa¬
tients, some to just a few. For example,
if a patient with congestive heart failure
was considered to be severely ill, then
the intensive care unit should be used.
Second, we used clinical judgment to
assign scores (points) to each process
criterion based on how likely a patient
was to benefit from it. For example, use
of the intensive care unit for very sick
patients was assigned seven points,
whereas use of the intensive care unit
for moderately sick patients was as¬
signed three points. Third, the process

Downloaded From: by a University of Pennsylvania User on 12/13/2018

 Table 1 .—Examples of Process Criteria and Performance Levels Before and After Introduction of the PPS*

Patients to Whom Criteria

Patients to Whom Criteria Were Applicable Who Met
Were Applicable, % Process Criteria, %

Criteria Disease Before PPS After PPS Before PPS After PPS
Physician Cognitive Scale
Within the initial 2 days of hospitalization the
physician should document each of the following
in the medical record as noted or not noted
Past surgery Congestive heart failure 100 66f
Lung examination day 2on Congestive heart failure 58 711
Alcoholism or smoking habits Acute myocardial Infarction 100 100 61 64
Jugular veins Acute myocardial infarction 100 100 61 68t
Tobacco use or nonuse Pneumonia 100 100 52t
Lower-extremity edema Pneumonia 75t
Previous cerebrovascular accident Cerebrovascular accident 100 100 48 53t
Gag reflex Cerebrovascular accident 100 100 35 38
Mental status Hip fracture 70
Pedal or leg pulse Hip fracture 100 100 62 67t
Nurse Cognitive Scale
On day 2 of the hospitalization at least three blood
pressure readings should be noted
>3 blood pressure readings noted Congestive heart failure 100 100 78 84t
>3 blood pressure readings noted Pneumonia 100 100 69 79t
>3 blood pressure readings noted Cerebrovascular accident 100 100 79 86t
Technical Diagnostic Scale
Within the initial 2 days of hospitalization an electro¬
cardiogram should be obtained
Electrocardiogram obtained Congestive heart failure 100 100 87 91t
Electrocardiogram obtained Cerebrovascular accident 86t
Electrocardiogram obtained Hip fracture 100 100 90 93t
Within the initial 2 days of hospitalization a serum
potassium determination should be performed
Serum potassium level determined Congestive heart failure 100 100 93 97t
Serum potassium level determined Cerebrovascular accident 100 88 94t
Serum potassium level determined Hip fracture 100 100 89 94t
Technical Therapeutic Scale
po,<60 mm Hg, use oxygen therapy or intubate
If
Oxygen therapy or intubation done Congestive heart failure 16 20 87 93t
Oxygen therapy or intubation done Pneumonia 23 90t
Begin antibiotic therapy for patients with pneu¬
monia in a timely manner
Within 4 hours of admission for
nonimmunocompromised patients Pneumonia 91 88 28 32t
Within 2 hours of admission for
nonimmunocompromised patients Pneumonia 12
Monitoring With Intensive Care and Telemetry Scale
For patients who are moderately sickt use the
intensive care unit; telemetry Is not sufficient but
is preferable to no cardiac monitoring
Used Intensive care unit on day 1 Congestive heart failure 16 16 43 46
Used telemetry on day 1 Congestive heart failure 16 16 24t
Used intensive care unit on day 2 Congestive heart failure 49
Used telemetry on day 2 Congestive heart failure 31t
Used intensive care unit on day 1 Pneumonia
Used telemetry on day 1 Pneumonia 17 19 10t
Used intensive care unit on day 2 Pneumonia
Used telemetry on day 2 Pneumonia 12t
For patients who are very sickt use the Intensive
care unit; telemetry Is not sufficient but is
preferable to no cardiac monitoring
Used intensive care unit on day 1 Congestive heart failure 71
Used telemetry on day 1 Congestive heart failure
Used intensive care unit on day 2 Pneumonia
Used telemetry on day 2 Pneumonia

*PPS indicates prospective payment system.

tP<05 compared with before PPS.
tModerately sick was defined as a score of 5 or 6 and very sick as a score &7 on each hospital day, with points assigned as follows: chest pain, 1 point; shortness of breath,
1 point; confusion, 2 points; heart rate ^130 beats per minute, 2 points; respiratory rate =;30/min, 2 points; and diastolic blood pressure ==105 mm Hg and systolic blood
pressure <90 mm Hg, 3 points.

scores accounted for the use of different
interventions. Very sick patients re¬
ceived seven of seven points for use of
the intensive care unit, three of seven
points for use of telemetry, and no
points for no cardiac monitoring.
Process Scales
Using clinical judgment we grouped
process criteria according to what con¬
cept we thought they measured and
then tested our groupings by comparing
them with those suggested by a Likert
scaling model.13 Use of these methods
yielded five process subscales and one
overall process scale: physician cogni¬
tive, nurse cognitive, technical diagnos¬
tic, technical therapeutic, monitoring

Downloaded From: by a University of Pennsylvania User on 12/13/2018

 with intensive care or telemetry, and telemetry monitoring scales are much RESULTS
overall process. less dependent on styles of documenta¬
The physician diagnostic cognitive tion.
Reliability and Validity
of Measures
scale measures the physician's perfor¬ To produce these scales, we combined
mance as a gatherer of data about the some process measures applicable to all Compliance was high for most of the
patient's medical history and current patients with those applicable to sub¬ explicitly stated process criteria (Table
symptoms and the performance of phys¬ sets of patients. Sicker patients and 1). However, for 21% of our patients
ical examinations during the hospital those with longer hospital stays had a with congestive heart failure, 16% of
our patients with acute myocardial in¬
stay. The nurse diagnostic cognitive greater number of applicable process
scale measures the nurse's performance criteria than did less-sick patients. In farction, and 24% of our patients with
as a gatherer of data about the patient's general, compliance with criteria that pneumonia, the presence or absence of a
functional status, current symptoms, were applicable only to sicker patients heart murmur was not noted in the med¬
and vital signs. The technical diagnostic was lower than compliance with criteria
ical record. For 19% of the patients with
process scale measures use of diagnostic that were applicable to all patients. To congestive heart failure, 26% of the pa¬
tests (eg, venous laboratory studies, ar¬ avoid a bias when combining criteria to tients with pneumonia, and 17% of the
terial blood gas tests, roentgenograms, form scales, we standardized all process patients with cerebrovascular acci¬
and electrocardiograms) that are indi¬ criteria to have a mean of 0 and an SD of dents, fewer than three blood pressure
cated given the patient's daily burden of 1. The overall process scale represents readings were taken on day 2 of the
illness. The technical therapeutic pro¬ an average of the five subscales. A pa¬ hospitalization. Five percent of the pa¬
cess scale measures use of treatments tient who underwent an average pro¬ tients with congestive heart failure, 6%
of the patients with acute myocardial
(eg, medication, surgery, and physical cess of care has an expected process
therapy) that are indicated given the score of 0 and an SD of 1. infarction, 9% of the patients with cere¬
patient's daily sickness level. The inten¬ lb validate our process scales we brovascular accidents, 10% of the pa¬
sive care or telemetry monitoring scale used logistic regression to examine the tients with pneumonia, and 10% of the
evaluates the monitoring of patients as a relationship between in-hospital pro¬ patients with hip fractures did not have
function of their level of illness. Where¬ a serum potassium study done on day 1
cess scale scores and mortality 30 and
as both the physician and nurse cogni¬ or 2 of the hospital stay. One fourth of
180 days after admission after adjusting
tive scales are somewhat dependent on for disease-specific sickness at admis¬ the patients sick enough to be hospital¬
styles of documentation in the medical sion. " Linear regression was also used ized for congestive heart failure did not
record, the technical diagnostic, techni¬ to determine the association of the PPS have a serum creatinine study done in
cal therapeutic, and intensive care or with change in process. the first 2 days, while one third of the
patients with congestive heart failure
admitted in a moderately sick or very
Table 2. —Relationships Between Mortality Rates After Admission Adjusted for Sickness at Admission and sick condition did not have any creati¬
Overall Process Scale for Five Diseases nine phosphokinase enzyme studies
Mortality Rates 30 Days After Relative Risk
done on day 1 or 2 of the hospitalization
Admission, Adjusted for Sickness of Adjusted 30-Day to rule out an acute myocardial
at Admission,14 by Overall Death for Poor infarction.
Process Scale Score Category, %* Compared With
'-'-> Good Care For patients hospitalized with con¬
Disease Medium Poor
Good Pf Process4:
gestive heart failure, acute myocardial
Congestive heart failure_107_12j)_18^6_<0)1_1.74 (0.23) infarction, pneumonia, or cerebrovas¬
Acute myocardial cular accident, better process is signifi¬
infarction_23^9_22JD_301_<01_1.26 (0.11)
Pneumonia_14ji_152_202_<a>1_1.36
Cerebrovascular
(0.16) cantly associated with a lower 30-day
accident_18T_203_25^5_<.Q1_1.36 (0.14) mortality rate. For patients with con¬
Hip fracture 5.1 5.2 4.6 >.05 0.90 (0.22)
gestive heart failure, the mortality rate
30 days after admission, adjusted for
'Patients were rank-ordered according to process scale scores. Patients with process scale scores in the highest
25% were considered to have experienced good process, those with scores in the lowest 25% poor process, and sickness at admission, was 11% for pa¬
the remainder medium process. tients who experienced good process of
tFrom tests of the significance of the process coefficients in the logistic regressions of mortality on process and
sickness at admission. care, 13% for those who experienced
JValues In parentheses are approximate SEs. medium process, and 19% for those who

Table 3.—Relationships Between Mortality Rates 30 Days After Admission Adjusted for Sickness at Admission and Process Scales for Five Diseases
Mortality Rates 30 Days After Admission, Adjusted for
Sickness at Admission," by Process Scale Score Category, % *

Congestive Acute Myocardial Cerebrovascular

Heart Failure Infarction Pneumonia Accident Hip Fracture
Good Poor Good Poor Good Poor Good Poor Good Poor
Process Subscale Process Process Process Process Process Process Process Process Process Process
Physician cognitive 12 16t 23 28t 15 19t 18 24t 6 5
Nurse cognitive 11 17f 24 27f 15 19t 19 24f 4 6
Technical diagnostic 11 16t 24 29t 14 19t 19 25f 4 5
Technical therapeutic 11 211 29 211 15 211 § § 5 5
Monitoring with intensive care and telemetry 18 13 21 28t 19 15 23 21 10 5$
Overall 11 19t 24 30t 15 20f 19 26t 5 5

Patients rank-ordered according to process scale scores. Patients with process scale scores In the
were highest 25% were considered to have experienced good process,
and those with the lowest 25% poor process.
scores in
tP<05 using logistic regression of mortality, adjusted for sickness at admission, on process
^Paradoxical P<05—a better process was associated with a worse outcome.
§The technical therapeutic scale was not measured for cerebrovascular accident.

Downloaded From: by a University of Pennsylvania User on 12/13/2018

 Table 4.—Process Scores Before and After Introduction of the PPS*

Congestive Heart Failure Acute Myocardial Infarction Pneumonia

Expected Change Expected Change Expected Change

in Mortality Rates in Mortality Rates in Mortality Rates
Change in After PPS, Change in After PPS, Change in After PPS,
Process Percentage Points4; Process Percentage Points4: Process Percentage Points}
Score After Score After Score After
Process Subscale PPSf 30-Day 180-Day PPSf 30-Day 180-Day PPSt 30-Day 180-Day
Physician cognitive + 0.31§ -0.5 -0.7 + 0.24§ -0.7 -0.7 + 0.24§ -0.4 -0.5
Nurse cognitive h0.36§ -0.6 + 0.22§ -0.5 ^0.42§ -0.8 -1.3
Technical diagnostic h0.26§ -0.4 -0.0 ^0.21§ -0.6 -0.5 + 0.23§ -0.4
Technical therapeutic f0.09§ -0.2 -0.1 + 0.16§ + 0.211 + 0.311 + 0.15§ -0.3 -0.4
Monitoring with intensive
care and telemetry + 0.21S, -0.9II h 0.05 -0.1 + 0.111
Overall process -0.42§ -1.2 -1.0 + 0.27§ -0.8 -0.7 h0.43§ -1.0 -1.1

*PPS Indicates prospective payment system.

tScores are rated on a scale with a mean of 0 and an SD of 1.
^Mortality rates are adjusted for sickness at admission.
§P<05 for change in process score after the introduction of the PPS.
¡[This expected change in mortality rate is included for completeness; however, the process-outcome link was not sufficiently strong for this process scale to accurately predict
a change in the mortality rate from the change In the process score.
UParadoxical P<05 for the process-outcome relationship—a better process was associated with a worse outcome.
#The technical therapeutic scale was not measured for cerebrovascular accident.

experienced poor process (P=.0002).
The relative risk of adjusted 30-day
death as process changed from good to
poor ranged from 1.74 for congestive
heart failure to 1.26 for acute myocardi¬
al infarction (P<.05, Table 2). We were
unable to demonstrate a process-out¬
come link for patients with hip frac¬
tures, partly because 5% of patients
with hip fractures died, and this low
death rate limited our power to detect a
process-outcome relationship.
In addition, a significant process-out¬
come relationship existed for four of the
five process subscales for congestive
heart failure, acute myocardial infarc¬
tion, and pneumonia and for three of the
four process subscales for cerebrovas¬
cular accidents (Table 3). We found a
clinically sensible process-outcome link
for the monitoring with intensive care
and/or telemetry subscale only for pa¬
tients with acute myocardial infarction;
we defined the need for such monitoring
more precisely for acute myocardial in¬
farction than we did for the other
diseases.
Process of Care Before and After
Introduction of the PPS
For each process scale, for all five
diseases, we found better process of
care after the introduction of the PPS
(Table 4). In all instances the improve¬
ment was significant (P<.05), except
for monitoring with intensive care and/
or telemetry, for which the process
changed significantly (P<.05) only for
congestive heart failure and hip frac¬
ture. The improvements in process af¬
ter the introduction of the PPS were
apparent both for process measures
that could have been influenced by
changes in documentation in the medi¬
cal record (eg, the physician and nurse
cognitive scales) and for process mea¬
sures that were unlikely to be affected
by such potential biases (eg, the techni¬
cal diagnostic and technical therapeutic
scales).
We used the previously demonstrat¬
ed process-outcome link to translate the
better process of care after the intro¬
duction of the PPS into mortality reduc¬
tions. For example, for patients with
congestive heart failure, the improve¬
ment in the process of care of 0.31 SD on
the physician cognitive process scale
was associated with an expected 0.5
percentage point reduction in the 30-
day postadmission mortality rate and an
expected 0.7 percentage point reduc¬
tion in the 180-day postadmission mor¬
tality rate. Similar improvements in
process on the nurse cognitive scale
were associated with expected de¬
creases in mortality of 0.8 and 0.6 per¬
centage points at 30 and 180 days, re¬
spectively. Except for hip fracture, the
improvements in the overall process
scale after the introduction of the PPS
were associated with an expected re¬
duction of 0.1 to 1.4 percentage points in
the 30-day mortality rate and an expect¬
ed reduction of 0.4 to 1.6 percentage
points in the 180-day mortality rate. Ag¬
gregating across our four medical dis¬
eases, process improvements after the
introduction of the PPS were associated
with a 1.0 percentage point reduction in
the expected 30-day mortality rate (95%
confidence limits, 0.6 to 1.4 percentage
points). Given that the observed raw 30-
day mortality rate for our four medical
diseases was 18.7%, the 1.0 percentage
point change represents a 5.3% decline
in expected mortality associated with
the improvements in process.
The improvements in process scale
scores paralleled those found in individ-
ual items (Table 1). For example, 58% of
patients with congestive heart failure
had documentation of a day 2 lung ex¬
amination before the introduction of the
PPS compared with 71% after the intro¬
duction of the PPS. Nurses documented
at least three blood pressure readings
on day 2 for 78% of patients with conges¬
tive heart failure before the introduc¬
tion of the PPS compared with 84% after
the introduction of the PPS. The use of
oxygen (or intubation) on day 1 for hyp-
oxic patients (ie, p02 <60 mm Hg) im¬
proved from 87% before the introduc¬
tion of the PPS to 93% after the
introduction of the PPS.
COMMENT
We have demonstrated the validity of
our process scales by establishing pro¬
cess-outcome links. If our process
scores only reflected recording rather
than what happened to patients, we
would have been unable to find a statis¬
tically significant relationship between
better processes of care and lower mor¬
tality. In addition, if we were measuring
only improvements in recording after
the introduction of the PPS vs before
the PPS, we would have found improve¬
ments in process after the introduction
of the PPS only for those process mea¬
sures that depend heavily on recording
(eg, physician and nurse process). How¬
ever, we have demonstrated a signifi¬
cant process-outcome relationship con¬
sistently acrossdiseases and across
types (ie, recording-sensitive and -in¬
sensitive) of process measures. We
found the process of care to be better
after the introduction of the PPS.
The lack of a consistent process-out¬
come relationship for scales based on
intensive care and telemetry monitor¬
ing was disappointing. We believe the

Downloaded From: by a University of Pennsylvania User on 12/13/2018


Cerebrovascular Accident
Expected Change
in Mortality Rates
Change in After PPS,
Process Percentage Points}
Score After-"->
PPSt_30-Day_180-Day
+
0.36§_-0.8_-0.4
+
0.46§_-0.8_-0.5
+
0.25§_-06_-0.4
# # # American people.
+
0.08_-0.2II_— 0.2(1
-1.4 -1.0
+ 0.49§
problem lies in our imperfect measure¬
ment of the if in the if-then process
statements. We need to better under¬
stand how to identify the group of pa¬
tients for whom use of intensive care
and telemetry monitoring makes a
difference.15
It is notable that we found a signifi¬
cant process-outcome relationship for
patients with all four of the medical dis¬
eases but not for patients with hip frac¬
tures. This may be because short-term
mortality occurs less often for patients
with hip fractures than for those with
medical diseases. Alternatively, mor¬
tality may not be the best outcome to
study for patients with hip fractures.
Another possibility is that the medical
record does not provide an adequate
data source for evaluating surgical,
particularly intraoperative, processes.
Methods for better evaluating surgical
processes of care are needed.
Our consistent findings across pro¬
subscales and diseases suggest that
cess
the process of care has improved from
1981 to 1986. The implementation of the
PPS was not associated with a deterio-
References
1. Donabedian A. Evaluating the quality of medi-
cal care. Milbank Q. 1966;44:166-203.
2. Brook RH, Appel RA. Quality of care assess-
ment: choosing a method for peer review. N Engl J
Med. 1973;288:1323-1329.
3. Wyszewianski L. Quality of care: past achieve-
ments and future challenges. Inquiry. 1988;25:13\x=req-\
22.
4. McAuliffe WE. Studies of process-outcome cor-
relations in medical audits: a critique. Med Care.
1978;16:907-930.
5. Rubenstein LV, Mates S, Sidel VW. Quality of
care assessment by process and outcome scoring.
Ann Intern Med. 1977;86:617-625.
6. Greenfield S, Cretin S, Worthman L. Compari-
son of criteria maps to a criteria list in quality of
care assessment for patients with chest pain: the
relation of each to outcome. Med Care. 1981;19:255\x=req-\
272.
7. Draper D, Kahn KL, Reinisch EJ, et al. Study-
Downloaded From: by a University of Pennsylvania User on 12/13/2018
Hip Fracture
Expected Change
in Mortality Rates
Change in After PPS,
Process Percentage Points4.
Score After -"->
PPSt_30-Day_180-Day
+
0.16§_+0.111_ 0 0||
+
0.31§_-0.111_-0.311
+
0.22§_-0.011_-0.211
'
0.29§_-0.1|i_-0.2I
-0.04§_-0.111_-0,21
-0.4
+ 0.41§ -0.0II
ration in care, even in those areas that
were most sensitive to the financial in¬
centives provided by prospective pay¬
ment to decrease the level of services,
such as nursing activities and the use of
intensive care units. This is an encour¬
aging finding and indicates that the 24%
decrease in length of stay after the in¬
troduction of the PPS was not associat¬
ed with a deterioration in the process of
in-hospital care. If anything, improve¬
ments in the process of care after the
introduction of the PPS should lead to
about a 1 percentage point reduction in
the 30-day mortality rate. This effect
was produced by improvements in both
physician and nursing care.
Our other notable result was the large
difference in mortality, adjusted for ad¬
mission sickness, between patients at
the top and bottom ends of our overall
process scale. For the four medical con¬
ditions combined, the adjusted mortal¬
ity rate went from 17.0% to 23.6% as the
process went from the highest to the
lowest quartile, an effect more than six
times greater than the effect on expect¬
ed mortality of process improvements
ing the effects of the DRG-based prospective pay-
ment system on quality of care: design, sampling,
and fieldwork. JAMA. 1990;264:1956-1961.
8. Kahn KL, Chassin MR, Rubenstein LV, et al.
Medical Record Abstraction Form and Guidelines
for Assessing Quality of Care for Hospitalized Pa-
tients With Congestive Heart Failure. Santa Moni-
ca, Calif: RAND Corp; 1988. N-2798-HCFA.
9. Rubenstein LV, Kosecoff J, Kahn KL, et al.
Medical Record Abstraction Form and Guidelines
for Assessing Quality of Care for Hospitalized Pa-
tients With Cerebrovascular Accident. Santa Moni-
ca, Calif: RAND Corp; 1988. N-2802-HCFA.
10. Kosecoff J, Rubenstein LV, Kahn KL, et al.
Medical Record Abstraction Form and Guidelines
for Assessing Quality of Care for Hospitalized Pa-
tients With Acute Myocardial Infarction. Santa
Monica, Calif: RAND Corp; 1988. N-2799-HCFA.
11. Sherwood MJ, Kahn KL, Kosecoff J, et al.
Medical Record Abstraction Form and Guidelines
after the introduction of the PPS. Fur¬
ther analyses may identify which types
of patients and hospitals tend to fall in
the lower or upper process quartiles.
With this information an ongoing clini¬
cal system for improving care can be
developed through which professionals
can improve the care they give and
thereby improve the health of the
This study was supported by cooperative agree¬
ment 18-C-98853/9-03 issued by the Health Care
Financing Administration of the US Department of
Health and Human Services. The entire costs to the
RAND Corp for this project ($3.9 million) were
funded with federal money.
We gratefully acknowledge the collaborative ef¬
fort of the five professional review organizations
that enabled this work to be completed. Partici¬
pants from the professional review organizations
included medical directors, physician-consultants
and -reviewers, project directors, and review coor¬
dinators from each of the five study states. In par¬
ticular, we appreciate the keen clinical insight of
the specialist physicians from the professional re¬
view organizations with whom we consulted
throughout this study. We also acknowledge the
many contributions of Harry Savitt, PhD, project
officer from the Office of Research and Demonstra¬
tions of the Health Care Financing Administration
of the US Department of Health and Human Ser¬
vices, whose administrative skills, astute commen¬
tary, and continuing support helped us immeasur¬
ably. Stanley S. Bentow, MS; Maureen Carney,
MS; Patricia A. Damiano; Carole Chew, RRA,
MPH; Linda Ferguson; Caren Kamberg, MSPH;
Nancy J. Palmer; and Carol Roth, RN, MPH, were
key contributors to project development and imple¬
mentation. Andrea Steiner, MS, gave valuable edi¬
torial advice. Our policy advisory board (John C.
Beck, MD; Barbara J. Burns, PhD; Monroe T. Gil-
mour, MD; Paul F. Griner, MD; Charlene Harring¬
ton, RN, PhD; T. Reginald Harris, MD; Rosalie
Kane, DSW; Shirley Kellie, MD; Judith R. Lave,
PhD; Charles E. Lewis, MD; Joseph Martin; Fran¬
cis D. Moore, Jr, MD; Richard N. Pierson, Jr, MD;
and James F. Rodgers, PhD) provided astute ad¬
vice and guidance. We acknowledge the important
contributions of the 297 hospitals whose medical
records we reviewed. Without the efforts of these
individuals and institutions, this evaluation could
not have been successfully completed. Finally, we
thank Florence McGinty, without whose secretari¬
al skills this article would not have been produced.
for Assessing Quality of Care for Hospitalized Pa-
tients With Hip Fracture. Santa Monica, Calif:
RAND Corp; 1988. N-2800-HCFA.
12. Roth CP, Kahn KL, Sherwood MJ, et al. Medi-
cal Record Abstraction Form and Guidelines for
Assessing Quality of Care for Hospitalized Pa-
tients With Pneumonia. Santa Monica, Calif:
RAND Corp; 1988. N-2801-HCFA.
13. Nunnally JC. Assessment of reliability. In:
Nunnally JC, ed. Psychometric Theory. New York,
NY: McGraw-Hill International Book Co; 1978:193\x=req-\
197.
14. Keeler EB, Kahn KL, Draper D, et al. Changes
in sickness at admission following the introduction
of the prospective payment system. JAMA. 1990;
264:1962-1968.
15. Griner PF. Medical intensive care in the teach-
ing hospital: costs versus benefits: the need for an
assessment. Ann Intern Med. 1973;78:581-585.