Running Head: Multivariate Dynamic Criteria Job Performance as Multivariate Dynamic Criteria: Experience Sampling and Multiway Component Analysis By Seth M. Spain, University of Illinois, Urbana-Champaign Andrew G. Miner, Target Corp. Pieter M. Kroonenberg, Leiden University Fritz Drasgow, University of Illinois, Urbana-Champaign
Author Notes.
Portions of this research were presented at the 2007 Annual Meeting of the Academy of Management, Philadelphia, PA. This manuscript is based on the first author’s master’s thesis. Correspondence and requests for reprints should be addressed to Seth Spain, Department of Psychology, 603 E. Daniel St., Champaign, IL, 61820 or by email at sspain@illinois.edu The authors would like to thank Chuck Hulin, Sungjin Hong, and Theresa Glomb for their assistance and for comments on earlier drafts of this manuscript.
PAPER PUBLISHED IN
MULTIVARIATE BEHAVIORAL RESEARCH
, VOLUME 45, ISSUE 4, PAGES 599 – 626.
ABSTRACT
Questions about the dynamic processes that drive behavior at work have been the focus of increasing attention in recent years. Models describing behavior at work and research on momentary behavior indicate that substantial variation exists within individuals. This paper examines the rationale behind this body of work and explores a method of analyzing momentary work behavior using experience sampling methods. The paper also examines a previously unused set of methods for analyzing data produced by experience sampling. These methods are known collectively as multiway component analysis. Two archetypal techniques of multimode factor analysis, the Parafac and the Tucker3 models, are used to analyze data from Miner, Glomb, & Hulin’s (in press) experience sampling study of work behavior. The efficacy of these techniques for analyzing experience sampling data is discussed, as are the substantive multimode component models obtained.
Keywords:
Three-way principal components analysis, Parafac model, Tucker3 model, experience sampling, dynamic criteria Job Performance as Multivariate Dynamic Criteria: Experience Sampling and Multiway Component Analysis Historically, measures of job performance have been of great interest for organizational scientists attempting to establish the validity of selection systems (Austin & Villanova, 1992). For instance, if a city uses a cognitive ability measure to select their police officers, applied psychologists want to validate those cognitive ability scores against a criterion such as the new officers’ arrest records, where arrest records serve as a measure of job
performance. Validity for predicting a criterion is commonly evaluated via the correlation between the predictor measure and criterion. For the police example, this would be the correlation between the cognitive ability scores and the arrest record. Of central interest to applied psychologists is how to define job performance, both conceptually and operationally (e.g., Austin & Villanova, 1992; Borman, 1991). Most validation research treats job performance as a monolithic and static construct. There is considerable empirical evidence that job performance is multidimensional (e.g., Campbell, 1991; Campbell, 1994; Campbell, McHenry, & Wise, 1990; cf., Borman & Motowidlo, 1997). Furthermore, it is possible that job performance is not stable over time (Hulin, Henry, & Noon, 1990; Keil & Cortina, 2001). In fact, job performance data can usually be classified by three modes: the individuals assessed, the variables measured, and the times of measurement (e.g., Cattell, 1952; cf., Smith, 1976). Ghiselli (1956) postulated three systematic sources of variance in job performance data, i.e. all three modes show multidimensionality. Dalal and Hulin (2008) have called for job performance studies which include changes over time, referring to this approach as
multivariate dynamic
. Even this perspective ignores qualitative differences in performance and thus potential dimensionality in the individuals mode. This paper will deal with multidimensionality of job performance in all its modes and takes an individual differences perspective to multivariate dynamic aspects of job performance. In order to validly measure the frequency and the patterning of mental processes in everyday-life situations procedures are needed which capture variations in self-reports of those processes. To this end,
experience sampling methodology
has been developed in which a participant at random or specific times has to report on his or her mental state or those activities in which he or she is involved at that moment. To capture those reporting instances participants are supplied with beepers, or more recently with palmtop computers. With the help of these devices, several brief surveys each day are administered to participants (Larson & Csikszentmihalyi, 1983; Csikszentmihalyi & Larson, 1987). As experience sampling allows information to be gathered from individuals about several variables over time, the procedure provides data to study Ghiselli’s three sources of job performance variance. A number of methods for analyzing such experience sampling data have been used in the literature, in particular spectral analysis and multilevel modeling. This paper will demonstrate techniques that have not been frequently used in the organizational literature, and explore their usefulness for understanding experience sampling data.
Recompense su curiosidad
Todo lo que desea leer.
En cualquier momento. En cualquier lugar. Cualquier dispositivo.
Sin compromisos. Cancele cuando quiera.