Surg Endosc (2013) 27:854863
DOI 10.1007/s00464-012-2524-9
A study of psychomotor skills in minimally invasive surgery:
what differentiates expert and nonexpert performance
Erlend Fagertun Hofstad Cecilie Vapenstad
Magdalena Karolina Chmarra Thomas Lang
Esther Kuhry Ronald Marvik
Received: 17 February 2012 / Accepted: 27 July 2012 / Published online: 6 October 2012
Springer Science+Business Media, LLC 2012
Abstract
Background A high level of psychomotor skills is
required to perform minimally invasive surgery (MIS)
safely. To assure high quality of skills, it is important to be
able to measure and assess these skills. For that, it is
necessary to determine aspects that indicate the difference
between performances at various levels of proficiency.
Measurement and assessment of skills in MIS are best done
in an automatic and objective way. The goal of this study
was to investigate a set of nine motion-related metrics for
their relevance to assess psychomotor skills in MIS during
the performance of a labyrinth task.
Methods Thirty-two surgeons and medical students were
divided into three groups according to their level of
experience in MIS; experts ([500 MIS procedures),
E. F. Hofstad (&)  C. Vapenstad  T. Lang
Department of Medical Technology, SINTEF Technology
and Society, PB 4760 Sluppen, 7465 Trondheim, Norway
e-mail: erlend.hofstad@sintef.no
M. K. Chmarra
Department of Circulation and Medical Imaging, Faculty
of Medicine, Norwegian University of Science and Technology
(NTNU), Trondheim, Norway
E. Kuhry  R. Marvik
Department of Cancer Research and Molecular Medicine,
Faculty of Medicine, Norwegian University of Science
and Technology (NTNU), Trondheim, Norway
E. Kuhry  R. Marvik
Department of Surgery, St. Olavs Hospital, Trondheim
University Hospital, Trondheim, Norway
E. Kuhry  R. Marvik
National Center for Advanced Laparoscopic Surgery, St. Olavs
Hospital, Trondheim University Hospital, Trondheim, Norway
123
and Other Interventional Techniques
intermediates (31500 MIS), and novices (no experience in
MIS). The participants performed the labyrinth task in the
D-box Basic simulator (D-Box Medical, Lier, Norway).
The task required bimanual maneuvering and threading a
needle through a labyrinth of 10 holes. Nine motion-related
metrics were used to assess the MIS skills of each
participant.
Results Experts (n = 7) and intermediates (n = 14) per-
formed significantly better than the novices (n = 11) in
terms of time and parameters measuring the amount of
instrument movement. The experts had significantly better
bimanual dexterity, which indicated that they made more
simultaneous movements of the two instruments compared
to the intermediates and novices. The experts also per-
formed the task with a shorter instrument path length with
the nondominant hand than the intermediates.
Conclusions The surgeons performance in MIS can be
distinguished from a novice by metrics such as time and
path length. An experienced surgeon in MIS can be dif-
ferentiated from a less experienced one by the higher
ability to control the instrument in the nondominant hand
and the higher degree of simultaneous (coordinated)
movements of the two instruments.
Keywords Assessment  Box trainer  Minimally
invasive surgery  Motion analysis  Psychomotor skills 
Training
An increasing part of surgical interventions is done by
minimally invasive surgery (MIS), such as laparoscopy [1].
Compared to traditional open surgery, a new set of psy-
chomotor skills is required to perform MIS safely, in
addition to skills related to knowledge and judgment [2, 3].
Surgical simulators, both virtual reality (VR) and analog
Surg Endosc (2013) 27:854863
box trainers, are already an important part of surgical
training [4, 5]. Training on simulators outside of the
operating room is risk-free for the patient. It has the
potential to be fair and reliable for all participants with
identical setups, and to objectively assess surgical perfor-
mance. However, it is essential to make sure that the skills
learned on the simulator actually improve surgical perfor-
mance, by establishing predictive validity [6, 7]. To
determine predictive validity and, in general, to assess MIS
skills, a participants performance can be evaluated, for
example, during surgery using assessment methods such as
the Global Operative Assessment of Laparoscopic Skills
(GOALS) [8]. Still, there is a demand for more automatic
assessment methods, which are objective and demand less
effort from experienced surgeons in the evaluation process.
An automatic assessment method requires a set of
performance metrics. A great deal of research has been
performed to identify the psychomotor skills in laparo-
scopic surgery that discriminate between different levels
of surgical proficiency [9]. Similar studies have also been
performed regarding training for robotic surgery [10, 11].
Finding metrics that define the quality of a surgical
performance is not merely useful in the validation of the
effect of the training tool. It is also important for pro-
viding objective feedback to the user during training on
simulators. Whether the results presented by the simulator
reflect the real quality of the performance, i.e., the sim-
ulator has construct validity, can be found by checking
whether the built-in assessment tools are able to distin-
guish the performance of surgeons with different levels of
experience [2]. When a training curriculum on a simu-
lator is developed and validated, the residents can train
on the simulator until a proficiency-based level is
reached. This takes into account the variations in the
participants learning curves, in contrast to traditional
time-based training [12]. The simulator has the potential
to indicate when a resident is ready for the next level of
training, and tests on the simulators can be used in a
certification scheme to determine when the resident has
attained the psychomotor skills required to perform sur-
gery on patients. Looking further into the future, moni-
toring the motions of, e.g., laparoscopic instruments in
the operating room could provide improved patient safety
by indicating potential harmful adverse events before
their occurrence.
The goal of this study was to investigate a set of motion-
related metrics for their relevance to assess psychomotor
skills in MIS. Most of the metrics have previously been
presented elsewhere [9]. However, the formula for calcu-
lation bimanual dexterity was defined in this study. Nev-
ertheless, the novelty of the study is not limited to
identifying performance measures that have not previously
been published. Every training setup and tasks need to be
855
validated to ensure relevant skills, in this case for mini-
mally invasive surgery, are trained and assessed. To our
knowledge, such a validation has not previously been
published for a similar setup as the one presented here: a
labyrinth task in an analog box trainer. Laparoscopic sur-
geons with various levels of experience participated in the
study, along with medical students and interns. The
assessment was based on the time to complete the task and
eight metrics calculated from the movements of the lapa-
roscopic instruments. The instruments were tracked by an
electromagnetic tracking system.
Materials and methods
Participants
The participants were divided into three groups on the basis
of their experience level, indicated in a questionnaire
stating their experience and dominant hand. Experience
was measured by the number of laparoscopic procedures
they had performed. Experts had performed 500 or more
laparoscopic procedures; intermediates, 31500 proce-
dures; and the participants without laparoscopic experience
were novices. The novices were recruited from medical
students in the 5th and 6th year of study and interns in their
1st year of practice after medical school.
Task
In order to measure the participants psychomotor skills,
they were asked to perform a task in a box trainer: D-box
Basic simulator (D-Box Medical, Lier, Norway). A laby-
rinth exercise was performed where they had to maneuver
and thread a needle through a loop of 10 holes (Fig. 1). The
needle was controlled with laparoscopic graspers (Endo
Clinch II, AutoSuture, Covidien, Mansfield, MA, USA),
one in each hand. The position of the tip of the two
instruments was followed throughout the exercise using
electromagnetic position tracking. The time to complete
the exercise was measured as well. A camera was placed
inside the box trainer, and the participants navigated the
instruments by viewing the images displayed on a screen
placed in front of them. The camera remained in a fixed
position during the test so that it showed the entire
labyrinth.
Before starting the test, each participant was briefly
instructed on how to perform the task. All participants
performed the test twice, where the first attempt was con-
sidered a warm-up. The results from the second trial were
used to study the differences in performance between
experts, intermediates, and novices.
123
856 Surg Endosc (2013) 27:854863

Fig. 1 Experimental setup. The box trainer with the labyrinth

exercise seen on the monitor, and the electromagnetic field generator
located on the left side of the box trainer (photo by Pall Jens
Reynisson)

Position tracking

The position of the instruments were tracked with the Fig. 2 Definition of the orientation of the laparoscopic grasper
Aurora Electromagnetic Measurement System (Northern
Digital Inc., Waterloo, ON, Canada). The system consists axis (Fig. 2). The following nine metrics were used in the
of a system control unit, maximum four system interface evaluation of the performance of the participants:
units for position sensor inputs, and a field generator that (1) (1) Time (T) Time from start to completion of the
generates an electromagnetic tracking volume. task, measured in seconds.
A positioning sensor with six degrees of freedom was (2) Bimanual dexterity (BD) The participants ability to
placed close to the tip of the laparoscopic instruments and a control two instruments at the same time. BD is found
reference sensor was placed inside the box trainer. The by calculating the correlation between the velocity of
sensors measured positions with an accuracy of 0.48 mm, the tip of the instruments controlled by the left and the
according to the manufacturer. The positions were sampled right hand:
at a frequency of 40 Hz. PN   
n1 vleft n  v left vright n  vright
BD q
PN   2 PN  2
Evaluation metrics
n1 vleft n  v left n1 vright n  v right

The acquired motion data were analyzed by in-house where v is the velocity of the instruments and v denotes
developed motion analysis software. The software was the average velocity over the duration of the exercise.
written in Matlab 7 (MathWorks, MA, USA). (3) Path length (PL) Total movement of the tip of the
In order to eliminate high-frequency background noise, instrument for the duration of the task, measured in
the motion data were filtered with a sixth-order low-pass meters:
Butterworth filter with cutoff frequency at 6 Hz. s
All motion-related metrics were derived from the posi- ZT  2  2  2
dx dy dz
tion and the orientation of the instruments. The position PL dt
dt dt dt
was defined by xt; yt; ztTt0 . The orientation was 0

defined by at; bt; ctTt0 , where a and b measure the (4) Angular length (AL) The total change in the angle of
orientation in two planes perpendicular to the instruments the tip of the instrument in the plane perpendicular to
axis and c measures the rotation around the instruments the instruments axis, measured in degrees:

123
Surg Endosc (2013) 27:854863 857

s

  2 Results
ZT
da 2 db
AL dt
dt dt A total of 32 participants were included in the study: 7
0
experts, 14 intermediates and 11 novices. Five participants
(5) Depth perception (DP) DP is calculated by the total were women (1 intermediate and 4 novices), and 3 were
distance traveled by the tip of the instrument in the left-handed (1 intermediate and 2 novices).
instruments axis direction, measured in meters: Figure 3 shows box plots of the results from the three
groups for all the nine parameters included in the motion
ZT s
dz analysis. Separate values for the instrument controlled by
DP  dt
 dt  the dominant and the nondominant hand are presented for
0 seven of the parameters. Time and bimanual dexterity have
(6) Response orientation (RO) The total amount of the same value for both hands.
instrument rotation around its axis, measured in The P-values comparing the results of each pair of the
degrees: three groups of participants are presented in Table 1. Sta-
tistical significant differences were obtained between
ZT s experts and novices (EN in Table 1) for seven metrics:
dc
RO  dt time, bimanual dexterity, path length, angular length, depth
 dt 
0
perception, response orientation, and number of submov-
ements. Between the intermediates and novices (IN in
(7) Motion smoothness (MS) The total change in Table 1), significance was found for six metrics: time, path
acceleration of the tip of the instrument. The motion length, angular length, depth perception, response orien-
smoothness is measured in m/s3, and is normalized by tation (only for the dominant hand), and number of sub-
the duration of the task: movements. The results of the experts and intermediates
v (EI in Table 1) show that there are three parameters with
u ZT
u significant differences: bimanual dexterity, path length, and
J u1 d3 x
MS ; J t j2 dt; j 3 depth perception. For path length and depth perception,
T 2 dt
0 significance was found only for the nondominant hand.
For bimanual dexterity, the intermediates and the nov-
(8) Number of submovements (NoS) A submovement is ices performed at approximately the same level, as shown
defined by a movement of the tip of the instrument in Fig. 3, while the experts were superior in simultaneous
containing a velocity peak of at least 15 mm/s. use of both hands. This is in contrast to most of the other
(9) Average velocity (AV) The average velocity of the tip parameters, where both the expert and the intermediates
of the instrument for the duration of the task, outperformed the novices.
measured in mm/s. The novices scored significantly lower values for the
Bimanual dexterity is a subjective metric of the nondominant hand for the motion smoothness compared to
GOALS [8] assessment method. To make it more the experts and intermediates. No statistically significant
objective, the formula for calculating the bimanual dex- difference between the three groups was found for the
terity from motion data was defined in this study. The average velocity.
rest of the metrics have previously been used in other The correlation between the metrics identified as mea-
studies [9]. Considerations were taken in selecting task- suring statistical significant differences is shown in
independent metrics, which can be calculated for any Table 2. Time, path length, angular length, depth percep-
laparoscopic training module. tion, and number of submovements are found to be inter-
correlated and are thus partially dependent. Response
orientation and bimanual dexterity have less dependency to
Statistical analysis the other metrics and to each other.

The statistical analysis was performed by SPSS software,

version 19 (SPSS Inc., Chicago, IL, USA). The Mann
Whitney U-test was performed, and a probability of
P \ 0.05 was considered statistically significant. In order
to find correlation between the metrics, Pearsons test was
applied to the metrics in which statistically significance
was identified with the MannWhitney U-test.
Discussion
Our goal was to find a set of performance metrics by means
of motion analysis of the laparoscopic instrumentsmet-
rics that can be used to objectively assess psychomotor
skills on the basis of surgical proficiency. A total of 32

123
858 Surg Endosc (2013) 27:854863

A B

C D

E F

Fig. 3 AP Results of the labyrinth test presented as box plots of the whiskers show the fifth and 95th percentile. Outliers are plotted with
9 metrics. The middle band shows the median value; the bottom and circles and extreme outliers are marked with asterisks
top of the boxes show the 25th and 75th percentile; and the ends of the

participants were included in the study, divided into three
groups: experts (n = 7), intermediates (n = 14), and nov-
ices (11).
The participants proficiency level was determined by
self-assessment of the number of laparoscopic procedures
they had performed. One might argue that self-judgment of
competence can be unreliable [13], and that having per-
formed more procedures does not necessarily imply higher
psychomotor skills [14]. The intermediates, who had per-
formed up to 500 procedures, might already have reached
the highest level of skills. However, the lack of an external
objective assessment of the skills of each of the participants
left us with the number of procedures performed as the
indication of competence level [15].
Statistically significant differences in performance were
found for seven of the nine metrics when comparing the

123
Surg Endosc (2013) 27:854863 859

G H

I J

K L

Fig. 3 continued

group of experts to the group of novices, six when com-
paring intermediates to novices, and three when comparing
experts to intermediates. The participants with laparoscopic
experience (experts and intermediates) were superior to the
novices considering the time to complete the task and the
instrument movement metrics; the total traveled distance of
the instruments (path length), the number of submov-
ements of the instruments, the total traveled distance along
the instruments axis (depth perception), the amount of
movement perpendicular to the instruments axis (angular
length), and the total rotation of the instruments (response
orientation). These results are in accordance with other
studies of similar tasks, i.e., transfer of objects with lapa-
roscopic instruments, performed either by using box
trainers or VR simulators. Pellen et al. [16] found construct
validity for the time and path length metrics for an object-
positioning task on the ProMIS simulator. Woodrum et al.
[17], in addition to time and path length, found construct

123
860 Surg Endosc (2013) 27:854863

M N

O P

Fig. 3 continued

Table 1 Results of the descriptive statistics on the metrics of the three study groups (P-values)
EI IN EN

Time 0.2 0.001* 0.001*

Bimanual dexterity 0.01* 0.1 0.02*
Dominant hand Nondominant hand
EI IN EN EI IN EN

Path length 0.09 0.001* 0.001* 0.04* 0.001* 0.001*

Angular length 0.4 0.002* 0.002* 0.3 0.03* 0.004*
Depth perception 0.2 0.001* 0.001* 0.04* 0.001* 0.001*
Response orientation 0.7 0.006* 0.01* 0.3 0.1 0.04*
Motion smoothness 0.7 0.07 0.06 0.5 0.03*a 0.02*a
Number of submovements 0.2 0.002* 0.002* 0.2 0.001* 0.001*
Average velocity 0.8 0.6 0.7 0.9 0.9 0.7

E expert, I intermediate, N novice

* Significant at P \ 0.05
a
Statistical significance was found, but the group with the lowest values was the opposite of the expected result

validity for the angular length metric (right instrument significant differences in the performance on tests per-
only) for the Lifting and Grasping module on the Lap- formed in a box trainer for time, path length, and depth
Sim VR simulator. Chmarra et al. [15] measured statistical perception.

123
Surg Endosc (2013) 27:854863 861

Table 2 Correlation between the metrics identified as measuring statistically significant differences
Hand Time Bimanual Path Angular Depth Response Number of
dexterity length length perception orientation submovements

Dominant hand
Time 100 44 95 92 94 63 97
Bimanual dexterity 44 100 38 30 38 11 38
Path length 95 38 100 92 100 67 99
Angular length 92 30 92 100 90 62 93
Depth perception 94 38 100 90 100 68 99
Response orientation 63 11 67 62 68 100 68
Number of submovements 97 38 99 93 99 68 100
Nondominant hand
Time 100 44 95 72 95 43 98
Bimanual dexterity 44 100 43 42 45 38 40
Path length 95 43 100 81 100 49 95
Angular length 72 42 81 100 82 53 67
Depth perception 95 45 100 82 100 49 95
Response orientation 43 38 49 53 49 100 36
Number of submovements 98 40 95 67 95 36 100
Pearsons test was used to calculate the correlation matrix for the metrics identified as statistically significant in Table 1. The correlations are
calculated for the dominant (upper) and the nondominant (lower) hand separately

Statistically significant differences were found for three
metrics when comparing experts to intermediates: bimanual
dexterity, path length, and response orientation. For the latter
two metrics, significance was found only for the nondomi-
nant hand. The bimanual dexterity measures the ability to
control both instruments simultaneously, which requires a
high level of control of the nondominant hand, as well as the
dominant hand. This ability appears not to be fully devel-
oped among the participants in the intermediate group. The
intermediates are moving the instrument in the nondominant
hand more than the experts, but they have less simultaneous,
coordinated movements. The results of this study suggest
that the control of the nondominant hand and the higher
degree of simultaneous hand movements are metrics that
differentiate between surgeons who have performed less
than 500 laparoscopic procedures from those who have
performed more than 500 procedures. The skill of simulta-
neous and coordinated hand movement appears not to be
fully acquired until a later stage in the learning process,
which is supported by the fact that the intermediates did not
outperform the novices in bimanual dexterity.
By investigating the correlation of the results, most of
the metrics were found to be strongly dependent. However,
bimanual dexterity (and response orientation) had a lower
correlation to other metrics. This supports the fact that
bimanual dexterity is measuring a different aspect of psy-
chomotor skills than the traditionally parameters like time
and path length.
The main purpose of assessing a surgeons psychomotor
skills is to assure the quality of surgical performance by
quantifying performance levels, which can be used to
evaluate the effect of training or objectively assess dex-
terity as part of a certification test [9]. Even though
bimanual dexterity separates the performance of the most
experienced laparoscopic surgeons from the ones with less
experience, care should be taken in purely assessing psy-
chomotor skills on the basis of the bimanual dexterity
metric. It would be easy to improve the score of the metric
by moving both hands at the same time, without improving
the efficiency or outcome of the procedure. Nevertheless, it
might be beneficial for the laparoscopic surgeons who have
reached a certain level of dexterity to focus more of their
psychomotor skills training on the nondominant hand, i.e.,
by the use of simulators with modules that exercise the
control of both hands, including simultaneous hand
movements.
In general, it is expected that the more experienced
surgeons have smoother motions and thus obtain lower
values for the motion smoothness metric, compared to the
less experienced ones. However, the results showed the
opposite; the group of novices attained the lowest motion
smoothness. This could be related to the nature of the
exercise, where the participants are motivated to pass the
needle through the labyrinth of holes in the most efficient
manner possible, and the exercise required larger move-
ments compared to normal surgery. The more experienced

123
862
surgeons performed the exercise the quickest and were thus
likely to have moved the instruments between the holes in a
rapid manner that increased the motion smoothness value.
Because no penalty points were introduced for committed
errors (e.g., bumping the instruments into obstacles or
dropping the needle), we expect that all the participants
tried to perform the task as fast as possible. Introducing
penalty points might influence performance of the task
because the participants would have to take into account
the preferred manner of performing the task. Including
penalty points would require an observer evaluating the
performance, either during the task or by video assessment.
Error scores are, however, common on VR simulators,
where they are automatically detected by the computer
[18]. Our result is similar to the findings of Chmarra et al.
[19] and Maithel et al. [20], neither of whom was able to
separate the groups of different levels of experience by the
motion smoothness metric.
No statistically significant difference was found between
the groups for the average velocity of the tip of the
instruments. This is consistent with the results found for the
time and path length. The novices needed more time to
complete the task and moved the instruments more than the
experts and intermediates. The more experienced surgeons
moved the instruments in a more efficient manner, but
those movements were not necessarily faster than those of
the inexperienced participants.
To establish a passfail rate in a certification scheme,
it might not always be necessary to know what dis-
criminates between a surgeon at an expert level and the
one at an intermediate level, as they both normally have
skills sufficient to perform surgery safely. It is, however,
important to be able to recognize when residents are
ready to move to the next training level, or which sur-
geons have superior skills that allow them to perform
specific surgeries. Knowledge of the expert level of
performance is important when evaluating training cur-
ricula. This knowledge is also essential in the validation
process of simulators. When determining construct
validity, the built-in assessment metrics are tested in
order to find out if they are appropriate to measure dif-
ferences in performance level. For instance, one can
assess whether the most experienced surgeons outperform
the less experienced ones by comparing results provided
by the simulator. Because it is essential to ensure that
correct feedback is given to the ones who train on the
simulator, it is necessary to further investigate what that
feedback should contain and how it should be presented
to them. Ideally, once construct validity of the training
setup is established, indication on the skills needed to be
improved and a list of suitable training modules to
improve those skills should be presented after a training
session.
123
Surg Endosc (2013) 27:854863
Conclusions
Motion analysis is a valid objective way of assessing
psychomotor MIS skills. Experts and intermediates out-
performed novices for six out of nine tested metrics. The
difference between experts and intermediate was smaller;
separation between the two groups was found for three
metrics. An experienced MIS surgeon can be distinguished
from a less experienced one by the higher ability to control
the instrument in the nondominant hand and the higher
degree of simultaneous (coordinated) movements of the
two instruments. This finding indicates that less experi-
enced surgeons could benefit from training nondominant
hand dexterity and bimanual dexterity.
Acknowledgments The authors thank the surgeons, residents,
medical students, and interns for participating in this study.
Disclosures Erlend F. Hofstad, Cecilie Vapenstad, Magdalena K.
Chmarra, Thomas Lang, Esther Kuhry, and Ronald Marvik have no
conflicts of interests or financial ties to disclose.
References
1. Hallam DM, Anwar MA, Khan IM, Memon MA, Steele K, Lidor
A (2008) Access to the abdomen. In: Scott-Conner CEH (ed) The
SAGES manual of strategic decision making. Springer, New
York, pp 19
2. Feldman LS, Sherman V, Fried GM (2004) Using simulators to
assess laparoscopic competence: ready for widespread use? Sur-
gery 135(1):2842
3. Subramonian K, DeSylva S, Bishai P, Thompson P, Muir G
(2004) Acquiring surgical skills: a comparative study of open
versus laparoscopic surgery. Eur Urol 45(3):346351
4. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR (2009)
Virtual reality training for surgical trainees in laparoscopic sur-
gery. Cochrane Database Syst Rev (1):CD006575
5. Basdogan C, Sedef M, Harders M, Wesarg S (2007) VR-based
simulators for training in minimally invasive surgery. IEEE
Comput Graph Appl 27(2):5466
6. Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK,
Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for val-
idation of virtual reality surgical simulators. Surg Endosc 19(12):
15231532
7. Gallagher AG, Ritter EM, Satava RM (2003) Fundamental
principles of validation, and reliability: rigorous science for the
assessment of surgical education and training. Surg Endosc
17(10):15251529
8. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K,
Stanbridge D, Fried GM (2005) A global assessment tool for evalu-
ation of intraoperative laparoscopic skills. Am J Surg 190(1):107113
9. Oropesa I, Sanchez-Gonzalez P, Lamata P, Chmarra MK, Pag-
ador JB, Sanchez-Margallo JA, Sanchez-Margallo FM, Gomez EJ
(2011) Methods and tools for objective assessment of psycho-
motor skills in laparoscopic surgery. J Surg Res 171(1):e81e95
10. Narazaki K, Oleynikov D, Stergiou N (2006) Robotic surgery
training and performance: identifying objective variables for
quantifying the extent of proficiency. Surg Endosc 20(1):96103
11. Judkins TN, Oleynikov D, Stergiou N (2009) Objective evalua-
tion of expert and novice performance during robotic surgical
training tasks. Surg Endosc 23(3):590597
Surg Endosc (2013) 27:854863
12. Satava RM (2008) Historical review of surgical simulationa
personal perspective. World J Surg 32(2):141148
13. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe
KE, Perrier L (2006) Accuracy of physician self-assessment
compared with observed measures of competence: a systematic
review. JAMA 296(9):10941102
14. Schout BM, Hendrikx AJ, Scheele F, Bemelmans BL, Scherpbier AJ
(2010) Validation and implementation of surgical simulators: a crit-
ical review of present, past, and future. Surg Endosc 24(3):536546
15. Chmarra KM, Klein S, de Winter F, Jansen JC, Dankelman FW et al
(2010) Objective classification of residents based on their psy-
chomotor laparoscopic skills. Volume 24. Springer, Heidelberg
16. Pellen MG, Horgan LF, Barton JR, Attwood SE (2009) Construct
validity of the ProMIS laparoscopic simulator. Surg Endosc
23(1):130139
863
17. Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L,
Gauger PG, Minter RM (2006) Construct validity of the LapSim
laparoscopic surgical simulator. Am J Surg 191(1):2832
18. Thijssen AS, Schijven MP (2010) Contemporary virtual reality
laparoscopy simulators: quicksand or solid grounds for assessing
surgical trainees? Am J Surg 199(4):529541
19. Chmarra MK, Kolkman W, Jansen FW, Grimbergen CA, Dank-
elman J (2007) The influence of experience and camera holding
on laparoscopic instrument movements measured with the
TrEndo tracking system. Surg Endosc 21(11):20692075
20. Maithel SK, Villegas L, Stylopoulos N, Dawson S, Jones DB
(2005) Simulated laparoscopy using a head-mounted display vs
traditional video monitor: an assessment of performance and
muscle fatigue. Surg Endosc 19(3):406411
123