Está en la página 1de 1178

ICAO - Human Factors

Flight Safety Human Factors


The publications on this page are free and can be downloaded directly from this web site by clicking the langauage
icon next to the title (EN= English; FR=French; ES=Spanish)

Articles from the ICAO Journals


Table of Contents
Top Ten Recommended Introductory Reading List
of Free ICAO Material
Accident / Incident
Avionics
Crew Resource Management
Fatigue
Flight Operational Quality Assurance (FOQA), Line Operations Safety Audit (LOSA),
Normal Operations Safety Survey (NOSS), & Threat and Error Management (TEM)
Language
Maintenance
General Seminars
Training
Miscellaneous
Cross-Cultural Issues

Top Ten Recommended Introductory Reading List of Free ICAO Material

Article Language
Degani, A., & Wiener, E. (1995). Designing coherent flight-deck procedures for use in advanced technology
aircraft. ICAO Journal, 50(2), 23-25.
ICAO Secretariat (1996). Awareness grows of importance of human factors issues in aircraft maintenance and
inspection. ICAO Journal, 51(1), 19-21, 24.
ICAO Secretariat (2003). Effectiveness of security inspections depends on human proficiency. ICAO Journal,
58(1), 21-24.
MacBurnie, E. (1993). Human Factors in Aviation. ICAO Journal, 48(7), 7-26.
MacBurnie, E. (1996). Flight-Deck Automation: A pilot's perspective. ICAO Journal, 51(5), 5-30.
MacBurnie, E. (1996). Human Factors in Aviation. ICAO Journal, 51(8), 4-29.
MacBurnie, E. (1999). Human Factors Today: managing human error. ICAO Journal, 51(8), 5-30.
Maurino, D. (2000). ICAO human factors programme expands scope beyond flight deck and ATC facility.
ICAO Journal, 55(1), 15-16, 29.
MacBurnie, E. (2002). The LOSA experience: safety audits on the flight deck. ICAO Journal, 57(4), 5-31.
MacBurnie, E. (2004). Aviation Language Proficiency. ICAO Journal, 59(1), 4-27.

Accident / Incident

http://www.icao.int/ANB/humanfactors/icaojournallist.htm (1 of 5)16-02-2007 22:36:23


ICAO - Human Factors

Article Language
Australian Transport Safety Bureau (2005). Improvised approach has catastrophic consequences for Ilyushin
76 freighter. ICAO Journal, 60(1), 22-25, 30.
Bureau of Air Safety Investigation (1995). Australian accident report includes examination of organizational
and management failures, ICAO Journal, 50(7), 9-12, 24.
Matthews, R. (2004). Ramp accidents and incidents constitute a significant safety issue. ICAO Journal, 59
(3), 4-6, 25.
Maurino, D. (1993). Efforts to reduce CFIT accidents should address failures of the aviation system itself.
ICAO Journal, 48(4), 18-19.
Maurino. D. (1997). Many safety analysts perceive need to apply proactive safety approach to investigations.
ICAO Journal, 52(2), 21-22, 29.
Maurino, D. (1998). Full integration of human factors knowledge in investigation process would further
enhance safety. ICAO Journal, 53(3), 14-15, 25.
Maurino, D., Reason, J., Johnston, N., Lee, R. (1995). Exploring the role of the transportation system and
human factors in the crash of Flight 1363. ICAO Journal, 50(7), 14-17, 26.
Maurino, D., Reason, J., Johnston, N., Lee, R. (1995). Six years after the Dryden tragedy, many accident
investigation authorities have learned its lessons. ICAO Journal, 50(8), 20-25.
National Transportation Safety Board (2002). Report explains accident involving MD-82 overrun after
landing on a wet runway. ICAO Journal, 57(1), 6-10, 24-26.
Paries, J. (1994). Investigation probed root causes of CFIT accident involving a new-generation transport.
ICAO Journal, 49(6), 37-41.
Transportation Safety Board of Canada (2000). Safety Board issues report on crews loss of control on go-
around attempt. ICAO Journal, 55(1), 18-20, 27-28.

Avionics

Article Language
Boorman, D. (2001). Todays electronic checklists reduce likelihood of crew errors and help prevent
mishaps. ICAO Journal, 56(1), 17-20, 36.
Clark, L. (1997). Avionics incorporating human-centered design improves pilot interface with automated
system. ICAO Journal, 52(4), 11-12.
Degani, A., & Wiener, E. (1995). Designing coherent flight-deck procedures for use in advanced technology
aircraft. ICAO Journal, 50(2), 23-25.
MacBurnie, E. (1996). Flight-Deck Automation: A pilot's perspective. ICAO Journal, 51(5), 5-30.
Maurino, D. (1998). ICAO supports proactive approach to managing human factors issues related to advanced
technology. ICAO Journal, 53(5), 17-18, 27, 29.
Newman, T. and Courtney, H. (1998). Standards for addressing human factors during aircraft certification
prove difficult to define. ICAO Journal, 53(3), 11-13, 24.
Shun, C. (2003). Ongoing research in Hong Kong has led to improved wind shear and turbulence alerts.
ICAO Journal, 58(2), 4-6, 28.
Spruston, D. (1998). A number of safety issues related to flight deck technology require our urgent attention.
ICAO Journal, 53(3), 9-10.
Tarnowski, E. (1999). Understanding design philosophy can help pilots benefit from modern automated flight
systems. 54(9), 22-24, 29-30.
Veitengruber, J., & Rankin, W. (1995). Use of crew-centred design philosophy allows the introduction of new
capabilities and technology. ICAO Journal, 50(2), 20-22.

Crew Resource Management

Article Language
Svatek, N. (1990). Techniques in CRM training. ICAO Journal, 45(10), 12-13.

http://www.icao.int/ANB/humanfactors/icaojournallist.htm (2 of 5)16-02-2007 22:36:23


ICAO - Human Factors

Taggart, W. (1987). CRM- a different approach to human factors training. ICAO Journal, 42(5), 13-16.
Weisman, G. (1991). A new partnership in CRM training. ICAO Journal, 46(10), 10-12.

Fatigue

Article Language
Chittick, J. (1998). Preferential scheduling for aircrew can help address problem of short-term accumulated
fatigue. ICAO Journal, 53(3), 16-17.
Graeber, R., Rosekind, M., Connell, L. & Dinges, D. (1990). Cockpit Napping. ICAO Journal, 45(10), 6-11.
NASA Ames Research Center. (1997). Crew fatigue research focusing on development and use of effective
countermeasures. ICAO Journal, 52(4), 20-22, 28.
Singh, J. (2003). Study of pilot alertness highlights feasibility of ultra long range flight operations. ICAO
Journal, 58(1), 14-15, 30.

Flight Operational Quality Assurance (FOQA),


Line Operations Safety Audit (LOSA),
Normal Operations Safety Survey (NOSS), &
Threat and Error Management (TEM)

Article Language
Proceedings of the Fourth ICAO-IATA LOSA and TEM Conference, Toulouse, France, 16-17 Novemeber
2006
Holtom, M. (2000). Properly managed FOQA programme represents an important safety tool for airlines.
ICAO Journal, 55(1), 7-11, 26-27.
Logan, T. (1999). Trend toward wider sharing of safety data is resisted by industry concerns, ICAO Journal,
54(1), 7-9, 27.
MacBurnie, E. (2002). The LOSA experience: safety audits on the flight deck. ICAO Journal, 57(4), 5-31.
Maurino, D. (1998). Human factors training would be enhanced by using data obtained from monitoring
normal operations. ICAO Journal, 53(1), 17-18, 23-24
Maurino, D. (2004). ICAO examining ways to monitor safety during normal ATS operations. ICAO Journal,
59(3), 14-16.
Proceedings of the First LOSA Week, Cathay City
Hong Kong, 12 to 14 March 2001
Proceedings of the First Threat and Error Management Workshop, San Salvador, El Salvador, 30 April to 1
May 2002
Proceedings of the Second LOSA Week, Panama City, Panama, 27 to 29 November 2001
Proceedings of the Third LOSA Week, Dubai United Arab Emirates, 14 t 16 October 2002
Proceedings of ICAO/ASPA Regional Seminar on Safety Reporting, Threat and Error Management (TEM)
and Cabin Safety, Mexico City, Mexico, 10 to 11 March 2004
US Airways (2003). Newly implemented line operations safety audit produces valuable data for air carrier.
ICAO Journal, 58(1), 11-12, 28-29.
First ICAO-LATA LOSA & TEM Conference Dublin, Ireland, 5 to 7 November 2003
Second ICAO-IATA LOSA & TEM Conference
Seattle, Washington, 3 to 4 November 2004

Language

Article Language

http://www.icao.int/ANB/humanfactors/icaojournallist.htm (3 of 5)16-02-2007 22:36:23


ICAO - Human Factors

Douglas, D. (2004). Language tests can promote safer communications in international civil aviation
operations. ICAO Journal, 59(3), 17-18, 25-26.
MacBurnie, E. (2004). Aviation Language Proficiency. ICAO Journal, 59(1), 4-27.

Maintenance

Article Language
Australian Transport Safety Bureau (2004). Maintenance safety deficiencies highlight need for periodic
review of safety systems. ICAO Journal, 59(3), 9-12, 24-25.
ICAO Secretariat (1996). Awareness grows of importance of human factors issues in aircraft maintenance and
inspection. ICAO Journal, 51(1), 19-21, 24.
National Transportation Safety Board (2003). Poor maintenance practices led to crash of Alaska Airlines
Flight 261. ICAO Journal, 58(2), 19-23, 30.
Purdue Unviersity (2000). Survey results suggest need for more effective reporting of aircraft maintenance.
ICAO Journal, 55(1), 17, 28-29.

General Seminars

Article Language
Egypt sponsors regional seminar on safety and human factors. (1992). ICAO Journal, 47(12), 24.
Human factors in spotlight at second regional safety seminar. (1991). ICAO Journal, 46(12), .24.
Human Factors the focus of milestone Leningrad seminar. (1990) ICAO Journal, 45(5) 28.
Maurino, D. (1994). ICAO workshops gather important knowledge on perceptions of human factors training.
ICAO Journal, 49(3), 22-23.
Stewart, J. (2001). Safety seminars planned for regions where maximum safety gains may be achieved. ICAO
Journal, 56(1), 21, 36-37

Training

Article Language
First Iberoamerican Conference on Safety and Training in Civil Aviation, Madrid, Spain 5 to 7
November 2002. (Spanish version only)
Johnston, A., & Maurino, D. (1990). Human Factors training for aviation personnel. ICAO Journal, 45(5), 16-
19.
Maurino, D. (1995). ICAO annex amendment introduces mandatory human factors training for airline flight
crews. ICAO Journal, 50(7), 13, 24-25.
Orlady, H. (1994). Airline pilot training programmes have undergone important and necessary changes in the
past decade. ICAO Journal, 49(3), 5-10.

Miscellaneous

Article Language
Courville, B. & Thisselin, J. (2004). Applying take-off thrust on unsuitable pavement surface may have
hidden dangers. ICAO Journal, 59(3), 7-8.
Edwards, C. (2000). Aircraft operators have built a generic hazard model for use in developing safety cases.
ICAO Journal, 55(1), 12-14, 27.

http://www.icao.int/ANB/humanfactors/icaojournallist.htm (4 of 5)16-02-2007 22:36:23


ICAO - Human Factors

Foreman, P. (1998). Proposed free flight environment raises a number of pressing issues for the worlds
pilots. ICAO Journal, 53(5), 9-12, 27.
Gerstenfeld, A., & Stein, E. (1995). Simulation is a vital tool in the air traffic control human factors and
technical research, ICAO Journal, 50(3), 19-20.
ICAO Secretariat (2003). Effectiveness of security inspections depends on human proficiency. ICAO Journal,
58(1), 21-24.
ICAO Update (2003). Council President meets with authorities in France and Spain, addresses several events.
ICAO Journal, 58(6), 32-33.
MacBurnie, E. (1993). Human Factors in Aviation. ICAO Journal, 48(7), 7-26.
MacBurnie, E. (1996). Human Factors in Aviation. ICAO Journal, 51(8), 4-29.
MacBurnie, E. (1999). Human Factors Today: managing human error. ICAO Journal, 51(8), 5-30.
Maurino, D. (1991). Education is key to ICAO's human factors programme. ICAO Journal, 45(10), 16-19.
Maurino, D. (1991). Management decisions have an impact on flight safety. ICAO Journal, 46(10), 6-9.
Maurino, D. (1998). ICAO supports proactive approach to managing human factors issues related to advanced
technology. ICAO Journal, 53(5), 17-18, 27, 29.
Maurino, D. (2000). ICAO human factors programme expands scope beyond flight deck and ATC facility.
ICAO Journal, 55(1), 15-16, 29.
Maurino, D. (2001). Amendment to PANS-OPS includes human factors related provisions for aircraft
operations. ICAO Journal, 56(1), 7-9, 34-36.
Mawdsley, D., & Maurino, M. (2005). Cabin safety can be enhanced through application of human factors
strategy. ICAO Journal, 60(1), 17-19, 30.
Pooley, E. (1999). Putting air safety management into practice demands a positive corporate safety culture,
ICAO Journal, 54(1), 10-14.
Stewart, J. (1993). System approach to risk management focuses resources on most serious hazards. ICAO
Journal, 48(9), 12-13.

Cross-Cultural Issues

Article Language
Proceedings of the First ICAO Regional Seminar on Cross-Cultural Issues in Aviation Safety, Asia and Pacific
Region, Bangkok, Thailand, 12 to 14 August 1998
Proceedings of an ICAO/ASPA Regional Seminar on Cross-Cultural Issues in Aviation Safety, Mexico City,
Mexico, 5 to 6 March 2003

http://www.icao.int/ANB/humanfactors/icaojournallist.htm (5 of 5)16-02-2007 22:36:23


Awareness grows of importance of human factors
issues in aircraft maintenance and inspection
Until recently, the role of human factors in maintenance operations was not adequately addressed.
That is changing, because it is clear that human error in aircraft maintenance can have a serious
impact on flight safety.

nance procedure was "proscribed" (i.e


prohibited) in a service bulletin. The tech-
nician reading this concluded that the pro-
cedure was "prescribed" (i.e. defined, laid

T
RADITIONALLY, efforts to improve down) and proceeded to perform the for-
awareness of human factors in avia- bidden action. These types of problems
tion have been directed towards are becoming more prevalent now that air
flight crew performance and, to a lesser carrier aircraft are being manufactured all
extent, towards the performance of air traf- over the world. Sometimes the technical
fic controllers. Until recently, available lit- language of the manufacturer does not
erature showed little consideration of the translate easily into the technical language
human factors issues which could affect of the customer and the result can be
aircraft maintenance personnel who maintenance documentation that is diffi-
inspect and repair aircraft. There is at pre- cult to understand. Since so much mainte-
sent a growing awareness of the impor- nance information is written in English,
tance of human factors issues in aircraft there is a strong case to be made for use of
maintenance and inspection. The safety "simplified" English. Words that mean onc
and effectiveness of airline operations are thing to a certain reader should mean t h t
also becoming more directly related to the same thing to every other reader. For
performance of the people who inspect and example, a "door" should always be called the pylon attachment area. It is believed
service the aircraft fleets. Following is a a door. It should not be referred to as a that if this experience had been shared
review of key human factors issues in air- "hatch" or a "aanel." with other operators of similar aircraft, the
craft maintenance operations: communica- Communication with the aircraft manu- accident at Chicago might not have hap-
tion, training, hours of work, medical facturer, as well as between airlines, can be pened.
considerations and impact of the work crucial. If an operator, in maintaining its Before airlines are likely to engage in
environment. aircraft, discovers a problem that could such cooperation, however, they must be
degrade safety, then that problem should confident that such information will be
Importance of good communication be communicated to the manufacturer and used for accident prevention purposes only.
Communication is possibly the most to other operators of the same aircraft type. The use of such information to gain a mar-
important human factors issue in aircraft This is not always easy to do. Industry cost keting advantage over the reporting airline
maintenance. Without communication control measures and competitive pres- can only result in stifling all safety-related
among maintenance managers, manufac- sures may not place a premium on commu- interactions among operators.
turers, dispatchers, pilots, the public, the nication among airlines. However, civil avi-
government and others, safety standards ation authorities can play an important role Personnel training
would be diicult to maintain. by encouraging operators under their juris- Training methods for aircraft mainte-
It is most important that maintenance diction to interact frequently with one nance technicians vary throughout the
information be understandable to the ta-get another and the manufacturer of the air- world. In many States a common proce-
audience. The primary members of this craft they operate. dure is for a would-be technician to enrol in
audience are the inspectors and technicians A number of accidents could have been a relatively short-term course of training at
who undertake scheduled aircraft mainte- prevented if incident information from air- an aircraft maintenance technician training
nance and diagnose and repair aircraft mal- lines had been made known to the indus- centre. These centres provide training in
functions. New manuals, service bulletins, try. For example, the investigatio~lof the the skills required to pass examinations
job cards and other information to be used American Airlines DC-10 accident at Chica- given by the civil aviation authority (CAA)
by this audience should be tested before go in 1979 revealed that another airline had for the airframe and powerplant (A&P)
distribution to make sure that they will not been using the same unapproved engine technician's licence or certificate. In addi-
be misunderstood or misinterpreted. change procedures, in which the pylon and tion, it is possible in many States to obtain
Sometimes maintenance information is engine were removed and installed as a certification through an apprenticeship-
conveyed through a less-than-optimum unit rather than separately. Unlike Ameri- type programme whereby, over a period of
selection of words. Anecdotal evidence can, however, the other airline had discov- years, individuals learn their craft using on-
suggests a case where a certain mainte- ered that the procedure caused cracks in the-job training (OJT) methods.
and mphishted eI%@mkwskm 3t is
bp&t to provide &merive ehsmoom-
based training on u&d~lj$ng 6ystei prizlei-
pla. This is dBmIt to do wkde thejob

u~fi.11in maintaining generd &&on &-


waft+are not often. m d e d in m&n@in"tng
the airline Beet of cmn$kx, turbine-pm- with smdlw ~arriemand, in such

no p h r tdnfng in t r r i n g catres, In
the= GIBE%, the airlines are required to
provide prachlly all of the trzdniw.
Airline training &auld be a of &m-
mquire;d to maintain a fist of m o d m air-
cr&, mi&may indude taking maximal

j d a r or less tzqxs5-d g e m m 73%


tm?m.ef@ expected b th.ts$ming
a d dmm&ate'thi$ newlg tm@$re-$ how&
.edge g,e the m~skction0-f the t r & If~dl~

she and w m t l w t i o n of the t3paid.n~pro-


gmgum. However, mo& of the mmftuter-
bmd bstruction ccrsrendy Iln use wauld

damroam mdhod~.Certainly these new


tr8'id h light. wheduled mdntenanca Inkingkdumlogie~
would be expected ta
m p h 01d~sQIeCBI.

Usually M e or na remedial im&ucBan is

.aide mare Twm&


ICAO JOURNAL
agement should be aware of the hazards of conditions such as well lighted, comfort- situations. Task lighting needs a range of
such activities as repetitive inspection of able hangars for aircraft maintenance work, brightness, from 200 to 500 foot-candles,
identical items. A long history of research such is unlikely given the cost of building depending on the task. Affordable portable
shows that operator vigilance declines and operating these facilities at every air- lighting units which can be positioned near
rapidly on these tasks and error can follow. port served by airlines. Consequently, work areas or attached to adjacent struc-
Similarly, use of certain types of equipment much aircraft maintenance is performed tures for the performance of specific tasks
is associated with work error. Old-style under less-than-ideal conditions including are available in various sizes and intensities.
inspection devices rely heavily on techni- outdoor, night work in inclement weather. Outdoor, night-time activity demands
cians' skill in manipulating equipment and One of the most important work para- careful attention to lighting needs. A great
in detecting and interpreting subtle instru- meters in aircraft maintenance is lighting. deal of aircraft maintenance is performed
ment indications. Couple these difficulties It is very difficult to provide adequate light- under these conditions. There is an unfor-
with a fatigued technician, and the probabil- ing for all aspects of maintenance work, tunate tendency to rely on flashlights or
ity for error increases dramatically. including inspection and repair. Poor ambi- ambient lighting from open hangar doors
Shift supervisors need to be especially ent illumination of work areas was identi- for this work, because adequate portable
observant of technician fatigue and should fied as a significant deficiency during some lighting is either unavailable or time-con-
oversee and perform follow-up checks of accident investigations. suming lo obtain and set up. Management
tasks to discover any resulting errors. A great deal of lighting for specific tasks must be aware of the importance of provid-
Inspection during daylight hours of mainte- is provided by hand-held torches or flash- ing and requiring the use of adequate area
nance work accomplished the previous lights. The advantages of these lights are and task lighting. It is not a trivial issue.
night could also go a long way towards Adverse occurrences, resulting, at least
reducing the probability of an error.

Medical considerations
4 New training
partly, from lack of adequate lighting, are
often identified in many accident investiga-
tion reports.
Noise is another important factor affect-
Technician health and physical status technologies are being ing performance. Aircraft maintenance
can also influence work performance. Air-
craft maintenance and inspection activity is developed which may operations are usually intermittently noisy
sometimes physically demanding, especial- because of activities such as riveting,
ly on the maintenance technician who is complement or even machinery operation inside hangars, or
overweight, sick or poorly conditioned, and engine testing or run-up on ramps. Noise
could result in work being skipped, uncom- replace on-the-job can cause speech interference and can also
pleted or improperly performed. The need have health implications. Loud or intense
for good vision and sometimes for normal training and classroom noise tends to rcsult in heightened
colour vision is important as well. Older response of the human autonomic nervous
people frequently need vision correction in
methods. system. One of the results can be fatigue.
the form of glasses or contact lenses. Perhaps more important is the effect of
At present, there are no medical require- noise on hearing. Regular exposure to loud
ments for aircraft maintenance technicians. that they are portable and require no set- noise can result in permanent hearing loss.
As is the case with many people, techni- up time. Disadvantages include the lack of Lower-intensity noise can cause temporary
cians may not attend to visual deficiencies brightness and the fact that they usually hearing loss which can have safety implica-
on time, especially when we consider the encumber one hand, sometimes forcing tions in the workplace. Missed or misun-
fact that without periodic examinations, maintenance work or inspection activity to derstood communication resulting from
detection of gradual visual deficiency is dif- be performed with one hand only. noise interference or hearing loss can have
ficult until vision has deteriorated signifi- One frequently noted problem in several serious consequences.
cantly. Moreover, the technician may expe- observed maintenance hangars is poor Steps that can be taken by operators to
rience job insecurity and therefore avoid area lighting. Often hangar area lighting is deal with noise problen~sinclude control-
reporting failing eyesight. provided by ceiling-mounted units. These ling sources of noise by enclosing or insu-
Currently it is rare to find an operator or hard-to-reach units are frequently dust- or lating machinery, isolating noisy activities
administration that requires regular med- paint-coated, and burnt-out bulbs some- so that fewer people are exposed, provid-
ical screening of technicians to detect dis- times go unreplaced for a long time. In ing workers with hearing protection (and
orders that may impair their work perfor- addition, the number and placement of requiring its use), reducing engine run-up
mance. However, because of the increasing these units are sometimes insufficient to or testing to the minimum acceptable, and
correlation between aviation safety and the provide good area lighting conditions. Area measuring noise levels in work areas.
performance of maintenance technicians, it lighting in hangars should be at least in the Noise monitoring can identify where prob-
may be timely to consider implementing order of 100 to 150 foot-candles to provide lems exist, thereby enabling management
regular medical screening. adequate lighting. to take corrective actions. The serious con-
Maintenance and inspection tasks per- sequences of noise exposure should be
Work environment formed beneath aircraft structures and with- stressed so that workers see the need for
To understand human error in mainte- in confined spaces pose difficult lighting hearing protection and for controlling the
nance, it is essential to understand the problems. The structure shades work points level of noise wherever possible. Exposure
technician's responsibilities and to be from area lighting and, similarly, cramped to noise levels above 110 decibels should
familiar with the work environment in equipment compartmcnts will not be illumi- not exceed 12 minutes in an eight-hour
which maintenance is performed. nated by ambient hangar lighting. Special period, and continuous exposure to 85
While it is desirable to have ideal work task lighting should be provided for these continued on page 24

21
New panel to focus on
legal framework for GNSS Cost-benefit analysis
continued from page 15
The ICAO Council decided recently to establish a new panel
of legal and technical experts to evaluate a possible legal
By and large the productivity gains have not been retained by the
airlines over the long term since there has not been an improving
framework for the global navigation satellite system (GNSS).
Under terms of reference established by the Council, the
trend in financial performance. Although the impact of productiv-
panel of experts will consider the different types and forms of
ity improvements has been offset to some extent by the increases
in real input prices in the 1970s, most of the cost savings associ-
a long-term legal framework for GNSS, citing strengths and
ated with the full range of labour, fuel and aircraft productivity
weaknesses of various alternatives, and will also explore the
improvements have been passed on to the consumer in the form
possible need for a convention.
The panel will be composed of approximately 20 to 25
of lower fares and rates.
members nominated by Contracting States. Several interna-
This evidence suggests that the CNS/ATM systems may gen-
tional organizations have also been invited to nominate
erate reductions in fares and rates paid by passengers and ship-
pers of air freight. Lower fares and rates may eventually lead to a
observers 17
further round of benefits associated with an increase in travel
..................................................................................................................... generated by the price reductions.

MD-1I improvement programme Conclusion


continuedfrom page 9 Cost-benefit analysis of implementation of the CNS/ATM sys-
tems has been illustrated here by application of the net present
Passengers and crew will appreciate the ER's satellite commu-
value methodology in a single State in a process which involved
nications (satcom) system, which allows them to maintain voice,
many assumptions. The impact of alternative assumptions was
facsimile or personal computer connections on long over-water
investigated as part of an assessment of the financial risk.
flights. And airlines benefit from an additional revenue source
Because many assumptions undcrlic the cost-benefit evalua-
while enhancing service.
tions, it is not advisable to focus on a particular figure as repre-
Closely related to the satcom is the provision of future air nav-
senting the net economic impact of the implementation of the
igation (FANS) equipment for using the satellite-based communi-
CNS/ATM systems in a State However, consideration of the
cations, navigation and surveillance/air traffic management
range of possible cost-benefit results, and the probabilities or like-
(CNS/ATM) systems. This concept combines the elements of the
lihood of the associated combinations of assumptions, can indi-
aircraft navigation using satellite signals with ground-based air
cate whether investment in the new system would be likely to
traffic control (ATC) via a data link. Thc ER model will also fea-
yield a rate of return which should be sufficient to cover the
ture reduced vertical separation certification, which will allow
financial cost of funds required.
preferred routing on heavily congested routes such as over the
The economic impact of the new systems will differ among
North Atlantic.
States because of varying conditions and geographical features.
The MD-11ER provides operators with the ability to navigate
More important, the impact in any one State, and in a region, will
using the US. global positioning system (GPS) and to maintain a
depend on the success of international cooperation in the imple-
link with air traffic control using the satcom system. As FANS
mentation process. 0
develops, MD-11ER operators will benefit from more flexible nav-
igation, along with more timely traffic and weather information. .....................................................................................................................
These features can be retrofitted into previously delivered MD-
11s.
Human factors
McDonnell Douglas is committed to having the MD-11ER avail- continuedfrom page 21
able to customers by the end of the first quarter of 1996. decibel noise levels requires hearing protection. Both noise and
light levels can be easily measured with relatively inexpensive
hand-held meters. These are tasks that can be accomplished by
the operator's health or safety departments or by supervisors
INTERNATIONAL CIVIL AVIATION ORGANIZATION who have been trained in the use of this equipment.
Toxic materials in aircraft maintenance have become more preva-
Third Flight Safety and Human lent with the advent of more sophisticated aircraft that use compos-
ite materials in their structure or other hazardous substances such
Factors Global Symposium as tank sealants or structural bonding chemicals. Some non-
destructive evaluation methods such as X-rays are also potentially
hazardous. Employees should be informed of the hazards associat-
Aviation 2000: Integrating Human Factors Knowledge
ed with handling toxic materials. They should be instructed in prop
and Practice into Tomorrow's Aviation System er handling methods and provided with protective devices such as
protective clothing, rubber gloves and goggles.
9-1 2 April 1996
Although lhe above irilorrriatiori addresses rrlaintenance tech-
Sheraton Auckland Hotel and Towers, Auckland, New Zealand nicians' health and safety considerations, it has obvious implica-
No Registration Fee tions for aviation safety. It is evident that technicians whose per-
For information and registration contact: formance is impaired because of lack of health and personal safe-
Daniel Maurino or Haile Belai, InternationalCivil Aviation Organization, ty provisions will be more likely to commit errors affecting the
1000 Sherbrooke St. W., Montreal, Quebec, Canada H3A 2R2 overall safety of aircraft operation. This is of great concern
Telephone. (514) 286-6381 /286-6409 Facsimile: (5 14) 285-6759 because, as a general rule, the effects of human error in mainte-
nance are usually manifested in a time and location far displaced
from the maintenance shop. 0

ICAO JOURNAL
uman Factors
Magazine of the International Civil Aviation Organization

VOL. 48 - NO. 7 SEPTEMBER 1993

FEATURES
7 New approaches and commitments will be needed before we can resolve
the human factors problem in aviation.
8 CRM training has achieved notable success, but if the technique is to reach
its potential, there is a need for further dcvelopment of training curricula
and instructional methods.
10 The inexperienced pilot tends to accept readily the importance of human
factors as an integral aspect of pilot training.
14 Aer Lingus' innovative multi-crew training course integrates human factors
and technical training to prepare ab initio airline pilots for aircraft type
transition training.
18 In time it may be possible to optimize training by tailoring the instruction and
scenarios to recognize the great differences that exist between modern and
traditional cockpits.
20 We are only now beginning to understand what constitutes effective decision
making and what skills should be taught.
23 Conclusions from the report on the CFIT accident near Kathmandu on
31 July 1992.

DEPARTMENTS
27 ICAO Update
30 Posts Vacant

COVER
Human factors play a critical role in every aviation activity, from flight training to air-
line management. Statistics attribute about 75 per cent of aircraft accidents to lapses in
human performance, and ICAO has given the highest priority to increasing awareness
of the human factors considerations in all aviation disciplines.

Editor: Eric MacBumie Production Clerk: Denise Cooper-Altuve


Edztorzal Asszstant. Lyne Bertrand Deszgn Consultant' Rodolfo Bore110
THE OBJECTIVES of the Journal are to prov~dea conclse account of the actwltles of the IntemaDonal Cw11
A v ~ a t ~ oOrgan~zatlon
n and to feature a d d ~ t ~ o nmformatlon
al of Interest to Contractmg States and the
mntemabonal aeronaut~calworld. Reproduct~on~nwhole or m part of all uns~gnedmatenal 1s freely authonzcd.
For rlghts to reproduce s~gnedartxles, please wnte to the edltor. Published 10 times annually In Enghsh,
French and Span~sh
OPINIONSEXPRESSED in signed articles or in advertisements appearing in the ICAO Journal are the
author's or advertiser's opinions and do not necessarily rejkct those of ICAO. The mentlon of specific
companies or products m art~clesor advert~sementsdoes not imply that they are endorsed or recommended by
ICAO ln preference to others of a similar nature which are not mentioned or advert~sed.
NOTICE:ADVERTISERS have been adv~sedthat ~t1s preferable to publish advertisement text ~nthe language of
each edlbon. Those advertlsements which do not appear m the language of each ed~tlondo so at the lnslstence
of theu sponsors.
ADVERTISINGREPRESENTATIVE. Westem Med~aSales, Execuhve Office Pla~a,50 West Hillcrest Dr~ve,
S u ~ t e215, Thousand Oaks, CA 91360, U.S.A. Western U.S. and Paclfic Reg~onalRepresentative: He~de
Thorstenson. Telephone: 1805) 496-3500. Telex: 182640 WKVG. Fax: (805) 496-3522. All other reglons.
Western Medla Intemat~onal,800 Blvd Rent'-Lhesque West, Su~te2450, Montreal, Quebec, Canada H3B
4V7. Telephone: (514) 849-5892. Fax: (514) 393-3965.

OTHER CORRESPONDENCEshould be addressed to: The Editor, ICAO Joumal, Sulte 327, 1000 Sherbrooke
Street West, Montreal, Quebec, Canada H3A 2R2. Telephone. (514) 285-8222. Telex: 05-24513. Fax. (514)
288-4772. Publ~shedin Montreal (Canada). Second-class mail registration No. 1610. ISSN 0018 8778.
SUBSCRIPTIONS: US$20.00 per year (by surface), US$30.00 per year (by cur). For related communications,
D.M. Newman contact ICAO Document Sales U n ~at
t the above address, or telephone (514) 285-8026
Headed in the right direction
New approaches and commitments will be needed before we can resolve
the human factors problem in aviation.

5 AVIATION moves towards the Z l s t ten- ~ ~


.............. ..........................................
~ ~ ~ not ~always predictable,
. . with ~occasionally dire

A tury, one of our key objectives is the suc-


cessful application of human factors
knowledge in all aspects of the industry's - opera-
.
DR. PHILIPPE ROCHAT
ICAO SECRETARY GENERAL
..............................,,.........................................
results. Traditional approaches to personnel train-
ing and accident prevention must be re-evaluat-
ed. Second, we must improve the dialogue within
tions. the international civil aviation community. Today,
ICAO has initiated a sustained campaign t o increase the aware- more than ever, designers, regulators, trainers, safety investigators,
ness of the pervasiveness of human error in aviation. Today, the researchers and operational personnel must maintain an open and can-
aviation community can take advantage of employment selection cri- did exchange. Each needs t o know the solutions that the others are
teria that can predict successful on-the-job performance, and tech- exploring to improve aviation safety. The best engineering solution
nology gives us the potential, by way of highly automated may perhaps be in conflict with limitations inherent to humans. The
equipment, t o "engineer" the human error out of aviation. best training solution may not be applicable due t o constraints inher-
In spite of these commendable endeavours, statistics attribute ent t o design. Results of research may not be relevant t o the needs of
most accidents t o lapses in human performance, under the regret- an operational environment. Prevention lessons learned through the
tably recurring label of human error. I raise the question: is it possi- investigationof accidents may perhaps prove difficult to translate into
ble by way of education, training and new technology to improve action unless these are advanced in a meaningful context.
these statistics? Are we on the right track? The President of the Finally, as it relates t o aviation safety, we must think in collective
Council of ICAO, in opening the 29th Session of the ICAO Assembly rather than in individual terms. We must think in terms of system
last October, stated that ICAO attaches considerable importance to error rather than individual human error. This is nowhere more evi-
human factors. "While the safety record of civil aviation is highly dent than in the implementation of the ICAO CNSIATM systems.
commendable," he stated, "the fact remains that about 75 per cent Observing a systems approach to the design and implementation of
of all aviation accidents are due to human error." He further added, these systems, we achieve - again potentially - the synergistic
and in my own view this is the crux of the matter, "new skills, new combination of the best technology can produce and the best
approaches, new commitments are needed to resolve this particular humans can perform. If we do not, we may squander the significant
problem." system benefits. This must not be allowed t o happen.
I believe we are headed in the right direction. However, we will Attitudes towards human factors are changing. But change is sus-
make tangible progress only when we make the commitments and tainable only when it starts at the top. Change, as well as resources
adopt the approaches the dynamic international civil aviation system and safety, must be managed. Those who can best effect change are
demands. those who, by virtue of their positions, can make strategic decisions
Specifically, first we must recognize the challenge of increased tech- t o change direction and commit resources as necessary.
nology and automation, both in the interest of improved safety and Of course, t o implement management initiatives, those in charge
efficiency. The interactions between people and high technology are of the practical, hands-on implementation must possess the neces-
sary skills to achieve it. New challenges require new skills. New skills
can be acquired through training, but if new training approaches
are t o be developed, they must be preceded by a process of educa-
tion of the strategic decision makers in aviation. Aviation managers
must understand the concepts and challenges involved in these new
approaches t o safety.
One thing is clear: there is a disparate level of understanding
about aviation human factors in different regions of the world. We
are trying t o overcome this imbalance through the educational
efforts of the ICAO Flight Safety and Human Factors Programme,
which is designed to increase awareness of middle and senior man-
agers within the international aviation community about the impor-
tance of human factors in civil aviation.
ICAO is also using i t s Technical Cooperation Programme to pro-
vide assistance t o developing States in implementing human factors
The Second ICAO Flight Safety and Human Factors Global Sympo-
endeavours. The Trainair Programme, a major initiative recently
sium was held in Washington in April 1993. Shown at the sympo-
sium (I-r) are: Don Newman, Representative of the Unitedstates on
established by ICAO t o promote effectiveness of training within
the Council of ICAO; Dr. Philippe Rochat, ICAO Secretary General; international civil aviation, is an additional asset which will be used
Robert D. Cook, member of the ICAO Air Navigation Commission; to pursue human factors training and education. 0
Garland Castleberry,Associate Administrator for Aviation Standards,
This article is an adaptation of the opening address given by Dr. Rochat at the Second
U.S. Federal Aviation Administration; and Dr. William Fromme, Flight Safety and Human Factors Global Symposium. Copies of the symposium pro-
Director of the ICAO Air Navigation Bureau. ceedings (ICAO Circular 243) can be obtained from the ICAO Document Sales Unit.

SEPTEMBER 1993
CRM training has achieved notable success, but if the technique is to reach its potential,
there is a need for further development of training curricula and instructional methods.

................................................................................. tation of CRM will not be complete until fic controllers, mechanics, dispatchers,
ROBERTL. HELMREICH acceptance of its concepts is more uniform ground operations personnel, etc. Many of
THEUNIVERSITYOF TEXASAT AUSTIN among the organizations that use it. the observed problems in the aviation sys-
Just as CRM training has evolved over tem involve the interfaces among these
(UNITEDSTATES)
time, the research associated wtih it has groups (for example, miscommunication
................................................................................ changed in direction and focus. We are no and misunderstandings between pilots and
NSTRUCTION in the human factors of longer concerned with the basic question air traffic controllers). A number of new
crew coordination and communication, of whether human factors training has a courses incorporate these ideas, providing,
now known as crew resource manage- measurable impact, but rather with a new for example, specialized training for first
ment (CRM) training, has become world- set of questions that have arisen from the officers upgrading to captaincy. This cap-
wide in the last decade. Our research earlier investigations. taincy course concentrates on the interface
group has been studying the impact of It is important to recognize that most of the cockpit crew and especially the c a p
CRM throughout this period. Our data new CRM courses are very different from tain with other organizational elements, as
show that initial and recurrent training in early programmes that were derived from shown in Figwe 1. It also focuses on
these concepts, combined with practice in corporate management development train- improving specific behaviours by the cap-
line oriented flight training (LOFT), and ing. We are now encountering a '"third gen- tain that were discovered in our research to
continuing feedback and reinforcement, eration" of CRM programmes that focus on be the most important determinants of crew
lead to continuing improvement in crew specific behaviours and behavioural strate- effectiveness. These are the use of briefings
performance over time. gies and take a systems approach to as a means of building the team concept,
The notable successes of CRM training human factors. However, the success of communications and decision making, inter-
should not be interpreted as meaning that CRM does not mean that the technique personal skills, and leadership.
CRM training is now fully developed and has reached its potential. There is a need We have been studying the impact of
optimally effective. The data show large for further development of training curric- LOFT in a number of airlines and have
and significant diifcrcnccs bctwccn organi- ula and instructional methods. found that crews rate this training highly
zations in the impact of training and in the One of the most important insights for both human factors and technical pur-
levels of performance achieved. Even with- reflected in the new programmes is that poses. However, we have also observed
in organizations, large variability in perfor- CRM concepts and training should not be considerable unevenness in the quality of
mance continues to exist and recognizable limited to the flight deck. In the course of scenario designs, inadequate and incom-
subcultures that differ in human factors their work, crews interact with a number of plete briefings and debriefings, instructor
practices can be identified. The implemen- outside groups -flight attendants, air traf- focus on technical rather than human fac-
tors elements, and failure to simulate the
air traffic environment realistically. The
flaws in LOFT execution do not negate its
current usefulness; they only indicate how
much more can be achieved. An important
research question, especially for airlines
that lack extensive simulator facilities, is
whether we can achieve the same impact
in a training device as in a high fidelity sim-
ulator. We need also to understand the util-
ity for human factors training of more
limited simulations that do not encompass
a full mission.
As demonstrated in analyses of commu-
nications during accidents and in LOET
research, crews must simultaneously cope
with multiple tasks at the group level and
differ greatly in the demonstrated ability to
do this. Additional research is needed to
understand the concept of multi-tasking and
FIGURE '1. Captain interfaces during normal flight operatioh
to determine if CRM training can help

ICAO JOURNAL
ors training programmes in aviation." In Cockpit Rex dern~cPress

crews become more effective in handling ing CRM effective. We need to assess crew examining the aviation system. As we
multiple tasks. Another area of importance performance in the ability to reinforce become a global village and begin to see
is operations involving ultra-long flights that effective behaviour and to determine what further integration of crews from differing
require augmented crews, especially in areas need particular attention in training. cultures, we need to be sensitive to these
advanced-technology transports. Such While encouraging progress has been issues and to develop training strategies
extended teams raise issues of leadership, made in developing evaluation methodolo- that are sensitive to cultural differences.
shift changes, and the utilization of extra gies, much remains to be done to improve Consider, lor example, the fact that cul-
crew members in emergency situations. instructor and evaluator skills. tures differ greatly in relationships
Research into these issues could result in It is also crucial that we understand how between subordinates and superiors and in
new guidelines for operations and special- human behaviour is manifested in abnor- their individualistic versus collectivist ori-
ized new CRM training. mal circumstances such as those sur- entation. It seems likely that a better
CRM training has developed outside the rounding accidents and incidents. S. Pred- understanding of such cultural issues will
domain of traditional technical training and more, a specialist in human factors, has allow us to make training in leadership and
checking. This strategy has been highly refined a methodology for classifying and communications more effective in cultures
effective, but it has hindered the integra- coding crew verbal behaviour from cockpit that differ in these dimensions. Our
tion of technical and human factors train- voice recorder tapes and transcripts. Simi- research group has made cross-cultural
ing. Many participants conceive of CRM as larly, digital flight data recorders provide investigations a central part of its research.
something outside of and in addition to an objective record of control inputs and CRM provides an excellent example of
their technical training and evaluation. aircraft performance. These analyses, par- the interplay between basic and applied
CRM concepts need to be fully integrated ticularly of crews showing very effective research. Many of the findings that came
with all aspects of training and checking. responses to catastrophic mechanical fail- from basic research into attitudes and
One strategy for integration that is proving ures, demonstrate that CRM concepts group dynamics have been translated into
successful is to provide instructors and apply generally to extreme situations. How- specific practices in the aviation communi-
check airmen with advanced CRM train- ever, we also need a better understanding ty. We have reached a turning point in the
ing that concentrates on evaluating perfor- of human factors issues in all incidents that development of CRM where we need both
mance and debriefing and reinforcing occur in the system. No approach current- to broaden our scope and to build on the
effective behaviour. ly in use seems to be able to capture all solid base that has been established. It is
We now realize that technical expertise, of the relevant human factors compo- unfortunate that the severe, international
aptitude and training are not sufficient to nents. This should be a primary goal for economic crisis, particularly in aviation,
make an optimally effective aviator. Pilots research. poses a threat to further work. It is our
need strong interpersonal skills as well as The model of crew performance shown responsibility to disseminate the message
technical competence. Indeed, personality in Figure 2 emphasizes the multiple factors that the pay-offs from investments in this
factors may limit the effectiveness of CRM that influence the way groups behave and area will be great in terms of the safety and
training. We must devote energy to res- ultimately the outcome of each flight. A effectiveness of the aviation system.
earch into improving selection strategies if particularly important and often ignored
we are to optimize the aviation system, and element of the model is the influence of The author IS Dlrector of the Aerospace Crew Research
this research must consider the impact of organizational and national cultures on Project sponsored by the National Aeronautics and
new technology such as the "glass cockpit" crew behaviour. Some human factors Space Admrn~strat~on, Unlversrty of Texas and Federal
Avrat~onAdmrnistration The research reported here
on crew motivation and satisfaction. specialists stress the importance of under- was supported partly by NASA Ames Research Center
Evaluation is a critical element in mak- standing cross-cultural issues when and by the FAA.

SEPTEMBER 1993
chinq of human factors
to the ab initio student pilot
The inexperienced pilot tends to accept readily the importance of human
factors as an integral aspect ofpilot training.

tions, there is high-face validity in the school and flight instruction, which tends
Ross A. TELFER equal esteem given each of the course to consolidate the perception that theory is
components. Just as engineering con- inferior to practice. This gap can become a
tributes to aircraft systems, ergonomics chasm when a few intractable personalities
and psychology contribute to their efficient are operating to protect and advance each
operation. Because of the acceptance of a territory, when theory is presented in a
body of knowledge to be transmitted, and dislocated and uninspired manner, or when

F
OR TWO reasons, the teaching of because of this receptivity by the students, instruction is perceived only as preparation
human factors to ab initio pilots is teaching methods can vary widely. for a test. For trainees, a major source of
probably greeted without joy by fly- As is the case with all subjects of study integration is in the planning and presenta-
ing schools: first, because of trainees' lack for pilots, the link between the human fac- tion of thoughtful, individualized briefings
of familiarity with aviation operations, and tor and actual operations is vital. Ground and debriefings. The ability to present the
second, because it further extends training instruction needs to refer to examples big picture in a meaningful way is entirely
time and cost. However, ab initio status can from and implications for flying; flight dependent upon the individual instructor.
be a considerable advantage. It provides no instruction demonstrates applications dis- There is no place for patter here, and there
opportunity for trainees to develop precon- cussed in ground school. Instructors on are major implications for the ways in
ceptions, and this neutral attitude pro- the ground and in the air have the vital which we choose to test ab initio pilots.
motes receptivity. Human factors can be responsibility of continually demonstrating The ideal is for integration of all theoret-
given parity of status with other studies the integration or synthesis of studies so ical preparation with the experience of
and requirements. For example, the BSc. that their interrelationships are apparent. flight, as the opportunity arises. Excellent
(Aviation) course at the University of New- The involvement of flight instructors in examples of this are available in the vari-
castle (Australia) consists of four sections: ground school, or having the one instruc- ous pilot judgement training manuals pro-
aeronautical engineering (including en- tor for both facets of teaching, would also duced in the 1980s by Transport Canada,
gines and systems, avionics, design, matc- accclerate integration of the total curricu- the U S . Federal Aviation Administration
rials); aviation science (including meteorol- lum. From the outset, it is vital that the and Australia's Civil Aviation Authority.
ogy, navigation); aviation management pilot does not classify learning into the These show how instructors can introduce
(including aviation law and computer appli- rigidity of the technical and human group- learning opportunities before, during and
cations); and human factors (including ings which haunt airlines' efforts to inte- after flying lessons. Another structured
medicine, psychology and ergonomics). grate line oriented flight training (LOFT) approach is through problem-based learn-
Each of the four subject areas is designed and licensing requirements. ing, in which groups are given the task of
to link theory and practice. Each is also To be coldly realistic, however, it has to using their interactive skills and collective
intended to be integrated with fight train- be conceded that in ab initio pilot training, knowledge to solve problems which relate
ing and, ultimately, with the task of safely the integration of any of the subjects of directly to the occupation for which they
controlling an aircraft. study with actual flying is not accom- are being prepared, instead of studying a
Human factors is no different. It is con- plished very well. For a start, there is the variety of subjects (such as meteorology,
ceded, of course, that the extent to which unfortunate distinction between ground navigation, law and medicine and princi-
such integration occurs in ab initio pilot
training is contentious. This is a major
issue outside the bounds of the present
I Table I.Proposed ICAO human factors training curriculum

focus, but worthy of industry attention. It


is, however, ironic that we should have an
expectation that human factors, the most
recent addition to pilot training, be inte-
grated, but accept the tradition of ground
school/flight training dichotomy in both
instruction and testing. For the inexpert
and inexperienced pilot, there tends to be a
ready acceptance of the importance of
human factors as an integral aspect of pilot
training. Provided the examples and appli-
cations are drawn from operational situa-

ICAO JOURNAL
ples of flight). The rationale for this
p m c m is that the learner is more motivat-
ed to learn if the subject matter is drawn
from an actual aviation cxme study. This is
by no means a short a t to h u m fact~w
instruction, but the additional effort is
worth it, The instructor's t a s k include
devidng a suitable casestudy; undertaking
a task analysis t;a reveal all the factors
involved: clearly stating objecthe%prepar-
ing work sheets and sufficient resources to
provide the information needed; and pro-
visling appropriate assessment methods.
The Human Factors Training Curricu-
lum provided by ICAO [Cir~ular2271
includes details af the eight modules
shorn in T@Me1. The time allomtion ref-
lects the number of sub-topics (which
TWe I does not present) within each mod-
ule. To integrate such aprograame means
tW the topice and sub-topics need to be
linked with the appropriate stage of fligtot In-ffight sssasment of human factors, coupled with feedback to student and
training. Relevaace and planned progres- ground instructor, is one means of integrtrting humen fadom instrucfion with ab ini-
sion are essential?so that there k a natural tin ffight tminifig.
rather than an, artificial inclusion of the
human factors in airborne ixlshction. the entire instructionalprocess. differsGrun the ICAO programme in b t it
A simple approach m d d be to teach Such irmporknt topie$ can be suitably is intended fur a universi@ underg;r&duate
each module in turn. This uninspired emphasized by an instruct30nd deslgn be& programme over a total of three academic
instruction leaves tbe weighty responsilsil- \visu&ed as a spiml. This ia baaed m the years (six semesters). Many ab initio pro-
Ety for integration squarely on the indlvid- principle that topics are treated briefly @ grammes will not have the 1uxuy of such
ual leastequipped to handle i t the trainee. the broad sweep at the bottom of the spi- an ertended period in which to m l o r e
A preferable alkrn&ve is for the instruc- nJj with the most important ones revisikd concepb research hdings, a d a wide
tnr to extract from each lCA0 module the as the e r a 1 narrows but extends in depth. range lrrf applications. It would be adtcranta-
subtopics which relate to thevarious p h - Key pibt activities such a judgement and geous tca extend the tabk to include finer
es of ab inftio pilot training, preseating dabion-making would be introduced ini- details on the phaws of pilottraining (such
them fn a developments4 and interlinked tially Catthe baaze of the spiral) as a simple as the eight stages identified in T~bde,I ) ,
waF A suggested design k chronoJogicaId go/no-go decidon because ryf weather or 'PIowever, the principIes of ifiteggation and
mirrorbg the .widening experiencesof the recurrence of themes are the same.
trainee progresrsing through first solo,
mow-country, igstrufnent fight and, possi- If human factors Tachin$ sequence
bly* aerobatics or multi-crew operations.
1
Into this structum it is possible to envbage teaching is to 1 Optima& fafar human factors eduation
tobeintegfedwithflighttrai~ng,itfol-
where topics such as visud ilhsion, Hti- I
gue, workload, stress mancgement, deci-
tion making and judgment could appear
instructorsmust 1 lows that the choice of content must be
,idedbythenabureoftheflyinga~tivity.
Thus initially the focus of the study of
when a flight instructor and ground school
hstructor collaborate on the teasing
1 becommittedtotheir human&~rsistheIn~dud~~in~lo
purpose opemgons. This includes the p$y&ologiqtJ
programme They could sequence topics aspect@of safe solo operations and the reli-
for ground school tre~tmentand alw Ifey able processing of data for decision making
them to flight mercises occurring at that airwaft unwrviceability, then would return and psycho-motoractMties. Impo~tantly,it
time. in farmore complex ~ i r c u m s h n m(involv- dso includes the ability to evaluate and
At a h m a n factors workshop in Aus- ing several factom s t a h as pew pressure, monitor student perfarmde so as to re-
tralia in November 1@2, a group of experi- time, fuel, wgulatitm and perwnal compe- ognbe poor judgement, hulty dedsion
enced instructors and pilots dsfded on tencej later in trainin&. Then the approprh chains and attitudhd ha~ards.Whether
eight trdmmg phase9 into w h i d human ate sjolutian is by no means as apparent, one is motivated from within or aternally
factam topics could be integrated. The Such key activities may be present in virtu- (termed "locus of control? and the extent
eight phases were pre-first flighk pre-cir- ally aU training sorties. Aspiral curriculum to which one accepts command respond-
cuig circuits; advanced upper air w r k ; enablss mgar to&s to bs treated initially, bility comes in here; social psychd~gyand
emergencks; basic instrument flighe nav- then revisited for cmsolidation and elabe- group dynamics are not intraduced until
igation; and pawenger carrying, It is not ration. multi-crew operations cammence. Similar-
only a mawr of aDoe&.ng topics to phms, An example ofthe linking of human he- ly, bmic in&uctimal skills md howledge
however. Some topics (a&asjudgement) tars topics to devel~pingpilot expertise are dististinmished from more advanced
are more importat and tend to pervade and experience is shown in Tile 2. This applications, following the expansian of the
limited role of a student pilot to the respon-
sibilities of the commercial pilot. The rele-
vance of human factors is determined by
the training and development of the stu-
dent. If a human factors topic (e.g. the
glass cockpit or circadian disrhythmia)
does not have application at a particuIar
stage of training, it can be included in a
i Initial Assessment FLIGHT INSTRUCTOR
final stage of human factors instruction. l-Feedback to Ground School
This final treatment can anticipate other
FIGURE 1. Human factors assessment as a means of integration
possible circumstances and will provide a
link with in-service human factors training
courses provided by employers, whether it tive, however. In some instances, a com- ity of inducing change in participants.
be in general aviation or the airlines. bination of approaches will be appropriate.
It's a matter of professional judgement. Assessment as integration
Instructional method This judgement by instructors is An interesting possibility would be the
Essentially, instructors have two choices: derived from their personal ability, person- use of assessment as a means of integra-
economy class, traditional ground school ality and experience (the art of instruction) tion. If the ultimate test of human factors
lecturing with a set of student notes and a combined with a knowledge of the underly- teaching is the performance of the pilot,
multiple-choice test; or, ing theory. It follows that employers have then the logical assessor is the flight
doing the job properly. . . which means to allow a degree of autonomy for instruc- instructor rather than (or, as well as) the
integration of human factors topics. tors to use their personal background and ground school instructor. The flight
For students who have come directly style in presenting and structuring human instructor could follow a structured test
from a high school education, pedagogical factors experiences. In designing human sheet, with specific behaviours listed, and
methods are certainly possible. This is not factors instruction, it is efficient to utilize could grade the trainee.
only a matter of trainee age, but also one of the before (presage), during (process) and Feedback on performance would be
familiarity and expectation. For older, expe- after (product) phases of instruction for given by the flight instructor to the ground
rienced trainees, such methods are ineffec- maximum effect and the greatest probabil- instructor, who could then modify content
or method appropriately and await further
feedback. This model is shown in Figwe 1.
Table 2. Integrating human factors and pilot training
Learning human factors
The individualizationof learning is, per-
haps, an idealist's goal for human factors. If
it can be achieved, however, what more
appropriate subject can there be?
Students will find human factors differ-
ent in terms of the nature of the material to
be learned, the nature of the examinations
(typically multiple choice questions), and
attention; workload, stress, personality.
I the application of the knowledge, skills and
values to operations. In human factors
teaching, this may be no more than a ques-
tion of education (knowing why) or training
(knowing how). The latter is by far the eas-
ier to attain, and companies may settle for it
Human Performance on the grounds of economy. They are not
mutually exclusive, of course, and human
factors instruction can encompass both.
Aviation instruction Human factors is as much education

rn (knowledge and values or attitude) as train-


ing (skill), so it follows that the process of
instruction is as important as content. It is a
more effective instructional technique to
structure the situation so that the trainee
Instruction experiences disorientation rather than
rn merely reading about it. Suitable instruc-
tional design would start with an explana-
tion of disorientation in the classroom, a
planned sequence leading to disorientation
in actual flight, and a discussion of the expe-
rience and its causes in a debrieiing session.
That is integration of human factors.

ICAO JOURNAL
Andher teasion is between theory and
pmdce. While the asriation induntry
should be able to expect the same mid-
ance from research in instructional meth-
ods as it obtains from ergonomics or
engineering in aircraft deign, the teaching
of h u m n factors appears to suffer in corn
parison. Part of this is due to ignorance of
the application of human factors to aviation
practice, the vital links between h u m fac-
tors and flight safety. Instructor commit-
ment can help make this link. & key
mdable in trtaehing success is the eathusi-
wn of the instructor, If human factors
teaching is to s w e e d , instructom must be
committed to their purpose, Training the
trainer may need to precede human factors
training.
Rnally, it is relevant to discuss the imp&
cations of trdnee piIotd approachea to
human factors learning. There are three
predominant approaches to learning deep,
~iur%ce,and achieving,The deap approach
to learn,ing is intrindcaIly motivated, with a
dedre tu be competent in fie area of study.
To achieve deep understanding, learners
read widely and integrate their new knowl- negative effect of a surface approach to ing, especially if they are keen to achieve.
edge with their exi~tingknowledge base. learning, and a tendency for positive For em, procesa-focused methods such
The surface-orientedlearner, however, is effects to emergefrom the deep appraad. as p u p dimtsion, role plays and simula-
motivated by anxiety and the dedre to do The implications of t h e ~ results
e for the tion are more appropi4ate. Vicarious learn-
the minimal amount of work required to integration of human factors into ab hitio ing thraugh approximations of reality
p a l the subject. 9urface-oriented strate- pilot training for airlines apply to toe (such as scenarios, accident investigation
gies include rots learning and reproduc- method of imtrzldon and the mailability reports, case studie~,"war stories" or crit-
tion af material prwided in courae notes or of resources. It is apparent that natural ical inddentsj are other means which get
manuds. The third approach, achieving, is ariosity and intrinsic motiva~onneed to quite claw to actual experience. In brief$
concerned with ego enhancement and be ut3zed as much powible. Reference theq the choice of method depend8 upon
organsng the time, source and place of to actual examples from aviation practice is the extent to which btructors and stu-
learning. a valuable teaching technique vehich dents wan1 surface or deep learning.
The relationships betveen approaches dependsupon thorough preparation where Intermtingl~~ deep approaches are those
to learning and performance in aviation the content (such as hypogar for example] which will promote greatest integration of
have now been examined frr several differ- extends beyond thc? immediate experience human factors. The deep learner requires
entpopulatioas as part of an ongoing pro- of the i n s ~ t o rR. videotape of actual pilot a wide range of learning resources: ex-
ject at the University of Mewcastle. An performance in a hypobarii: chamber is the per% an staff willing to have both formal
early study e m b e d approachea to learn- next best thing t~the experience itself, and and informaJ. dfseussion~books; videos;
ing (md their relationships with learning infinitely preferable to s h p l y hearing or charts: models; components; a facilitative
outcomesj in a wrnple of 62 commercial reading about the elfects of hypda. mvironment; appropriate Budget from the
pilot trainees, Data were gathered on Casestudies of incidents and accidents company; encouragement from company
approaches to learning and on individual relating to human bctors topics are valu- executivmii,and so en. The deep learner
performance in each of the nine ground able resources, as me co~kpitrecordings also requires the time and enc~uragernent
school topics (such as aerodynamics, nav- or their transcri@ions, Lectures, assigx- to use the&elearning reaurces to advan-
igation or flight plannind and the time it m a t s , reading*or group projects such as we. Instmetore who seek the minimum
took trainees to fly solo. accident md Incident arydp5iis and discus- performance nmessary to me& lkensing
The most pronninent finding was the ion, have appkation. The purpose and requirements will be rate.ariented, pre-
conai&ently significant negative relailon- result of each differs, howwar. senting notes to be learned and trial ques-
ship bemeen ground school scar& and The extent to which the hasas of schonl tions to be prac;tised"The aim urrill be ta
the surface approah to learning. Ah i m o teaching (pedwogy) or those of teaching pass a test, not to understand and apply
pilots wbo reparted adoptbg a surface adults Candragogy] provide theunderlying human factors knowledge, Introduciag
approa~hto learning wored lower on all principles of instruetion is determined by human factorsWpim needs pkming-, inte-
measures of ground school learning than the pmportioa of intrinsically-motivated, grating it into E i h t training requirm care-
those who adopted a lms surEace-oriented mature individuals who are tjelf-managed ful preparation and expert exeation. 17
approach. Trainees adopting a deep and have acquired a rwervoir of experi-
approach to learning aksD went solo earlier. ence upon which to draw, lkese are the Dr. Telfer is Head of the Department of Aviation a t the
This study demonsfmted the generally students who will awomplish deep learn- Univeoity af Newcastle.
Aer Lingus' innovative multi-crew training course integrates human factors and
technical training to prepare ab initio airline pilots for aircraft type transition training.

costs. These alone justify the Aer Lingus Classroom training


multi-crew training course, though the Basic technical and operational jnforma-
qualitative and operational benefits are also tion are first introduced in classroom lec-
substantial. tures and discussions conducted by an
There is an emphasis on group discus- experienced pilot. This training takes into
sions and experiential learning techniques, consideration the pilot's perspective, and

T
HE BASIC objective of the multi- including various didferent types of simula- anecdotes are used to emphasize practical
crew training course at Aer Lingus is tion. Trainees are allocated to two-member issues. There is an applied focus to this
to establish from the very beginning "crews" throughout their training (includ- training and it is specifically aimed at the
certain fundamental aspects of airline oper- ing ground school) and cooperative learn- development of effective operational under-
ations and modern cockpit management. By ing is encouraged. Crew pairings are standings and competencies, rather than
this means, Aer Lingus seeks to make a changed periodically. The basic objective an "end of course" examination.
clean break with the individualistic ethos of all generic training exercises is to pro- During the classroom phase, trainees
which characterizes most ab initio training. vide basic human factors skills, based upon are exposed to training in the use of air-
Practical aspects of crew coordination, com- the full operational integration of relevant craft public address systems, as well as
munications, and cooperative cockpit man- technical and human factors components. integrated emergency procedures training
agement are emphasized throughout. Simulation methods include written simu- in the company of experienced cabin crew.
Various methods are used to achieve this, lations, low fidelity computer-generated Practical training here includes actual fire-
some of which are reviewed here. simulations, generic flight management fighting in teams of two in a simulated
The course has a number of key training system (FMS) simulations and convention- aircraft cabin. There is also a general intro-
objectives, including the early integration al "full mission" simulator training. duction to the human factors consid-
of technical and human factors fundamen- In accordance with ICAO Annex I erations underlying standard operating
tals. The training is not aircraft type-specif- (Personnel Licensing) stipulations, some procedures (SOPs). This is followed by a
ic. Instead, "generic" or generalized, 25 hours of training in basic human factors detailed introduction to the operational
training has been designed specially for knowledge is provided during initial ab ini- application of specific Aer Lingus SOPs
the course, which comprises approximate- tio training. This is completed prior to and standard call-outs. In association with
ly three weeks of ground school and 12 starting the multi-crew training course and these discussions there is a review of key
four-hour simulator sessions. While this is provides a human factors knowledge base- aspects of crew resource management
a considerable training investment, there line. The multi-crew training course itself (CRM). Where possible, practical exercis-
are very considerable savings in later (air- emphasizes practical and applied human es involving crew management and com-
craft type-specific) training activities and factors skills. munications skills are embedded within
each training activity.

Low-fidelity simulations
The first integrated introduction to tech-
nical, operational and human factors issues
in airline operations involves the use of
"written"simu1ations. These are used to
generate questions and classroom discus-
sion. For example, one written simulation
integrates decision making in the light of
Aer Lingus operational policy with respect
to weather, along with the interpretation of
actual meteorological information and the
relevant navigation and approach charts.
During this simulation the relevant crew
briefings for cockpit and cabin personnel
are practised in a classroom setting.
A series of short low fidelity computer-
generated simulations (using Microsoft
Flight Simulator 4) are used to introduce

IGAO JOURNAL
ilppli~ratimof SOPS
&g the taMq
noise abatement, appma~hand lading
phams of QihL The last of them simula-

practical wpe& af CRll/r md cxrckpit m-


agemmt tmhnique%are explicitly ink*
d u & d There w-e otkw applied training
a&vitiee, hduding a wries of gene&
FMS laifo if om^ here again the emphasis
is on the Megrated applimtian of SOPS
atld stmdaad Gd~-0utS5 mmbined with the
coo use ofthe EMS aad d relevant

dam*The trahdnginvolves hquenk t%qm


sare ta short training scenarios (15 to 25
minuteis] ~ & C F than l o w Tndep&*t&n-

=a, rather tbm the place -Pv6k:reit is fXmt


acquired. &ly &nul&ar traiaing is dwat-
ed ta Bade aircraft handling and nmm-
flee, inducting fill apprimdaa of alI SCPs

retard b &omd .on a mrnmrkr to tot%&&


imtant aco~t$s
to the devant video &a
d m debrief@, By u&ng aspmid hand-
held d&m, the video record cm be
ment is that if we want pilots to perform as Training and checklist including real time liaison with cabin crew
a crew - as team members - we should design philosophy and passengers;
train them as a crew throughout. To fully Following is a description of the ratio- there should be positive reinforcement
optimize such training, all operational activ- nale underlying one specific multi-crew of good operating prastices throughout:
ities and crew decision-making behaviour training course simulator training tech- and,
must be grounded in operationallyrealistic nique: the use of generic checklists. training requirements should not cause
trainiig exercises. Technical drills and air- It was necessary to develop a generic any temporary suspension of operational
craft handling are thus viewed as being a airline "normal checklist" when designing "reality."
close second to CRM and multi-crew cock- the multi-crew course simulator training These basic design principles were
pit management skills, since the former syllabus. S i c e half of the simulator train- applied when developing the checklist. In
vary from aircraft type to aircraft type. In ing sessions are devoted to the operational addition, the accident record suggests that
this approach, aircraft handling and the management of non-normal events, a suit- certain important CRM principles are not
application of technical drills are treated as able '"on-normal checklist" was also always understood or actioned effectively
activities which are embedded within the required. It was felt that a generic non-nor- by pilots. It is, for instance, assumed that:
pilot's broader operationalunderstandings ma1 checklist which emphasized human aircraft control is formally allocated to
and competencies. factors considerations would offer unique one of the pilots;
During the early training design phase of training opportnnities. The following * situational awareness and fault diagnosis
the Aer Lingus mult.i-crew training course, points were considered when designing are jointly evaluated;
it was therefore decided that the simulator this checklist: drills are executed on the correct system
training of preference would expose trai- it should facilitate successful operational or engine;
nees to a representative series of opera- management of non-normal events by the there is an appropriate division of task3
tional events. By combining the common crew; between crew-members;
operational aspects and characteristics of human factors principles and crew co- the crew will promptly and effectively
normal and non-normal events, as consid- ordination issues should explicitly figure in address all operational consequences of
ered across various aircraft types, a eom- the checklist design and content; the failure;
prehensive and representative series of all communications generated by non- appropriate communications and coordi-
generic training scenarios was designed. normal events should be addressed, nation with the passenger cabin will occur.

Dublin-based Aer Lingus has introduced a multi-crew training course to emphasize practical aspects of crew coordination, com-
munications and cooperative cockpit management. (Photo: Boeing Commercial Airplane Group)

16 ICAO JOURNAL
The training intention behind the Aer
Gngw non-normal operations chedclist is
to structure, m d guide, trainee responges
and learnfng activitie~in light of thew
human perfomwce consideratima Rather
than t d trainees about the hnportaace of
a r t a h CBM behavioum, such as commu-
nications, the checklist is used to form*
and drive the desired codgit management
behavioum. Video debriefing is sub&-
qumtly used to provide feedback on a&d
cockpit perfmmance,
One selected exampb from the Aer
Lingus generic non-normal operatiom
checMi&is provided in Fig@%1.This Uut+
trates the generic "Sptems Mallunction"
&ecklist which is used in the event of a
failure or ma~hnctionof any aircraft @s-
tem, &light instrumnt. Proper see of the
ehecldist alao ensures that the relevant
cockpit acthities are mrbaliiaed and active-
ly coordinated between crew manbeus.
Conwquently it also serves as an aid to Aer Lingus uses low-fide!w smuwrron ro nerp rramecps ream now ro apply sranaard
effective crew dedsian making.
~pemfjngp~oredutes,checkIEsts. basic CRM skills and airline operating procedures.
The boxed items are "memory items*
to be called out md directed by the flying hwe an operatitad impact ti~n3.This meam that normal flight plan-
pilot. Su%sequently the non-flying pilot R ~ / ~ P E R A TC~bONNg A m L? ning, and all operatima1 documentatim,
reads the checklist Item aloud to confum Fuel Stab$: OptionsAvailable? are consofidakd into the actual simulator
that the check has been satkfa~torily PJon-standard Airc& Configuration mining enviroment h further mems that
complebd. There is only one technical Required? many of the individualtraining topies -for
"dean item on this checHtt -name- * P E H Q W C E DEGIEAD~T~o~? mample:,ele&cs and hydraulic failures -
ly to switch off any relevant system Aicrafk? arefimt encollfltered in a realigtic Bne can-
switches. Memwy items on the cheddht Rmmy: Lplngthand W&a Canditiom? text, where bath the techrricai problem1
sexk to ensure qproprhte c m coordina- C~MMUNICA~OMS REQUIRED: and the red tlme opemtionalmanagemmt
tion, communimtions and decision mak- Cabin Crw, of that problem, are matedas an integrat-
ing.It will be noted that the items on this Air Tr&k bntrok ed whole for trajaiag purp~-ses.
generic check1"it will have continuing Pawngem;
relevance after multi-crew training has Colrrgm, Concludon
been completed. The multi-crew course son-normal Aer LingusbuIti-crew murse training
Under tEre "Operational Conse~uence$ checklist provides trdnees with a concise has been sufficientlys u c c e far ~ the~ dr-
. . .Edua-te" heading, a nutinbe of supple- inb-oduction to the structured use of non- line to d e s i g ~a d m i h generic trainiug
mentary headings were recently added to normal &ems M m a mnlti-crew opw- course for command upgrade candidates.
p r m p t md assist the crew as they asswa ational enviromenL The checklist &elf TZle decision to conduct pre-command sim-
the operational impact of the relevant pmb- facilites the management of a compe+ ulator training d g generics &mulationis
lem, These rehrience the main areas in hensive q e c t r m of operationalproblem clealy a atrow endorsement of .the per-
wuch tlon-normal events most frequently d& simulator trtxhhg. These reprwen- ceived success and d u e of the entire
tative cockpit rnawgement issues all arise multi-crew training initiative. Trairxing
in a full mi&onn,real We, mulkrew oper- which empharii~eeoperationally realistic
I SYSTEMS MALFUNCTION 11 ational environment, with 'rpacks;en@mp
and "cabin crew" fully integrated in@ the
tasks - embeddecf within a fuRnfl-misf&tn
en\Pironment -offem a very effective
training situation. T h i ~
repesents a rich method of e f f e d ~ e l pintegrating h m a n
human factam anand CRM learningenviroa- factors.and*t skills,
ment for neu. pilob, The le~wazlearnedby Am Lingus is that
the development of multi-crew cockpit
T&mespt;cificsirnulatar training management skills through geaedc simu-
Given the gxepmtion pravided by the lati011 sessian~has a much wider a p p b -
multi-crevkaining c o m a it has been pou tion than the er;dini-ng of ab initio pilats.
sible to infroduce highly innovative type- Geneh simulation aho has the ~ignikant,
specMc @asitionsyllabi for gradtlathg the added benefit that it is mtremely coat-
course's ab iinitio trainee&. Here traditional effeclive,
sthtirat61- training p r d w IzaS been large-
$ ignored* with most training sessions Capt. J o h m t a n h head o f Training D e v e l o p m ~ n at
t
designed aound line operational sirnula- &r Lmgus
Cockpit resource management and flight
training for the advanced-technology cockpit
In time it may be possible to optimize training by tailoring the instruction and scenarios to
recognize the great differences that exist between modern and traditional cockpits.

three-pilot flight deck. nology cockpit. These lessons emphasize


To the credit of the industry, by the mid- not hardware and systems, but crew coor-
dle of the decade the problems were under dination demanded by automation. For the
control, and failure rates of crews transi- sake of economy, airlines wish to keep
tioning into aircraft equipped with the recurrent LOFT scenarios of a generic
glass cockpit were no greater than those (model-independent) nature. Transition

I
N JANUARY 1981 a group of leading usually found in conventional aircraft. The training creates the opportunity to tailor a
authorities in the field of pilot training industry could begin to prepare for the LOFT programme to the technology of the
met at the National Aeronautics and rapid acquisition of new aircraft, route cockpit.
Space Administration (NASA) Arnes expansion, mixed fleets of derivative
Research Center to discuss the fairly models and equipment differences within Philosophical questions
recent concept of line oriented flight train- the same models, extended range twin- The rapid development of cockpit
ing &OFT). This meeting, which would engine operations (ETOPS) of two-pilot air- automation, and the lack of operational
later be regarded as a milestone in pilot craft over the ocean, and a tidal wave of doctrine by which it could be governed, led
training, came coincidentally at a turning newly hired pilots, many with less flying some airlines to formulate a "philosophy of
point in the history of cockpit automation. experience than those previously obtained. automation." In 1990 Delta Air Lines adopt-
Just a few months earlier the first McDon- By the end of this century the glasscockpit ed a one-page automation philosophy that
nell Douglas MD-80 (originally DC-9-80) aircraft will be not the oddballs of the would guide its approach to equipment
aircraft had left Long Beach to join the worldwide fleet, but the mainstay. R.L. acquisition, training, and operational doc-
fleets of regional carriers. The MD-80 Helmreich summarized the situation in a trine for the carrier's rapidly growing fleet
brought to the short- and medium-haul car- paper presented to a 1991 symposium on of modern aircraft. Delta also implemented
rier a level of cockpit sophistication previ- aviation pyschology: a new training course which is required of
ously found only in wide-body aircraft. Boe- all pilots transitioning to glass-cockpit air-
The impact of cockpit automation presents a
ing's 767s were only months behind, craft for the first time. It precedes ground
number of challenges for future research.
propelling the aviation industry into an school and is model-independent. The pri-
While we know that crews are behaving dif-
even higher level of flight deck technology, mary goals of the new course are to
ferently in advanced technology and stan-
the era of the "glass cockpit." explain Delta's philosophy of automation
dard aircraft, we do not yet know whether
It is significant to note that at that his- and to relieve some of the anxieties and
these differences are reflective of training or
toric meeting there was no discussion of misconceptions that pilots often bring to
of characteristics of particular automation
the implications of the rapidly escalating ground schools for advanced technology
philosophies and aircraft designs. Most
sophistication of cockpit automation for aircraft.
LOFT scenarios being used are "generic"in
pilot training or for LOFT. Even such eso-
the sense of not being based on characteris-
teric topics as training for engine-out ferry Automation and crew coordination
tics and capabilities of advanced technology
flights were covered at this meeting, but Training experts in the airline industry
aircraft.
the impact that automation would have on and in government have generally assum-
pilot training was yet to be recognized. LOFT scenarios can be designed that ed that CRM training programmes are
The participants at that meeting proba- are ATC-intensive. For example, one air- essentially model-independent: identical
bly could not have imagined what lay line, America West, has produced an inter- training was delivered to all pilots at a
ahead in the next decade of pilot training. esting scenario in which a line of given airline, regardless of the type of air-
In the years to follow, airline training thunderstorms tempts the crew to deviate craft they were or would be flying. The effi-
departments throughout the world were to from course toward a military operations cacy of this can no longer be accepted as
experience the birth pangs of the glass area (MOA). In order to avoid penetrating true without proof, as there is mounting
cockpit. They would learn that a Boeing the MOA, assistance from ATC is required. evidence that crew coordination and com-
767 is not simply a 727 with some extra Realism is taken to the ultimate when a munication in the glass-cockpit aircraft is
boxes, and they would struggle with high- flight of fighters appears in the pilots' view. qualitatively different than in the tradition-
er failure rates in transition training than Several carriers have begun to insert al cockpit. If this proves to be the case,
they had never seen before. Not only were into their cockpit resource management then, at the least, modules of CRM pro-
the world's airlines facing an industrial rev- (CRM) and recurrent training pro- grammes should be devoted to crew
olution in the cockpit, but they were simul- grammes modules that demand the crew resource management in the advanced-
taneously witnessing the beginning of the coordination and workload management technology aircraft. Such a move would be
end of the era of the flight engineer and the required to successfully fly the high-tech- logical, and consistent with one of the

ICAO JOURNAL
guidelines pmpoml for GM training by
human factors e x p e r k "Customize the
training to reflect the nature and needs of
the orgznization." To these words could be
added, "and the generic type of cockpit
tmhnology as well."
At this h e there b no solid cxperhm-
tal evidence that these presumed differ-
ences actually exist, let alone that they are
worthy of, or addressable by, a tailored
CRM p r a g ~ m m for~ advanced-k&nolo.gy
aircmft. The q&o& study of B ~ e i n g757
crews points in this direction, but couldnot
by itself be taken as sufficient reason to
launch a CRM programme in mtomatioa

Implications for IhFT


Little has been written about the impli-
cations of the high-technology cockpit for
the design and conduct of LOFT, although
it could be argued that the modern cockpit
may require tailored CRM programmes or
modules.
Utilization of LOFT techniques may
appear in transition training, where the
issue may be forced by the needs to adapt
to a new cockpit. Automation-specific sce-
narios will be adopted more slowly in
recurrent training, where economic con-
siderations continue to dictate model-inde-
pendent scenarios.
The high-technology cockpit, with its Evidence is mounting thar crew coordination and communications in the glass cock-
strengths and weaknesses, offers novel pit may be qualitatively different than in the traditional cockpit. (Photo:Airbus Industrie)
opportunities for scenario design. For
example, in traditional aircraft it has been remainder of the flight. of augmented crews (one or two additional
necessary to induce abnormal conditions Thus automation provides the LOFT pilots) into a two-pilot cockpit provides new
(e.g. system failures) in order to elevate the designer with the opportunity to build sce- challenges for crew coordiiation.
workload and stress on the crew in a realis- narios that will directly address the prob-
tic manner, in order to create a situation lems and opportunities of two-pilot crews Summary
which demands crew coordination. Thus working in the advanced-technologycock- As we learn more about the crew coordi-
their proficiency at coping with these condi- pit, particularly those involving the aircraft nation and communication requirements of
tions can be evaluated, as well as preserved automation/ATC interface as it exists the glass cockpits, and sharpen our ability
on video tape for selfevaluation. The mod- today. The LOFT constructed for these air- to provide effective CRM and line oriented
ern cockpit has enough normal, built-in craft can and should exercise those pecu- flight training, we may be able to optimize
stressors to do the job, particularly in the liar characteristics of the modern cockpits, training by tailoring the instruction and
area of air traffic control instructions. and can easily create situations that scenarios to recognize the great differ-
The glass cockpit presents new opportu- demand that one practise CRM principles. ences that exist between modern and tradi-
nities for scenario design that do not tional cockpits. According to some experts,
require emergencies per se, just difficult Implications for the future the use of LOFT-type training has changed
problems at the human-automation inter- In a few years the glass-cockpit aircraft the texture of airline training. The author
face. Any pilot of a glass-cockpit aircraft will dominate the world's fleets; more believes that with the development of new
will be happy to tell you what they are. advanced models with new automatic fea- cockpit technologies, that texture will con-
tures are on the way. Furthermore, for rea- tinue to change, and possibly in ways that
Automation-rich LQFT scenarios sons external to cockpit technology, these are not easily predicted at this time.
The resourceful LOFT designer has aircraft likely will be operated under more
new tools to work with, and these clearly stressful conditions than we have known in
meet the criteria of realism and line validi- the past. Airlines will face new demands to This article is a condensation of a chapter by the author
in Cockpit Resource Management, by E. L. W~ener,8. G.
ty. One highly experienced LOFT designer meet market conditions, adapt the modern Kanki, and R. L. Helmreich, published in 1993 by
remarked to the author that the best LOFT aircraft to the ATC system (instead of the Academic Press. The author's work has been supported
he had ever constructed contained a single other way around, as it should be), and In part by a research grant from NASA's Ames Research
Center t o the University of M~ami,jointly funded by
failure: the autopilot dropped off early in absorb new hires who may be quite inexpe- NASA and the Federal Aviation Admmistration. The
the flight and could not be restored. This rienced compared to those that the airlines opimons expressed here are those of the author, and
failure produced implications for the have previously attracted. The introduction not of any organization.

SEPTEMBER 1993
Lessons from research on expert
decision making on the flight deck
We are only now beginniag to understand what constitutes effectiue
decision making and what skills should be taught.

portation Safety Board (NTSB). There we offered, what dimensions are relevant for
JUDITH ORASANU find cases of crews that flew into thunder- comparing models, and what we want in a
NASA AMESRESEARCH CENTER storms and encountered wind shear; decid- car. The task is a matter of deciding which
ed to reject a take-off after they were off car will best satisfy our needs. In contrast,
(UNITEDSTATES)
the ground; decided to land after retracting decisions in the cockpit are not ends in
the landing gear in preparation for a go- themselves - they are the means by
around; took-off with snow and ice on the which crews achieve their larger goal,

D
ECISION MAKING is an essential wings; or decided to fly on to their destina- namely, to deliver passengers and aircraft
component of a captain's expertise. tion on battery power rather than return to safely to their destination. Several features
The captain is responsible for mak- the airport from which they had just depart- characterize decision tasks in the cockpit
ing the hard decisions: choosing where to ed. NTSB reports from 1983 to 1987 impli- (as well as in other complex domains such
divert after a system malfunctions, when cate crew judgement and decision making as nuclear power, military command and
fuel is short and weather is deteriorating; in 47 per cent of the fatal accidents. control, medicine, and fire-fighting) and
determining how to cope with a passenger's Decision making is a component of i d u e n c e the nature of the decision
medical emergency; evaluating whether to most crew resource management (CRM) process:
take-off with a placarded system given past training courses given by major airlines Dynamic conditions. Conditions change
experience with the projected weather and (see US. Federal Aviation Administration over time, making the situation unstable
traffic at the destination. While the captain Advisory Circular No. 120-51, 1993). How- and unpredictable (e.g. weather or some
has ultimate responsibility for the decision, ever, little scientific research has been avail- system malfunctions).
the entire crew in the cockpit, in the cabin, able until recently to support that training. Ill-structured problems. It may not be
and on the ground can provide information We are only now beginning to understand clear to the crew what the problem is, what
and suggestions that contribute to a good what constitutes effective decision making options are available, or what would consti-
and safe decision. The captain's judgement and what skills should be taught. tute the best solution.
is most critical when conditions are Decision making in the cockpit is unlike High-risk. Decision errors, especially
ambiguous and no clear guidance is pro- decision making in several other situations. during abnormal and emergency condi-
vided in manuals, checklists, or company Most often we thiik of decision making as tions, can have severe consequences.
policy. choosing between options A, B, and C, Time pressure. Certain decisions must be
The significance of poor decisions is evi- such as when we buy a new car. In the case made very quickly. The aviation environ-
dent in reports from the US. National Trans- of car-buying we know what models are ment is unforgiving: correct decisions
made too late can be fatal.
Competinggoals. While safety is usually
paramount, economic considerations put
pressure on crews to save fuel and to be on
time. In addition, crews must conform to
government regulations and company
rules. Sometimes safety is pitted against
these other very real goals.
Multiple participants. Several partici-
pants (cockpit crew, cabin crew, company
dispatch, maintenance, ATC) may bring
different perspectives, knowledge, and
goals to a problem. Conversely, multiple
participants can monitor and evaluate each
other, reducing the likelihood that impor-
tant information is overlooked or that
faulty plans are adopted.
Expertise. While pilots may not be expert
decision makers, they bring vast knowl-
edge and experience to the decisions they
make.
Decision problems in the cockpit do not

ICAO JOURNAL
come neatly packaged, with Eh6 ~ptlans>
0 ~ 1 smd m & a I ~ &dmrly @&ci;Eied,
Before a deWm can be made, the crew
muat re~wbe that a &uakion edsts &at
fizxpiresattention, The nnzltlrre afthb @rob-
1m m ~ &be d&a&ned, ifs @ev&ly
a=~wd, md optkms wmkked. In Len-
WL de&ion making in dynamfoemiron-
men& &ni two a j a r compatmt.9:
itnation m e m e a t m d choice erf a
caw= of wtis1a.
Situatiosmmsment involve@intwpt-t-
ing the CUES that ~ignda problem and
jud&8 the levels ~f risk and .time prm- fxf~ msider relev& apfierm
ewe. $om$ d d & o n e re&v a &st rm- A mroIlary o'f good s b t i s n asestmmt
ponse, SIP& as r e j w t i ~a t h - o f f or L rmlbhg JNhgt Woxmticm is nmded in
dwidhg ts, go around, Others alllaw time! order to m&e a gaod deci&m ?%enseek-
mtkas and to gather ing it Emrf u&g it I-rr the w 12
4 r2fafl-rooJsiner
on. Some decidone r hydraulic fa8w in
maditi;onB thdtkiggef the crew must
a p r ~ d e rd ~ a &d&&%d n ~ &Xi- rewgnbxithe c o m q u w s of 1;Inehydrm-
some bvolue ckim from among lie fa3h-g: mmqal gwr and alterna&flap
~ptionz+~ mMpre4ulm prior- axteadon pr0r:ducs u-erequi i&
t4n
pro%lemsdvtngbewum ~ m the
x Irsnfinp
lutions ar options &&, be&a&e& SQa
Where do@ mgerthe mtw into this go-5trmlrdf~ mde&&le, rn&w=W a
picaawe? Studies af eq&-tsh domaim mn~lderatiofi.T h e mfisQ&ints &de
ranghi@&omp h y s b aad dhws tr)college f&&on 3 md d@w~ what
dm@dansmd me,di&ne show that mpm- taw a mod @olution,
60a Mutwm haw we p e k d v i ~ ahd B e @ d b of w h t h afwt
~ or a a ~ r ~
P-nd ta grablem*In g$m3-& ~ e . d 3 e : dehiexate tkWm mud be ma&, more
problem is that modern aircraft are so reli-
able that most pilots have little real experi-
ence coping with abnormal and emergency
1. Recognize and interpret cues I situations, and hence few learning opportu-
risk (present and future) nities. While this a situation for which we
time available are all thankful, it means that other oppor-
tunities must be found to allow crews to
practise the needed skills. Line oriented
flight training (LOFT) provides such an
opportunity, but these sessions offer limit-
ed time for handling diverse problems. Use
of low-fidelity simulators could be used to
exercise component skills. Classroom time
often is devoted to reviewing videotaped
reenactments of accidents and discussing
ideal solutions. However, discussion of
how you should respond is not quite the
same as doing it. Ideally, discussion of deci-
sion strategies would be complemented by
real-time practice.
The research community has not yet
discovered a "silver bullet" for training or
Gding that will guarantee that every deci-
sion is the best possible one. At this point we
effective captains manage the situation in Errors that result in poor decisions are can only recommend processes that assure
ways that allow them to make effective the converse of effective decision strate- that the crew understands the situation
decisions. A primary way to do this is to gies just described. NTSB accident reports and what would constitute an appropriate
plan for contingencies. They anticipate suggest that serious outcomes result from response. A few general recommendations
what might happen and set "triggers" for poor situation assessment, mainly when can be made:
themselves. Then, if and when conditions crews fail to appreciate the risk inherent in Understand what the problem is before
deteriorate, they are prepared rather than a situation. Crews that get into trouble uni- acting.
surprised. Effective crews shift their think- formly seem to suffer from severe cases of Assess the risk and time factors.
ing from a high workload, time-pressured "get-there-itis." Many accidents occur fol- Match your response strategy to the fea-
phase to a lower pressured one. lowing schedule delays or at the end of tures of the situation (e.g. fast vs. thought-
Similarly, when a complex problem re- long trips, when the crew is eager to get ful response).
quires evaluation of several options, more home. Crews exhibit a strong tendency to Set up contingency plans whenever pos-
effective captains arrange the situation so assume that ambiious cues fall within the sible.
they have thinking "space" to sort through range of normal, rather than being seen as Consider the implications- the nonob
the possibilities. They can do this by potentially dangerous, and carry on as if vious future consequences -before decid-
offloading tasks, including responsibility nothing is wrong until the consequences ing on a course of action.
for flying the plane. Or the captain may buy overtake them, Manage workload to allow time for deci-
time by requesting a holding pattern or Discrepant information often is exp- sion making when time and fuel permit.
vectors. This allows time to gather and lained (or wished) away, especially if it is Create a shared problem model by com-
evaluate information before committing to ambiguous. Consider the case of a Boeing municating with the crew (cockpit, cabin,
a course of action. Effective time and work- 737 on takeoff roll with snow and ice on its and ground). Be sure aIl understand what
load management strategies not only wings and engine sounds that suggested the problem is, what the plan is, and who is
reflect awareness of the demands of the less than full take-off power, despite the doing what.
problem; they also reflect sensitivity to engine pressure ratio (EPR) readings. Expertise comes with experience. By
one's own cognitive limitations, especially When the first officer brought the dis- practising the above in relatively benign
under high-stress conditions. crepant cues to the captain's attention, the situations, when workload is low, crews
An integral part of decision making in captain repeatedly affirmed that every- can get in the habit of thinking of these
general and task management in particular thing was OK. The captain was using only strategies. Experience may be a great
is communication. Effective captains do the pieces of information that fit his model teacher, but some lessons exact a high
not function as 'lone rangers." When faced of normalcy and ignored those cues that cost. The aviation industry is trying to pro-
with problem situations, they share their indicated a problem with engine power. vide opportunities to learn at low risk and
concerns, make clear what they want to Realistic assessment of the risk in a situa- low cost. How to do so is the challenge - to

accomplish, and invite participation by tion, facing the possibility of a worst-case our collective expertise.
other crew members. They also listen to scenario, and preparing for it before the sit-
the suggestions offered by other crew uation begins to unravel are probably the
members. This interaction allows the crew only strategies that assure safe decisions. Aeronautla and Space Admlnlstratlon'sAmes Research
to build a shared model that assures they Given the number of hours that flight Center. The opmions expressed In this article are the
are all working to solve the same problem crews accumulate, we might expect them author's and should not be construed as officlal p o k y
and facilitates contributions by all. all to be "expert" decision makers. The of any government agency.

ICAO JOURNAL
Conclusions from report on
CFIT accident near Kathmandu
The aircraft's flight data and cockpit voice recorders were instrumental
in providing information needed to determine the causes of the accident.

SYNOPSIS
The flight was conducting the Sierra
..."*,*.CI...l**1.*~~I.*.Ii...h.'.i*.*~*'i1.,**. .... .............
*...*a *.I..I I.,..,....,I '
I..,...,

VHF omnidirectional radio rangeldistance


DISSEMINATING INFORMATION ON CFIT OCCURRENCES
measuring equipment (VOFUDME) app- One type of accident that continues to take a heavy toll on international civil aviation is
roach t o runway 02 at Tribhuvan Interna- controlled flight into terrain (CFIT). In its 131st session (late 1992), the ICAO Air
tional Airport, Kathmandu, in instrument Navigation Commission agreed that CFIT was a critical flight safety problem and that
weather conditions. A flap fault occurred urgent action was required by ICAO. Accordingly, the Commission requested that the
while the flight was on the approach; this Secretariat develop a comprehensiveprogramme, one specifically aimed at taking preven-
caused the crew t o ask for a clearance tive actions on CFIT.
back t o Calcutta, a decision that was in
Even though there was initial success in curbing the number of CFIT oceurr :s fol
keeping with both Company and perfor-
mance requirements, which necessitate
lowing the development and implementation of ICAO provisions for the ground proximity
the use of full flaps for the steep final warning system (GPWS), these types of occurrences have increased in numbers over the
approach. Shortly (21 seconds) after mak- past severalyears. (See "ICAO initiates comprehensive programme to prevent occurrence
ing this request, at a distance of approxi- of CFIT accidents," ICAO Journal, January/February 1993.)
mately 12 nautical miles from the Kath- To provide timely dissemination of critical flight safety information, extracts of findings
mandu VOR, the flap fault was rectified from official investigations of significant CFIT occurrences will be published in the Jour-
by retracting and then reselecting the nal from time to time. This month the Journal presents the findings, causes and o
flaps. The crew determined that i t was
tional recommendations contained in the report of the Commission for the Accident
not possible t o continue the straight-in
tigation of T G 3 l l (Thai Airways International Airbus Industrie A31@304,HSTID, near
approach, due t o the steep descent angles
required and the position of the aircraft. Kathmandu, Nepal on 31July 1992).
The crew stated to the control tower that Readers should note that the Commission cident 1 tigation of TG3X1 was
they wished t o start their approach again established by the Government of Nepal for the purpose of advancmg aviation safety. It is
and requested a left turn back t o the not the function of the Commission to assign blame or determine civil or criminal liability
Romeo fix, which is 41 nautical miles Readers who would like to obtain the accident report should contact the Government
south south-west (202 radial) of the Kath- of Nepal. Write to: Ministry of Tourism & Civil Aviation, Department of Civil Aviation
mandu VOR. The controller, in the non-
Babar Mahal, Kathmandu, Nepal
radar environment, responded by clearing
the flight t o make the Sierra approach,
'................ 1..

I
which starts at the 202 radial and 16 nau-
tical miles from the VOR. The crew maintain an altitude of 11,500 feet and that the crew was in the process of
response t o the clearance was t o report was t o "proceed t o Romeo" and contact inserting "Romeo" and other related
that, at the moment, they could not land the area control centre (ACC) controller. navigational information into the flight
and t o ask again for a left turn back t o The flight, commencing a descent while management system (FMS), but were
Romeo t o start their approach again. in the turn, completed a 360-degree experiencing difficulties.
After further dialogue with the con- turn, momentarily rolling out on head- The flight continued towards the
troller, which included requests for a left ings of 045 and 340 degrees, and again north on a heading of 025 degrees and
turn, the crew unilaterally initiated a proceeded toward the north on a head- then, at about 16 nautical miles north,
right turn from the aircraft's 025 degree ing of 025 degrees magnetic. When the the heading was altered t o the left t o
heading and commenced a climb from flight was about five nautical miles 005 degrees. Slightly over one minute
an altitude of 10,500 feet t o flight level south-west of the Kathmandu VOR, the later, the ground proximity warning sys-
180, when the flight was about 7 nauti- crew contacted the area control centre tem (GPWS) sounded the warning "ter-
cal miles south of the Kathmandu VOR. and stated that the aircraft was "head- rain, terrain" followed by "whoop,
The crew reported t o the tower con- ing 025" and they wished t o proceed t o whoop, pull-up"; the aural warning con-
troller that the flight was climbing and Romeo t o start their approach again; tinued until impact approximately 16
the controller replied by instructing the adding they had "technical problems seconds later. Engine thrust was increas-
crew t o report at 16 nautical miles for concerned with the flight." It was again ing and "level change" had been
the Sierra approach. During the turn, established that the flight was t o pro- announced in the cockpit, just before the
there was more discussion between the ceed t o Romeo and the crew agreed t o impact occurred at the 11,500-foot level
tower controller and the flight, where it "report over Romeo." It was determined of a 16,000-foot peak; the accident site
was established that the aircraft was t o from the cockpit voice recorder (CVR) was located on the 015 radial (north-

SEPTEMBER 1993
north east) at 23.3 nautical miles from Sierra approach, however the clearance aircraft was five miles north of the
the Kathmandu VOR. All on board, 99 did not include the Romeo fix or a direc- Kathmandu VOR.
passengers and 14 crew members, lost tion of turn. 23. Because of mountainous terrain,
their lives, and the aircraft was 13. The flight crew continued t o ask the published safe altitude within 25
destroyed. for a clearance t o Romeo, specifying a nautical miles north of Kathmandu air-
left-turn direction, but they did not port is flight level 210, which was
CONCLUSIONS receive a clearance satisfactory t o them approximately 9,500 feet higher than the
Findings nor did they initially receive any other altitude of the aircraft.
1. The Sierra VOWDME approach t o further instructions. 24. The company en-route chart
Katmandu Airport requires steep aircraft 14. The crew of TG311 did not per- showed only a track of 021 degrees mag-
descent angles. ceive that the flight had a valid clearance netic for the airway joining Romeo and
2. Company procedures and aerody- for a new Sierra approach, but under- the Kathmandu VOR, whereas the track
namic performance considerations requ- stood that they were t o continue their t o Romeo is 202 degrees magnetic.
ire that full slats and flaps configuration present approach. 25. The very high frequencyldirection
be achieved by the 13 DME point for the 15. After four requests for a left turn finding (VHFIDF) equipment, which pro-
Sierra VOWDME approach t o Kathmandu and a review of the minimum obstruc- vides an indication of bearing of an air-
Airport. tion clearance altitude (MOCA), the crew craft from near the centre of the airport,
3. The visibility at the airport was of TG311 initiated a climbing right turn does not utilize the area control centre
below the operator's limit of 3,000 from an altitude of 10,500 feet, intend- radio frequencies and has only one indi-
metres as the approach was commenced; ing t o climb t o an altitude of FL180 cator, which is located in the control
continuing the approach with such visi- (above the minimum off-route altitude tower.
bility, until reaching the outer marker, (MORA) of 17,200 feet). 26. When the aircraft was north of
was permitted by the Company's Flight the airport, the flight was communicat-
Operations Manual (FOM). ing with the area control centre and thus
4. The flight profile was proceeding I The crew's response to its bearing from the airport was not indi-
normally until a slatiflap selection of 151 cated on the VHFIDF equipment.
15 was attempted, at which time a flap the GPWS warning was 27. When requesting the aircraft's
fault occurred. position, the area control centre and
5. The flap fault prevented extension
not in accordance tower controllers only asked for distance
of the flaps beyond 15 degrees because
of actuation of the screwjack torque lim-
with the manufacturer's I from the VOR, but not radial informa-
tion, thus the aircraft position was not
iter. procedures. determined.
6. The screwjack torque limiter actua- 28. When transmitting the aircraft's
tion was likely caused by increased position the crew of TG311 gave only dis-
screwjack system friction, aggravated by 16. The tower controller cleared the tance from the VOR (DME), not the radi-
the momentary extension of the right- flight t o descend back t o 11,500 feet, al, and thus the flight's geographical
wing spoilers. which caused the crew t o stop their location was never passed t o the ACC or
7. Despite the absence o f checklist ascent and commence a descent back t o tower controllers.
guidance, the crew was able t o recover 11,500 feet. 29. A heading report by TG311 o t
the normal operation of the flaps by 17. The tower controller did not ini- "025" was likely not heard by the ACC
retracting the slatslflaps t o 1510, in accor- tially indicate whether or not TG311 controller.
dance with the operator's Aircraft Oper- could proceed t o the Romeo fix. 30. It was not possible or appropriate
ating Manual (AOM) procedure. 18. While the aircraft was in the right for the controllers t o provide heading
8. Once the flap fault was rectified, 11 turn, while nearing the 202 radial of the vectors t o TG311 or any other aircraft.
minutes prior t o impact, the crew decided Kathmandu VOR, the tower controller 31. At least some of the crew's efforts
that the flight could continue t o cleared the flight t o the Romeo fix and t o input the Romeo fix and the Simara
Kathmandu, but the aircraft was too high instructed the flight t o contact the ACC non-directional beacon (NDB) on the
and too close t o the airport t o achieve the controller. FMS appeared t o be successful, but the
required approach profile and the straight- 19. The crew continued the right turn crew did not accept the information for
in approach could not be continued. and travelled toward the north-north unknown reasons; the aircraft was not
9. TG311 made an unusual request for east, which was opposite t o the direction turned t o the south-south west toward
a clearance t o the Romeo fix, specifying t o the Romeo fix. the Romeo fix.
a direction of turn, t o join the Sierra 20. At some point in the flight from 32. The captain likely assessed that
approach t o carry out another approach. the latter portions of the 360-degree turn the copilot was having difficulty
10. In the plan view of the operator's t o the right, the crew became unaware inputting data into the FMS.
approach chart, Romeo was misleadingly of where the flight was proceeding. 33. The crew's interpretation o f the
depicted as the start of the Sierra 21. The ACC controller issued another FMS navigation data appeared t o be a
approach. valid clearance t o Romeo; the clearance problem at certain times, commencing
11. The crew may have requested a was acknowledged by TG311. near the end of the 360-degree turn
clearance t o Romeo because of i t s depic- 22. Neither the ACC controller nor the until the impact.
tion on the operator's approach chart. TG311 captain succeeded in communicat- 34. The crew's use of the FMS for nav-
12. The tower controller issued a valid ing that the flight's progress was not in igation was uncoordinated and may have
clearance t o TG311 t o carry out the accordance with its clearance when the led t o confusing system outputs, thus

ICAO JOURNAL
reducing the crew's ability t o conduct 48. There is no certification require- crew when confronted with the discon-
effective navigation problem solving. ment t o provide compass cardinal head- tinued approach.
35. It is likely that the copilot realized ing letters. 50. The search, following the accident,
that the aircraft was in a potentially dan- 49. There was no indication that the was hampered by the expectation that
gerous flight situation approximately 30 crew had received simulator training for the aircraft was operating south of the
seconds before the terrain impact. Kathmandu, even though Kathmandu is airport, by weather difficulties at the
36. The copilot communicated his con- identified by the operator as an airport time of the search, and by an absence of
cern in a mitigated manner. with special operational considerations, immediate witness information.
37. The intent of the communication which led t o increased workload for the 51. The flight crew was certified and
may not have been understood by the
captain, perhaps because of the mitigat-
ed style of communication chosen by the
copilot or because the captain misinter- EPPESEN KATHMANDU, NEPAL
112.3 0 TRIBHWAN INTL
preted the comment, or possibly for both *ATIS
Approach Control through Tower
of these reasons.
38. The GPWS provided an excessive 'KATHMANDU Tower 118.1
terrain closure rate (Mode 2A)warning.
*Ground 121.9
Alt Set: MB Trans level: FL 150 MSA -.- - --
39. The crew's response t o the GPWS Rwy Elev: 148 MB Trans alt: 13500'(9187') Apt. Elev 4390'
I
warning was not in accordance with the 9-
NX* *
Is
@Pilots are requested to acknowledge
manufacturer's procedures. ike reception of ATlS broadcast t(1
~ ~ N c o n c e r n eon
d first contact.
40. The operator's procedures for
responding t o GPWS did not provide suf-
ficient guidance t o the crew.
41. The captain assessed the GPWS 7106
n 0
warning as false. %>

42. Because of the topography near


the accident site and aircraft perfor-
mance limitations, impact with terrain
likely could not have been avoided even
if the crew had reacted instantly t o the BETWEEN D9.OAND D6.0 USE
LS LCTR FOR TRACK GUIDANCE
GPWS warning. 7~1,
c,

43. It was unlikely that any cockpit


navigation displays had failed, but the
possibility that one FMS ceased working,
because of simultaneous programming,
cannot be eliminated.

py
44. The aircraft navigation systems AUTHORIZED. ACFT MUST MAIN-
TAIN -1 (6187') UNTIL
were operating sufficiently t o allow (L gO

$r
ESTABLISHEDI N B ~ U N D ~ N
02z9
[KTM R-202) AND CLEARED BY
effective navigation. ATC FOR FINAL APPROACH. I
-'#
45. During the approach, the crew's KIT ESTABLISHED INBOUND uD13.0
workload was increased because of com- 3 N 022" LKTM R-2021CAN EXPECT

munication difficulties between them


and the air traffic control agencies and
with the other aircraft on approach, due
t o the radio clarity, language difficulties
and the use of non-standard phraseology.
46. From the first contact with the
ACC controller after the 360-degree turn
until the copilot mentioned the north
direction, the TG311 crew never dis-
cussed the aircraft's flight path.
47. The electronic flight instrument
ISSED APPROACH:
limb STRAIGHT AHEAD to VOR.continue climboutboundonVOR R 4 2 2 to~g:jhen
IGHT turn onto 4 DME Arc, intercept and follow inbound on R-106 to VOR. Then proceedvia
CIIX
~~1.43

system (EFIS) and radio magnetic indica- rtbound VOR R-291 to crossWHlSKEY at or above 9500' (5187') and ioin holding pattern.
tor (RMI) compass displays do not con- STRAIGHT-IN LANDING R
y, - 2 CIRCLE-TO-LAND
tain the letters "N", "S", "E" or "W" t o
show cardinal headings, which might
have provided directional cues t o prompt
the crew.
MDNHJ~~0 (8077
Q@W!VIS!Bl,
I
MDA(HJCEI1-VLS
t NIGHT

Jeppesen chart for the Sierra approach


to Kathmandu, Nepal. (A copy of the
operator's approach chart was not avail-
able for publication.) ANGES: Marker added.
I
@ JEPPESEN SANDERSON, INC., 1985, 1993. ALL RIGHTS RESEW

SEPTEMBER 1993
qualified for the flight in accordance Aviation (DGCA) of Nepal.
with existing regulations. rew appeared to be
52. The air traffic controllers receive Operational Recommendations
training t o the standards required by working in an The aircraft's GPWS provided a warn-
ICAO; no individual licences or ratings
are issued by Nepal. uncoordinated manner 1 ing Of terrain, but too late t o avoid
impact. The Commission recognizes that

when operating the FMS. 1


53. The ACC controller on duty, com- the accident site was on the side of a
municating with the flight was a trainee very steep mountain face and that the
with nine months experience. GPWS provided warning in accordance
54. The controller supervising the with i t s design. ICAO and the GPWS
trainee controller was also fulfilling con- igation duties. manufacturers are currently studying the
trol duties. Contributing factors were: the mis- use of the GPWS and the system itself.
55. The aircraft was certified, equip- leading depiction of Romeo on the oper- The Commission was provided with con-
ped, and maintained in accordance with ator's approach chart used by the flight siderable information regarding the acci-
existing regulations and approved proce- crew; a flap fault, although corrected, dent because of the performance of the
dures. required that the initial approach be dis- flight data recorder (FDR) and the CVR.
56. There was no evidence of any air- continued; and radio communication dif- Information from this accident will be
frame failure prior t o or during the flight. ficulties between t h TG311
~ crew and useful in seeking improvements to both
57. The mass and centre and gravity of the air traffic controllers that stemmed GPWS equipment and procedures related
the aircraft were within prescribed limits. from language difficulties and ineffec- t o i t s use, not just for the A310 but for
tive discussion of apparent unresolved all aircraft requiring GPWS.
Cause problems. During the investigation of the acci-
The probable causes of the accident dent, the Commission noted that at sev-
were: TG311 flight crew's management SAFETY ACTION eral times during the flight the crew
of the aircraft flight path wherein the Interim Recommendations of the appeared t o be working in an uncoordi-
flight proceeded in a northerly direction Commission nated manner when aperating the FMS.
which was opposite to the cleared point On 20 October 1992, the Commission As noted during the investigation, the
Romeo t o the south; ineffective radio issued interim recommendations in order possibility exists that technical problems
communication between the area con- t o address some of the safety issues can be introduced as a result of both
trol centre controller and the TG311 revealed as a result of the investigation pilots inputting data into the FMS.
flight crew which allowed the flight t o up to that time. Letters were sent t o the Although no direct evidence was found
continue in the wrong direction, in that Department of Aviation CDOA) Thailand, that technical problems were created by
the TG311 crew never provided the air- the Directeur General de I'Aviation Civile the crew's action, the potential for such
craft's VOR radial when stating DME and (DGAC) of France, the Administrator of problems is real. These considerations
the controller never solicited this infor- the Federal Aviation Administration are not unique t o the A310 alone, but
mation and thus the aircraft's position (FAA) of the United States of America, are equally applicable t o other FMS-
was not transmitted at any time; and the Certification Director of the Euro- equipped aircraft. It is recognized that
ineffective cockpit crew coordination by pean Joint Airworthiness Administration crew training emphasizes the require-
the TG311 crew in conducting flight nav- (JAA) and the Director General of Civil ment for crew coordination when using
the FMS, but the events of this accident
reinforce the importance of this aspect
REPORTED CFIT ACCIDENTS 1 of training.
Accordingly, the Commission makes
The following CFIT accidents involving civil airliners were reported to ICAO d the following recommendations (to be
1992 and part of 1993. In some cases ICAO did not receive complete reports on coordinated by the DGCA of Nepal):
currences. GPWS manufacturers and ICAO be
A CFIT accident is one in which the aircraft is controllable but flies into obs made aware of the operational and tech-
o r terrain without the awareness of the crew. nical information related t o GPWS per-
3 January 1992 24 March 1992 6 January 1993 formance in this accident.
Beech 1900 Boeing 707-300 de Havilland DHC-8 Manufacturers of flight management
Gabriels, NY Athens, Greece Paris, France
%$@"
3.: .> systems be made aware of the operational
United States and technical aspects of this accident.
8 June 1992 13 January 1993 @&$ tm

20 January 1992 Beech 99 Airliner Embraer 100 Bandeirante $ti The operators and training centres be
Airbus A320 Fort McClellan. AL. Sellafreld. Cumbria +.I encouraged to continue to emphasize, t o
Strasbourg, France United States United Kingdom Q
if
,I operators of aircraft equipped with FMS,
the importance of coordinated use of
9 February 1992 31 July 1992 25 May 1993
Convair 640 Airbus A310 Metro I1 the FMS. 17
Kafountine, Senegal Kathmandu, Nepal Santa Fe, NM
United States
15 February 1992 28 September 1992 The operational safety recommendations contained in
McDonnell Douglas DC-863 Airbus A300 the report of the Commission fox the AecCdent
Kathmandu, Nepal Investigation of TG3ll are currently under review by
tbe ICAO Secretariat as part of it6 programme to
address the CFiT issue.

ICAO JOURNAL
J O U R N A L
V O L U M E 54, N O . 5 JUNE 1 9 9 9

HUMAN FACTORS TODAY:


Managing human error
- - -
ICAO providing the tools and leadership needed
for enhancing safety worldwide
Safety has been bolstered through recent efforts to heighten awareness of human factors, but a fill
commitment on the part of States and the industry is necessary before the aviation community can
achieve the highest level of safety possible.

knowledge about human factors. The practical since it must deal with real
DR.ASSADKOTAITE reason the community must respond to problems in a real world. Through this
ICAO COUNCIL this need is, of course, to ensure that programme, ICAO has provided the avi-
civil aviation continues to achieve its ulti- ation community with the means and
tools to anticipate human error and con-

H
UMAN FACTORS is a critical mate goal: the safe and efficient trans-
aspect of aviation safety, one portation of passengers and goods. tain its negative consequences in the
that ICAO began to address - Before the aviation community can operational environment. Furthermore,
primarily by sensitizing the aviation apply this knowledge with any success, ICAO's efforts are aimed at the system
industry to this new dimension of avia- however, there are two fundamental req- -not the individual.
tion safety - almost a decade ago. uisites that must be met. Firstly, the avi- It is important to have a broad, sys-
ICAO convened the first in a series of ation industry must ensure that temic approach to safety and human fac-
global symposia on flight safety and human-technology interaction remains tors because the requirement for
human factors in 1990. From the begin- human-centred. ICAO adopted a philos- integration goes well beyond the consid-
ning, when the first event was held in a ophy of human-centred automation as eration of isolated human-technology
city known then as Leningrad, there was early as 1991 and has made this one of interface issues. Human performance
a conviction that international aviation the pillars of its CNS/ATM systems con- takes place in an operational context,
could make enormous progress in cept. A philosophy of human-centred and human factors knowledge must
improving safety through the application automation is the only safeguard against therefore be applied to operational sys-
of human factors knowledge. the development of uncooperative tems. It is because of this that the activi-
That first symposium was a turning human-technology interfaces with the ties of the ICAO flight safety and human
point and set the stage for follow-up potential for safety breakdowns. factors programme must be integrated
meetings in the United States in 1993, in The second requisite deals with inte- with two closely connected, major sys-
New Zealand in 1996 and, finally, in Chile gration. The industry has designed temic safety initiatives.
in 1999. There have been encouraging excellent technology that has undoubt- The first of these initiatives, the global
developments since 1990, but it must be edly contributed to improvements in aviation safety plan (GASP), was devel-
emphasized that we still have challenges safety, but we need to take another look oped by the ICAO Air Navigation
to pursue: nine years after the Leningrad at the operational contexts into which Commission in 1997 and subsequently
Symposium, human error remains a sig- these technologies are deployed. In approved by the ICAO Council and
nificant safety concern. other words, human capabilities and Ern- endorsed by the ICAO Assembly. GASP
The purpose of the worldwide sym- itations should be taken into account was designed to coordinate and provide a
posia and 10 regional seminars which when defining the blueprint for the sys- common direction to the efforts of
were held in the past decade was to tem and before any system can enter into Contracting States and the aviation
increase the awareness of States, indus- operation. ICAO has taken the initiative industry to the extent possible in safety
try and organizations in all ICAO in this challenge by developing human matters. It is a tool that allows ICAO to
regions about the importance of human factors standards that consider human focus resources and set priorities, giving
factors. The ongoing implementation of performance in present and future oper- emphasis to those activities that will con-
the ICAO communications, navigation, ational environments, and the aviation tribute the most to enhancing safety. It
surveillance and air traffic management community is strongly encouraged to fol- should come has no surprise, therefore,
(CNS/ATM) systems concept has intro- low these standards. that the flight safety and human factors
duced new challenges, and also new pos- The ICAO flight safety and human fac- programme is among the six major activ-
sibilities for human error, thus creating tors programme is safety-oriented and ities that comprise the plan. However,
yet another opportunity to apply our vast operationally relevant. Moreover, it is before GASP can have much effect, it is

ICAO JOURNAL
crucial to achieve the highest
degree of global cooperation in
implementing the major ele-
ments. ICAO can provide the
framework for GASP to function,
but it will ultimately be the con-
certed effort of the aviation com-
munity, States and industry - a
global cooperation - that will
largely determine its success.
This worldwide cooperation
underlies another major sys-
temic safety initiative: the ICAO
universal safety oversight audit
programme. This is a pro-
gramme of regular, mandatory,
systematic and harmonized safe-
ty audits carried out by ICAO
in all Contracting States. The
information obtained from these
audits will allow States, given
reasonable time, to remedy defi- A deeper understanding of human error holds the key to continued improvement in the air
ciencies in safety oversight safety record.
responsibilities. Mechanisms
have been developed that will which flight takes place. On the one States engage in far-reaching coopera-
enhance transparency and increase dis- hand, the physical environment, with tion on safety problems related to the
closure of results, as well as eventually extreme temperatures and pressures, influence of human factors, and directed
expand the programme to all areas rele- makes unsupported human life impossi- ICAO to develop and introduce material
vant to civil aviation safety. There is no ble. In addition, speeds allowing ultra on various aspects of this issue. The goal
better example of global cooperation in long-haul, trans-meridian operations in now, as then, is to further improve safety
action. short periods of time require careful in aviation by making States and the
Profound social and political forces consideration of basic human perform- industry more aware of, and responsive
are reshaping our world on a daily basis, ance issues such as jet-lag and circadian to, the importance of human factors in
inexorably leading towards globalization. disrythmia. On the other hand, the civil aviation operations. This is accom-
Solutions to safety problems lie in the socio-economic environment, with mar- plished through the provision of practi-
cooperation that exists among countries, ket demands that require aviation organ- cal human factors material and measures
and among countries and the industry at izations to attempt to produce "more developed on the basis of experience.
large. All this needs to be supported by with less" in order to remain economi- The full understanding of human
a corresponding level of global coordi- cally viable, generates inevitable bur- error holds the key to continued success
nation in all aspects of civil aviation with dens upon those who operate, maintain in providing people everywhere with the
relevance to safety. Safety, after all, is not and control the system. safest means of transportation ever cre-
a national, regional or even continental One major obstacle remains to make ated. Humans, it must be said, can never
issue, but an issue on a global scale. As aviation's safety record all but perfect: outperform the system that bounds
a global body, ICAO can and will assume human error. Human error is a safety them. In no small way, safety practition-
its leadership role through various ini- obstacle not just in the cockpit but in ers in all disciplines of aviation are the
tiatives, but States -through their reg- every process surrounding flight opera- players who will define further strate-
ulatory bodies - and the international tions, from designing and manufacturing gies for understanding human error,
aviation community at large must be aircraft and navigation equipment to the making it possible to achieve our safety
fully committed to the attainment of the commercial decisions that affect daily objectives. 0
highest level of safety. operations.
The safety of civil aviation is particu- The objective today remains un- This article is an adaptation of the opening address
given by 1C40 Council President Dr. Assad Kotaite at the
larly notable when one considers the changed from 1986, when the ICAO 4th Global Fhght Safety and Human FactorsSymposium
extraordinarily hostile environment in Assembly proposed that Contracting held in Santiago, Chile from 12 t o 15 April 1999.

JUNE 1999
CRM training primary line of defence against
threats to flight safety, including human error

................................................................................. erable evidence has also been accu- recently defined CRM as "instructional
L. HELMREICH
ROBERT mulated to show the effectiveness of strategies that seek to improve team-
UNIVERSINOF TEXAS CRM training in changing behaviours work in the cockpit." While effective
(UNITEDSTATES) and attitudes. teamwork is clearly important, it is not
................................................................................. The situation is not, however, entirely the primary goal of CRM training.

S
INCE THE END of the 1980s the positive. There are four areas of con- The following description is a more
concept of crew resource manage- tention. Firstly, CRM has been faulted for accurate representation of the latest CRM
ment (CRM) has spread world- failing to eliminate pilot error and acci- programmes: crew resource management
wide, has come under some criticism, dents. Secondly,there is confusion among is a subset of the human factors disci-
and is now finding its place in our safety some members of the research and oper- pline which is concerned with human-
culture by focusing on error and threat ational communities about its definition. machine and human-human interfaces
management. CRM training is mandated Also, it is claimed by some critics that and the integration of human operators
for pilots in all ICAO Contracting States, CRM issues are too subjective to be eval- and maintenance personnel within an
and airlines of some countries of the Eure uated fairly, thus subjecting pilots to the overall system. CRM consists of the
pean Union will be required to evaluate risk of discrimination. Finally, there is application of human factors knowledge
individual proficiency in CRM-related confusion about the role of CRM pro- to the special case of flight crews and
behaviours. CRM training is also being grammes within organizations. their interactions with each other, with
extended to other personnel in the avia- Has CRM failed? The contention that other groups, and with the technology in
tion system, including flight attendants, it has not achieved its mission because the system. Broadly defined, CRM con-
dispatchers and maintenance technicians. accidents still occur is based on a pro- sists of the effective utilization of all
As far as its application is concerned, found misunderstanding about human available human, informational, and
CRM is certainly a success story: consid- capabilities and limitations. By their very equipment resources toward the goal of
nature, humans are prone to errors safe and efficient flight. More specifical-
and no amount of training, howev- ly, it is the active process employed by
er sophisticated or intensive, can crew members to identify existing and
change human nature. Humans potential threats and to develop, commu-
will make errors, and accidents nicate, and implement plans and actions
and incidents will occur in complex to avoid or mitigate perceived threats.
systems. CRM also supports the avoidance, man-
The definition of CRM is misun- agement, and mitigation of human
derstood. Although CRM pro- errors. The secondary benefits of effec-
grammes were clearly rooted in tive programmes are improved morale
efforts to reduce accidents caused and enhanced efficiency of operations.
by "pilot error", comprehension of Many are of the view that CRM issues
the programme goals over the are subjective and cannot be evaluated.
years has faded, perhaps in part Fear of inequitable evaluation is certain-
because of the extension of the ly a legitimate concern of pilots whose
training concept to flight atten- livelihoods may be threatened. However,
FlgUre I. A model ot threat ana error manage- dank and other personnel. An indi- contemporary CRM programmes focus
The model features external threats Iin cation of this misunderstanding on specific and well-defined behaviours.
red boxes), internal threats (blue box), CRM
behaviours (in dark green boxes) and outcomes can been seen in the work of scien- Those behaviours chosen for evaluation
(light green and black boxes). tists in the United States who in the Europcan Union (which are close-

ICAO JOURNAL
ly related to the "behaviourial markers"
defined by the University of Texas
research group) are objective and
observable.
What is the role of CRM in the organ-
ization? Some have argued that CRM
should ultimately disappear as it be-
comes fully integrated into technical
training. With hindsight, however, most
experts now realize that crew resource
management is, and should remain, a sep
arate aspect of training. Crew resource
management falls within the area bridg-
ing the safety, flight training and flight
operations departments, and training in
crew resource management is ongoing
and driven by objective data reflecting Flight operations are part of a complex system that is heavily influenced by national,
operational issues. CRM is not a one- organizational and professional cultures.
time intervention, but rather a critical
and continuing component of a safety tures characterized by high power dis- attitudes and policies about human error,
culture. tance show great concern for the group the openness of communications between
and harmonious relationships, and def- management and flight crew, and the level
Culture: the missing element erence to leaders. Another relevant of trust between flight crew and senior
Early CRM programmes and investi- dimension concerns the attitude toward management. Organizational culture also
gations of human error in accidents rules. Those high on this attribute influences norms regarding adherence to
viewed the cockpit as an isolated uni- believe that rules should not be broken, rcgulations and SOPs. Of great impor-
verse. With growing sophistication, it is that written procedures are needed for tance, the organizational culture deter-
now understood that flight operations all situations, and that strict time limits mines the level of commitment to safety
are part of a complex system that is should be observed. This dimension has and the strength of a safety culture.
heavily influenced by cultures. There are proved to be one on which there are Professional culture. Many professions
three intersecting cultures that surround large and highly significant cultural dif- exhibit strong cultures and develop their
every flight crew - national, organiza- ferences. At the high end, many Asian own norms and values along with recog-
tional, and professional. One expert has cultures are rule-oriented, while the nizable physical characteristics such as
defined culture very aptly as "the soft- United States and United Engdom are uniforms or badges. In aviation, the pos-
ware of the mind." More technically, cul- examples of the other end of the contin- itive aspects of the professional culture
ture consists of the shared norms, values uum, demonstrating much lower con- are shown in strong motivation to do
and practices associated with a nation, cern for rules and written procedures. well and a high level of professional
organization or profession. Some facets Looking at the cockpit environment, pride. There is also a negative compo-
of national culture may influence behav- national culture influences how junior nent that is manifested in a sense of per-
iour in the cockpit. crew members relate to senior pilots, sonal invulnerability. University of Texas
National culture. Two related dimen- including their willingness to speak up researchers have found that the majori-
sions of national culture have particular with critical information. It is demon- ty of pilots of all nations agree that their
relevance for aviation: individualism ver- strated in the way information is shared. decision-making is as good in emergen-
sus collectivism, and "power distance" Culture influences adherence to stan- cies as it is in normal situations, that
(i,e. the remoteness of those in society dard operating procedures (SOPs). It their performance is not affected by per-
having the authority to make major poli- has also been found, unexpectedly, that sonal problems, and that they do not
cy decisions; in a culture with low power culture is strongly associated with a per- make more errors when highly stressed.
distance, for example, decision makers ception of automation and with attitudes While the positive aspects of profession-
are generally accessible). about the appropriate use of automation. al culture undoubtedly contribute to avi-
Those from individualistic cultures Organizational culture. Organizations ation's excellent safety record, the sense
that also have low power distance tend to can function within a national culture or of invulnerability can lead to risk taking,
focus on the self, autonomy, and person- can extend across national boundaries. An failure to rely on fellow crew members,
al gain, while people from collectivist cul- organization's culture demonstrates its and errors.

JUNE 1999
CRM programmes built on data behaviours that have been associated with nal errors, the crews themselves may
CRM programmes provide a primary accidents and incidents (and that form the err and, again, CRM behaviours stand as
line of defence against the threats to basis for contemporary CRM training). the last line of defence. If the defences
safety that abound in the aviation system Data from LOSA provide a valid picture of are successful, error is managed and
and against human error and its conse- system operations that can guide organi- there is recovery to a safe flight. If the
quences. Today's CRM training is based zational strategy in safety, operations and defences are breached they may result
on accurate data about the strengths and training. A particular strength of LOSA is in additional error or an accident or inci-
weaknesses of an organization. Building that the process identifies examples of dent. In our three-airline database, 72
on detailed knowledge of current safety superior performance that can be rein- per cent of all flights faced one or more
issues, organizations can take appropri- forced and used as models for training. threats to safety, with the number of
ate remedial actions. Data collected in LOSA threats on a flight ranging from zero to
There are five criti- Type of incident A c are proactive and can 11.The average was two threats to safety
cal sources of data 01 be used immediately to on each flight. The most common
which CRM program- prevent adverse events. threats encountered were challenging
mes can be built, eacl University of Texas re- terrain (on 58 per cent of flights),
source illuminating a Figure 2. Threat and error rate at searchers have partici- adverse weather (28 per cent), aircraft
different aspect of flight three airlines ( A, B and t). pated in eight audits with malfunctions (15 per cent), unusual ATC
operations. Data can be a total of nearly 4,000 commands (11 per cent), external
obtained from formal evaluations of per- flights observed. The data from the errors, including ATC, maintenance,
formance in training and on the line, as three most recent audits, which include ground handlers, etc. (8 per cent), and
well as incident reports from non-puni- threat recognition and error manage- operational pressures (8 per cent).
tive systems that encourage open com- ment, are described below. These three Threats to safety arose most frequently
munication and surveys of flight crew LOSA projects were conducted both in in the descent, approach and landing
perceptions of safety and human factors. the United States and in international phases (40 per cent).
Quick access recorders provide informa- operations and involved three air carri-
tion on the conduct of the flight. It ers, including two based in the United A model of crew-based error
should be remembered, however, that States. Errors made within the cockpit have
while these data provide a reliable indi- received the most attention from safety
cation of what occurred, they do not indi- A model of threat and error investigations, and crew error has been
cate why it happened. Line operations Data are most valuable when they fit implicated in around two-thirds of air
safety audits (LOSA) are also an impor- within a theoretical or conceptual frame- ~ r a s h e s Our
. ~ analyses of human error
tant source of information. work. Our research group has developed have led us to reclassify and redefine
The nature and value of LOSA. Line a general model of threat and error in error in the aviation context. Opera-
operations safety audits are programmes aviation (see Figure 1, page 6). tionally, flight crew error is defined as
that use expert observers to collect data As the model indicates, risk comes from action or inaction that leads to deviation
about crew behaviour and situations on both expected and unexpected threats. fiom crew or organizational intentions or
normal flights. They are conducted under Expected threats include such factors as expectations. This definition classifies
strict non-jeopardy conditions, meaning terrain, predicted weather, and airport con- five types of error:
that no crew member is at risk for ob- ditions. Those that are unexpected include intentional noncompliance or viola-
served acti0ns.l Observers code threats air traffic control (ATC) commands, sys- tions of SOPS or regulations (examples

--
to safety and how they are addressed, tem malfunctions, and operational pres- include omitting required briefings or
errors and their management, and specific sures. Risk can also be increased by checklists) ;
errors made outside procedural, in which the intention is
the cockpit by, for correct but the execution flawed;
Intentional Procedural cornmunica,ion Proficiency Or;rnal example, A X , main- communication ervon that occur when
Type Of fleet noncornpllance
- tenance and dispatch. information is incorrectly transmitted or
External threats are interpreted (examples include incorrect
countered by the defen- readback to ATC or communicating
ces provided by CRM wrong course to the other pilot);
lxhvbum Wkn suc- groficiency enom that indicate insuffi-
cessful, these lead to cient knowledge or lack of stick and rud-
Figure 3. Percentage of error types within different types of fleets
at one airline. Included are data related to two types of advanced- a safe flight. der skills; and
technology flight decks and two types of conventional aircraft. In addition to exter- operational decision errors in which

ICAO JOURNAL
CREW RESOURCE U A N A G E M E

crews make a discretionary decision that The model of threat and error illus- and 3. (A consequential error is one
unnecessarily increasesrisk (examplesin- trated in Figure 1 aids analysis of all resulting in an additional error or an
clude extreme manoeuvres on approach, aspects of error, response, and outcome. undesired aircraft state.) This finding
choosing to fly into adverse weather, or The failure or success of defences such indicates the importance of airlines'
over-relianceon automation). as CRM behaviours can also be evaluat- determining the status of their own oper-
Three types of responses to these ed. Errors thus classified can be used not ations rather than assuming that their
errors have been identified. In the first only to guide organizational response but organization conforms to an industry
instance, the error is detected by the as scenarios for training, either in class- standard. Variability can result from dif-
crew and managed before it becomes room or line-oriented flight training ferences in the operating environment,
consequential or leads to additional error. (LOFT). as well as from differing organizational
In the second case, the error is cultures and subcultures. Of particu-
detected, but the crew's action or lar interest is the range of variables
inaction leads to a negative outcome. Phase Of flight Errors errors which became consequential, termi-
The last type of response is, in fad, a nating in an undesired state.
failure to respond. The crew does ( Certainly, there is differential risk
not react either because the error is at the three organizations which par-
undetected or because the crew ticipated in the study. The three air-
chooses to ignore it. Figure 4. Percentage of errors and consequential lines share some common factors,
errors experienced by three airlines.
Definition and classification of but the data presented here are
errors and responses are based on the shown primarily to indicate the type of
observable process, without considera- LOSA results information that can be obtained and uti-
tion of the outcome. There are three pos- Examination of the aggregate data lized for safety.
sible outcomes: in the first case, it is from the first three LOSAs in which Phase of flight is also strongly associ-
inconsequential, meaning that the error error was measured is instructive. ated with the occurrence of errors and
has no effect on the safe completion of Errors were committed by 73 per cent of their consequences, as shown in Figure 4.
the flight. This is the model outcome, the crews observed. The number of Consistent with analyses of worldwide
one that is illustrative of the robust errors on a flight ranged from zero to 14, approach and landing accidents, the
nature of the aviation system. In another with an average of two errors per flight. highest percentage of errors (and the
case, the error places the aircraft in a The most frequent error type identified highest percentage of consequential
condition that increases risk (undesired by the study was intentional non-compli- [serious] errors) occurs during this
state). This outcome includes incorrect ance (or violation), followed by those of a phase of flight. Clearly, special attention
vertical or lateral navigation, unstable procedural nature. The high percentage should be directed toward enhancing
approaches, low fuel state, and hard or of errors in the procedural non-compli- performance in the approach and land-
otherwise improper landings. A landing ance category is alarming (this finding is ing phase.
on the wrong runway or at the wrong air- described in greater detail below). Specific errors. Although a wide array
port would fall under this category. In Procedural errors doubtless have mul- of error classifications was observed,
addition, the response to error, as noted tiple causes. They can reflect the inherent some major problem areas emerged.
above, can result in another error that limitations of humans in accomplishing Earlier audits pointed to the use of auto-
again initiates the cycle of response. difficult tasks, often under high work- mated systems as an industry-wideprob
High risk situations can be mitigated load conditions, or they may be an indi- lem. Consistent with these findings, the
by crew action; for example, recognition cation that the procedures themselves most frequent classification of error in
of an unstable approach can trigger a are sub-optimal. Of all the errors LOSA involved the operation of automat-
decision to initiate a go-around. Response observed, 18 per cent were corrected ed systems (mode control panel and
to an undesired state may end in another before they could have any conse- flight management computer). Errors
error or, in the worst case, the result is quences, 5 per cent resulted in a negative included wrong settings, wrong modes
an accident or incident. outcome despite detection by the crew, and failure to verify settings, along with
and 77 per cent elicited no response. numerous others. Overall, these
1. In practice, members of the University of Texas Of particular significance was the fact accounted for 31 per cent of all errors.
project have trained observers from participating air- that there were very large differences in The second highest classification was
lines, and also serve as observers. Their presence
across all organizations allows the research team to threat, error, and the percentage of checklist errors (24 per cent) such as
make valid cross-airline comparisons. errors that became consequential non-standard terminology, procedural
2. Early investigations tended to focus on the crew as
the sole causal factor. Today, of course, it is known between fleets within one airline, and errors, performance from memory, and
that almost all accidents are system-related. between airlines, as shown in Figures 2 failure to use required challenge and

JUNE 1999
response methods. The third highest ization of operations cannot be achieved ior crew members, many current prob-
category, accounting for 13 per cent of with idiosyncratic adherence to proce- lems seem to be associated with weak
errors, consisted of sterile cockpit viola- dures. There is also compelling evi- leadership and the abdication of author-
tions. Fourth highest, at 8 per cent, dence that safety threats are associated ity. While the importance of the identi-
were ATC-related crew errors such as with violations. For example, a Flight fied markers is not surprising, the
missed calls, omitted information, and Safety Foundation (FSF) analysis of results do provide important validation
acceptance of instructions that increase global approach and landing accidents of the importance of CRM-related
risk unnecessarily (i.e. "slam d u n k found that more than 40 per cent of behaviours.
approaches). The fifth highest category these accidents involved violations of
(5 per cent) consisted of briefing errors, SOPS. Also, an analysis of LOSA data Data, CRM and safety culture
failure to conduct required brietings or indicates that those who commit inten- The analysis of data from a variety of
leaving out required information. The tional non-compliance errors are more sources (training evaluations, incident
remainder of errors fell into a variety of likely to commit other types of errors. It reports, surveys, and LOSA) aids organ-
categories. can be concluded that violations are izations in the diagnosis and under-
associated with greater risk in opera- standing of their culture and its
Violations matter subcultures. Without an under-
The research group was dis- standing of their own cultures,
mayed by the high proportion CRM not a one-time intervention.,
is organizations cannot mount
s
-a.
* "7
of intentional non-compliance effective programmes to opti-
errors found in the data. Several but rather'a critical and continuing combonent rnize them. Data on how crews
points regarding these violations deal with threats to safety and
should be considered. First, as avoid and manage error help
noted above, there were very organizations develop and main-
large differences between airlines and tions, and further analyses may give us tain a safety culture. LOSA data, in par-
between fleets within airlines. Hence, greater insight into the nature of this ticular, are of enormous value because
one cannot generalize from these data relationship. they are proactive and allow organiza-
about the general frequency of viola- tions to take appropriate action before
tions in the global aviation system. This CRM as a countermeasure accidents and incidents occur. Proactive
point is further emphasized by the fact One of the most informative aspects interventions are a defining characteris-
that the three carriers included in the of LOSA data is the ability to link threat tic of an effective safety culture.
study all came from countries that recognition and error management with Data also iden* critical areas for
scored very low on commitment to the specific behaviourial markers that ongoing CRM training. However, as noted
rules. It would be incorrect to assume form the core of crew resource manage- above, CRM is not a universal panacea for
that pilots from other cultures would be ment. These emerge very clearly in safety problems in the aviation system.
equally cavalier in disregarding formal observer ratings and provide a clear Accidents and incidents almost always
rules. On the other hand, the universal portrait of the actions taken by effective have multiple roots and many cannot be
pilot belief in personal invulnerability crews. Those who deal proactively with changed by training alone. Organizations
may foster a disregard for rules. The safety threats and error management nurturing a safety culture must also deal
fact that many rules are broken does not exhibit several behaviours. They brief with those issues identified by LOSA and
imply that pilots have a death wish or one another on known safety threats; other data sources. Interventions may
feel contempt for formal requirements. ask questions and speak up; constantly include making revisions to procedures,
One must also consider the possibility re-evaluate their decisions; communi-
continued on baae 29
that the proliferation of regulations may cate operational plans clearly; prepare
have created a contradictory, unwieldy and plan for safety threats; distribute the Robert L. Helmreich is a Professor at the University of
Texas at Austin and leader of the University of Texas
and inefficient operating environment workload and tasks; and exercise vigi- Team Research Project, an ongoing study of crew re-
that invites violations. lance. Captains show leadership. source management issues. The research reported on
here, focusing on the management of human error,
Although many violations may be Leadership is an overarching behav- commenced in late 1997.
committed with the good intention of iour that governs interaction on the The research described in this article was conducted
with financial support from the US. Federal Aviation
increasing operational efficiency, organ- flight deck. Although much of the atten- Administration and was made poss~bleby the partici-
izations cannot and should not tolerate tion in early CRM programmes was their patlon of three airlines and the full cooperation of
personnel, who also assisted with data analysis
disregard for established procedures. directed toward overcoming the effects and development of conceptual models.
previous article by Prof. Helmreich, with focus on
There are several compelling reasons of autocratic captains who failed to solic- theAeffect of national culture on flight crew behaviour,
for this. One is, of course, that standard- it or accept critical information from jun- appeared in October 1996 (pp. 14-16).

ICAO JOURNAL
Tools developed by aircraft builder designed
to help airlines manage human error
Boeing's latest h u m a n factors analysis tool, to be released to the industry later this year o n a no-cost
basis, focuses o n flight crew compliance with procedures.

occurrence, we can never be certain it today. So why are errors often blamed on
will not occur. Consequently, we must negligence or incompetence without
enhance our ability as an industry to looking more broadly at the system and
manage error in order to mitigate its the way it supports (or doesn't support)
consequences and to learn what system- human performance? Even when more

H
ELP for efforts to resolve human atic factors contribute to its occurrence. serious incidents and accidents occur, it
factors issues is available horn is rare to see a thorough human factors
aircraft manufacturers, and Boeing Error management tools analysis conducted.
in particular has identified several ways Aviation has traditionally relied on If the aviation industry is to make the
in which a manufacturer can play a s u p selection, training, licensing and detailed human performance gains necessary for
portive role. For example, Boeing is work- written procedures to assure saiety. dramatic reductions in the accident rate,
ing to develop and provide pilot training While these are important barriers to it needs more extensive and reliable
aids and to supply tools for managing human error, this emphasis ignores the feedback on how humans interact with
human error and mechanisms for feed- very real contributions that design, envi- technology in the real world. The indus-
back, and is also conducting human fac- ronment, and other factors make to try needs to foster further development
tors research. The company supports human performance. An over-reliance of human factors tools, databases and
initiatives in the human factors area by on discipline to make the system work support policies across all sectors of the
ICAO and the International Air Trans- well characterizes many government industry, not just for flight crews. Of
port Association (IATA). authorities as well as air carriers. The course, the biggest challenge will be the
Many readers will be familiar with the phrase "blame and train" probably best political and legal frameworks needed to
training aid designed to prevent con- describes the predominant attitude to- encourage honest reporting when
trolled flight into terrain (CFIT) acci- wards those who err and are caught. As human error occurs.
dents; developed by a task force formed a result, human performance issues are In 1991 Boeing initiated an effort to
by ICAO and the Flight Safety Foun- often not given the systematic level of shift the focus of accident analysis away
dation (FSF), the training aid was pro- analysis they deserve in order to prevent from primary cause to the development
duced by Boeing. A more recent joint their future occurrence. of accident prevention strategies. This
effort that involved manufacturers has Yet, there has always been an implicit was accomplished by reviewing and
culminated in the development of the assumption that the trained pilot or analysing commercial jet aeroplane acci-
upset recovery training aid. These re- mechanic can always be counted on to dents over a 10-year period (1982-91),
cent examples of training aids bear con- remain sufficiently
siderable similarity to the development flexible and creative
of the wind shear avoidance training aid to fill the gaps in the
in the mid-1980s. system to maintain
A manufacturer can help in capitaliz- safe performance.
ing on human factors to achieve a signif- Given the often unpre-
icant reduction in the accident rate. This dictable nature of the
is accomplished by developing and pro- aviation operating
- .pilot =
considerations; (C) non-flyinq
environment, there is adherence to procedure; (D) .embedded
viding methods and tools to manage
20
human error more effectively. Over the no doubt that this piloting skills; (E) design improvement;
past several years, our human factors uniquely human abili- (F) ca~ainlinstructor~ilot exercise of
lo
authority; and (G) maintenance or
specialists have been focusing on the ty has been a major inspection action.
fact that human error is inevitable. While factor in making avia- Boeing data
all of us can do our best to prevent its tion as safe as it is

JUNE 1999
HUMAN FACTORS

an accident or serious incident. Un- ture of the U.S. Air Transport Associa- the flight crew is not subject to punish-
fortunately, the latter approach is rarely tion (ATA) Human Factors Committee, ment or disciplinary action unless they
exercised in incident investigations, industry involvement has been expand- were deliberately neglectful or acted in a
resulting in a mostly negative view of ed to include a multi-cultural team in reckless manner. Within this framework,
flight crew performance when proce- order to adapt PEAT into an incident the crew's professionalinput is sought to
dures aren't followed. Together we con- analysis tool that meets global needs. improve the overall understanding of
sistently fail to consider the number of The industry team consisted of eight air- what happened and to gain insight as to
accidents that may have been prevented lines from the United States, Europe, why it happened. PEAT provides the
because the crew did deviate from the
procedures.
PEAT is similar in design to MEDA
and likewise assumes that there are rea-
sons the pilot either failed to follow the
procedure or made an error in following
it -that is, the error was not malicious.
This allows the analyst to interview the
pilot involved and document the error
and the reasons behind it. Both tools
take advantage of what we know about
the cognitive or decision-makingaspects
of procedure adherence, and they offer
an inventory of the types of procedural
errors that might occur and the factors
that can influence human use of proce-
dures.

Development of PEAT For some time now, human factors


Boeing first tried to devel-
op a better understanding c -
I specialists have been focusing
methodology for guiding the collection
of this input along with other relevant
non-adherence with proce- facts and data.
dures by returning to the on the fact that human error is inevitable. In contrast to the wide variability in
accidents that had been current airline investigation methods,
analysed for its accident pre- rn =-

vention strategies study, and applying an and Asia, working together with Boeing
PEAT provides consistency in applica-
tion and results. The PEAT form,
analysis that focused on the cognitive and the International Federation of Air designed to be used by a trained safety
factors that could be responsible. Line Pilots' Associations (IFAZPA) . The officer, can facilitate the investigation of
Despite repeated attempts to apply the team participated in an eight-month field specific types of incidents, i.e. those
analysis across various accidents, how- validation using preliminary paper ver- involving non-adherence to procedures.
ever, Boeing finally concluded that the sions of PEAT to investigate their own As such, it addresses all the pertinent
disparity in accident report quality, their incidents involving significant non-adher- analysis elements. The Boeing-industry
inconsistency in addressing human fac- ence to procedures, and to adapt PEAT team found that by asking such ques-
tors issues, and the inability to interview to better meet their requirements. tions they obtained information that sub-
crew members made it impossible to The software version of PEAT has stantially expanded their ability to
achieve a reliable result. been designed to facilitate a paradigm understand the incident.
Boeing then turned its attention to shift in how incident investigation is con- The adoption of the PEAT philosophy
serious operational incidents. As a ducted. PEAT is based on a philosophy by the safety officer also facilitates the
result, PEAT has developed into a struc- which acknowledges that professional shift of the investigation focus away from
tured, in-depth analytic tool based on a flight crews rarely fail to comply with a what happened and who is responsible to
cognitive approach. It is designed to procedure intentionally, especially if it is why it happened by focusing on the key
facilitate incident investigations and to likely to result in an increased safety contributing factors. As stated above,
aid in the development of countermea- risk. It therefore requires the airline to 1. Federal Aviation Administration, Human Factom
sures. While the initial accident-oriented explicitly adopt a non-jeopardy approach Team Report on the InterfacesBetween Flight Crews and
Modern Flight Deck Systems, Washington, D.C., 1996.
effort was accomplished within the struc- to incident investigation. In other words,

JUNE 1999
flight crews rarely make procedural ent types of information on similar or usually asked (thus expanding the scope
errors intentionally; however, there are related incidents and offer an opportuni- of the investigation); encourages ques-
circumstances and factors that affect ty to spot potential risk areas. tions no one dared to ask before; and
crew decisions and can contribute to Finally, PEAT provides a mechanism helps move investigators away from the
such errors. Therefore, the desired for Ieedback and data sharing. PEAT "blame and train" mindset. Feedback
change in crew behaviour can only be facilitates the communication of relevant also indicated that PEAT is flexible
accomplished by objectively addressing information to various departments, both enough to support airline-unique needs.
why the incident occurred. internal and external, to the airline
A model of human information pro- organization. For example, if an investi- PEAT implementation
cessing, depicted in Figwe 2, empha- gation reveals the need for improvement Boeing Flight Technical Services will

-
sizes this point. Crew actions are the in the area of procedural development, be responsible for worldwide industry
consequence of complex mental opera- the relevant information can be readily implementation of PEAT starting in
tions that are characteristic of human shared with both the flight standards and 1999. Effective adoption and application
cognition and that are clearly influenced trainjig departments. If maintenance has of the PEAT process and software
by available information and the sur- been identified as a cantributing factor, requires hands-on training. Training and
rounding environment, includ- implementation of PEAT at cus-
ing airline policies and culture 7 tomer airlines will be coordinated
as well as regional culture. 1 1 through the customer service
As pointed out by the FAA, representatives for each carrier.
some procedural deviations In this manner, operators can
have produced desirable out- reserve specific training slots
comes for safety. Therefore, it tensive and reliable feedback on how humans throughout the year. As part of
is important to obtain a bal- I the implementation plan, airline
anced perspective on flight nteract with technoloo senior management will be pro-
crew adherence to procedures. vided with an overview of the
PEAT's structure enables op- PEAT philosophy, process, re-
erators to do just that. Its format is struc- the pertinent information may be shared quired organizational support, and a
tured so that it can be used to help internally with airline maintenance, and model for successful airline implementa-
understand what contributed to a flight externally with the manufacturer. tion. To facilitate crew cooperation, air-
crew's correct decision regarding inten- A manufacturer needs to know when lines may also want to include pilot
tional deviation. This type of information a crew interface design may have con- representatives in such briefings. This
may eventually prove valuable in training tributed to a procedural deviation. PEAT will be followed by the training of safety
and in modifying existing standard oper- will enable Boeing to improve future officers for effective application of the
ating procedures. product design by furthering our under- PEAT process, and the training of ana-
By implementing more effective data standing of such critical incidents, PEAT lysts in the use of the PEAT software
collection and consistent analysis over also can foster the data sharing of "best database and analysis capabilities.
time, PEAT can make incident error practices" among operators, should air-
trends more visible. This trend informa- lines wish to share this information.The Conclusion
tion can provide more obvious opportu- software has built-in security features Today's air transportation system is
nities for early intervention both within designed to provide strict control over very safe, and many safety professionals
the airline and potentially across the the storage of data, access to data and have played a role in making that hap-
industry. This is also one reason why the nature of data shared. While airlines pen. However, there is still a significant
Boeing has sought to enable PEAT are encouraged to share PEAT data amount of work ahead as the industry
results to be readily integrated with among themselves and with Roeing,
those from less serious incidents. PEAT such sharing is not required.
analytic outcomes can be readily entered Perhaps the industry team's com- Dr. Graeber is Chief Engineer, Human Factors, a t
Boeing Commercial Airplanes, where he is responsible
into industry safety bases existing today ments after the field evaluation provide for managing Boeing's human factors activities in air-
which are typically used to track inci- the most concise summary from the craft design, product development, safety analysis and
accident investigation. Dr. Graeber serves on several
dents that do not require a formal inves- user's viewpoint about PEAT's value to industry committees including the JAA Human Factors
tigation, and which are often reported by an operator. According to the team mem- Steering Group, the lATA Human Factors Task Force
and the Flight Safety Foundation's lcarus Committee.
crews themselves. Thus, PEAT can be bers, PEAT legitimizes the depth of the This article is an adaptation of a paper presented
by Dr. Graeber t o the 4th Global Flight Safety and
used in conjunction with other available inquiry; provides a systematic approach Human Factors Symposium in Santiago. Chile from 12
industry safety tools to compare differ- to the investigation; raises questions not to 15 April 1999.

ICAO JOURNAL
New concept in human factors training develops
controllers' skills at efficient teamwork
Eurocontrol has developed a human factors programme which provides training in teamwork for
ATC staff: The course, adaptable to suit the experience and cultures of different countries, parallels
similar training which has been provided to flight crews and other airline personnel for some time.

services in the European Civil Aviation controller may have no further perform-
EUROPEAN ORGAN~SAT~ON FOR THE Conference (ECAC) area.' Increase in ance evaluations. Since explicit training
SAFETY OF AIR NAVIGATION traffic capacity, together with improved in the principles of teamwork is rarely
safety and efficiency, remain the main provided, the development of teamwork-
related skills depends largely on the

0
VER the last 20 years, airlines drivers for the entire EATCHIP or
have been increasingly success- EATMP work programme. nature of the controller's team. Some
ful in implementing the idea of The EATMP human factors integra- European countries have individual shift
enhanced teamwork. Many airlines tion programme concentrates on the rosters which do not allow for fixed-team
around the world apply the principles of development of harmonized and inte- structures and staff work in temporary
crew resource management (CRM) to grated human factors methodology for teams based on the random allocation of
pilots and other operational airline staff, current and future air traffic manage- work positions, similar to the crew com-
and it is perhaps surprising that "con- ment. The main objective of this work is positions in airlines.
troller resource management" did not to develop and apply human factors prin- For the purpose of developing a con-
develop in conjunction with this concept. ciples and methods for maximizing cept for team resource management, it
Although a great deal of effort and human performance and making the seemed necessary to concentrate more
expertise is devoted to training individu- best use of evolving technology. on teamwork processes than on team
als in the technical skills necessary for air structure and to use a more generic def-
traffic control (ATC), little has been done Teamwork versus teams inition of teams. Thus a team in ATS can
to train these individuals to function as The increasing complexity of ATC be defined as a group of two or more per-
team members. Incidents and accidents tasks requires use of a more structured sons who interact dynamically and inter-
in which inadequate teamwork has been approach to ensure that air traffic con- dependently within assigned specific
shown to be a factor indicate that more trollers have an opportunity to develop roles, functions and responsibilities.
attention needs to be focused on this the attitudes, knowledge and skills that Team members have to adapt continu-
important area, and the adoption of the promote efficient teamwork. However, ously to each other to ensure the estab-
title "team resource management" has in the ATC context, the attempt to define lishment of a safe, orderly and expedi-
been introduced to reflect the importance a team can be rather difficult. The term tious flow of air traffic.
of teamwork to the safe and efficient con- team is used in many European coun-
duct of air traffic services (ATS) . tries to describe the control staff or Feasibility study and
This article describes the back- groups who work together in one ATC concept development
ground, experience and future plans for unit (e.g. aerodrome, approach or area In July 1994 a study group was creat-
team resource management from the control). The definitions of team range ed to investigate the possibility of a team
perspective of the European air traffic from controllers working in the same resource management (TRM) pro-
management programme (EATMP),for- sector to controllers working in different gramme in the European Civil Aviation
merly known as the European air traffic sectors but belonging to one particular Conference (ECAC). The study group
control harmonization and integration shift or watch. Controllers usually reviewed the relevant CRM/TRM publi-
programme (EATCHIP). become members of a team after licens- cations, conducted a survey of team-
EATCHIP commenced in 1990 in ing and validation and may stay in the work-related ATC incidents, and distrib-
response to commercial pressure. The same team for the rest of their careers.
objective was to find communal answers Teams are often self-organized struc- 1. Currently ECAC has 37 member States of which
28 are also Eurocontrol member States. In the next
to the development and implementation tures, with specitic rules and roles. few years, it is expected that the remaining States
of improvements to the air navigation Apart from ratings, once validated, a will also become members of Eurocontrol.

JUNE 1999
uted a questionnaire to determine the Germany, Italy, Romania, Switzerland day-to-day examples of the controller's
attitude of controllers to teamwork in and the United Kingdom. Most of these work in the form of videos, incident
ATC. A TRM training survey was also countries ran TRM prototype courses in material and discussion. The prototype
undertaken. It identified current team 1998. In the meantime, the task force set course also includes a set of evaluation
training activities inside and outside the up national working groups to promote materials which will allow a common
ECAC area. TRM and to prepare the next phases assessment of the possible changes in
The results of these studies showed within their respective States. This course content, methods and opinions,
that failures in teamwork contribute to included presentations and discussions and allows an evaluation regarding the
incidents and often have a negative with management, operational staff and changes in attitudes towards teamwork.
effect on the performance of controllers, staff associations to foster organization- ECAC States were invited to partici-
clearly indicating the need for a TRM wide awareness of TRM ideas. Some pate in the customization and testing of
programme. States had already identified controllers the TRM material. Most of the partici-
In February 1995 a TRM task force who would become TRM facilitators. pating States required external support
was established for an initial 12-month However, the most important part of this in adapting the course; some States con-
period. The principal objective of the task phase was the communal development tracted companies with experience in
force was to produce guidelines for the of a prototype TRM course. crew resource management, while oth-
development and implementation of team ers were assisted by Eurocontrol. Many
resource management. A secondary Prototype course valuable lessons were learned while
objective was to foster awareness of the From the outset the idea was to start facilitating TRM course customization
benefits of teamwork in ATS, and to pro- with the harmonization and integration and facilitator training in Austria, Portu-
duce a draft syllabus on which TRM train- procedure as early as possible and devel- gal, Ireland, Denmark and at the Euro-
ing courses could be modelled. The task op a generic prototype TRM course on control Upper Area Control Centre at
which national organiza- Maastricht, Netherlands.
tions could build their A four-step method (described below)
own tailored courses. was chosen to guarantee efficient cus-
Active management support and a carefully Harmonization in this tomization and local "ownership" of the
respect assumed that 80 TRM material, and to achieve harmo-
prepared information campaign exert a critica per cent of the course nization.
would consist of com- Human Factors is - independent of
influence on oersonneli attitude to TRM. mon material and ins- culture - often seen as a rather "fuzzy"
tructions for the facilita- subject, and it was realized that a struc-
tors, and the remaining tured approach would certainly help to
force included ECAC State representa- 20 per cent would provide sufficient scope overcome this perception. There was
tives from Austria, France, Germany, for States to be able to adapt it to their also a benefit in applying a facilitation
Switzerland, the United Kingdom, Euro- needs and include their national exam- technique: the technique would work as
pean Organisation for the Safety of Air ples, exercises and cultural influences. a role model to show that facilitation of
Navigation (Eurocontrol) Headquarters, The course was prepared by a con- complex discussions is quite possible
the Eurocontrol Institute of Air Naviga- sultant with a proven record of develop (discussions were needed to help the
tion Services, and the International Fed- ing CRM courses for airlines. A proto- participants understand the relevance of
eration of Air Traffic Controllers' Associ- type course based on the TRM the different topics). A standard method
ations (IFATCA) - a mixed team of guidelines was delivered in September would also enable a comparison of cus-
active controllers, training staff and 1997.This development was undertaken tomization in different countries to be
human factors experts. In February 1996, in close cooperation with a second TRM made. Last, but not least, time would be
the task force completed its mandate and task force and members of the national gained. Initial custornization of all eight
the TRM guidelines were published. TRM working groups to ensure a highly modules took about four days, after
After the approval of the guidelines specific ATC product. which participants needed one more
the TRM task force received a new man- The course is a three-day facilitation week to finalize the product.
date. This work was to prepare and coor- programme for operational controllers The method was applied at least one
dinate the introduction, testing and eval- and covers the areas of teamwork, team month after an initial presentation of the
uation phases of team resource roles, communication, situational aware- TRM concept and the outline of the pro-
management. Meanwhile more States ness, decision making and stress. These totype to management and operational
joined the task force, which today com- rather academic topics have been inte- controllers. In this presentation the
prises members from Austria, France, grated in a highly practical way, using importance of the formation of a national

ICAO JOURNAL
This a%cls was w r i m By a?.Wlanfrad Barbarina,
Michiel Woldring and Dr. Anne Isaac, sll of the Ajr
TeaMe Man@ement Hmwn Resource8 Unit ar the
European Orp&&x foq t h ~RafQ of /air
Navlptbn ~EurqmroU,Bruss&&.
This a@.tMeis an adaptatioh Uf pr$wntad
by Eut~mtmlt.o the 4% GiabaY Wight LWy a8d
Human b&m %$nqmdvrnat san$iilg%Chile fm 11
.tp t 5 A@J. 19%.

JUNE 1999
Airport operators and regulators need to more
completely address human factors issues
Certain aspects of aviation security have been neglected in favour of expensive technological solutions.
This imbalance can be corrected by evaluating security operations from a holistic perspective.

supervision of people is the critical of selection, the objective is to recruit


element in any security system. Part of those people who are most likely to per-
this inadequacy is related to f i e uneven form well on the job. There are several
funding levels for technology versus factors to consider including an individ-
ual's capacity for dealing with stress, a

W
ITHIN the recent past it would human factors.
seem that, in North America at Human factors issues in the field of high workload, and interaction with dif-
least, the emphasis on resour- aviation security can be more completely ferent kinds of people while using state-
ce allocation in the field of aviation secu- addressed by applying a framework that of-the-arttechnology properly. Given the
rity has been placed on the development identifies principal dimensions along high turnover rate characteristic of most
and deployment of new technologies to which to expend available resources. security operations, the selection process
address all security requirements. This This article describes a human factors receives short shrift because of the pres-
approach has been adopted despite a his- framework that involving four principal sure to meet operational requirements
torical perspective that clearly demon- axes that must be considered. on a continuous basis.
strates that ignoring human factors Among the four dimensions are the Training of personnel usually includes
issues will lead to gaping holes in the human operator, technology, organiza- two components: classroom instruction,
technological solutions, no matter how tional and operational environments, and an activity that is increasingly displaced
infallible these systems may appear. certification of personnel and technolo- by computer-based training (CBT)
A compelling example of this trend gy. Attempting to achieve a more com- which may not be equivalent to previous
was reported recently in the Air Safety plete perspective on the human factors training; and on-the-job training. On-the-
Week newsletter of 21 December 1998by issues related to aviation security job training may be quite variable and its
a former head of U.S. Federal Aviation requires activity on at least these four relation to the initial training may not
Administration (FAA) security during the dimensions. always be optimal or obvious.
mid-1980s. Since the loss of Pan Am 103 The first dimension of a human factors Yet another component is the assess-
over Lockerbie, Scotland in December fkamework for aviation security is related ment necessary to determine whether
1988,Air Safety Week reported, the FAA to the human operator mandated to use a the training provided is appropriate and
has spent more than US. $307 million on particular technology deployed for secu- of sufficient depth and breadth to maxi-
aviation security, primarily on new tech- rity purposes. The focus here is on per- mize the probability that each individual
nologies related to explosive detection sonnel selection and training. In the case operator will perform to the desired
systems. Over the same period, how- level. A related element requires the
ever, only a minuscule amount, evaluation of personnel in achieving
approximately US. $5 million, has some predetermined and objective
been spent on addressing human fac- level of performance proficiency that
tors issues in aviation security. needs to be validated in the field.
Consequently,precious research and This required level of performance
development resources have been needs to be achievable through the
concentrated within a narrow and training provided.
limited scope. It is also necessary to objectively
The former head of FAA security, evaluate the cognitive strategies used
B.H. Vincent, asserted that the by security personnel. Although all
commitment to human resources operators may reach the minimal
performance standard required,
and the upgrading of The selection, training and supervisron or securi-
systems was at an inadequate level tv * .Dersonne/ is considered a critical element in
some individuals may demonstrate
and that the selection, training and any security system. significantly higher levels of profi-

18 ICAO JOURNAL
ciency possibly as a result of a different experience with computers; sibly leading to reduced turnover, allow-
strategy. It may be possible to feed these the sampling method required of ing airport authorities to maintain high-
successful strategies into the selection operators (this is especially critical in the ly skilled, trained and motivated staff for
and training processes. development of explosive detection sys- longer periods. This, in turn, results in a
The second dimension of the human tems); and high level of security.
factors framework is related to the tech- standardizingthe ergonomic elements Related elements include team issues.
nological developments required. This of the technology. Some security personnel, for example,
dimension, too, is concerned with the ele- One example of such standardization cannot work with other team members for
ments of training, but in this instance the would be to specify that all X-ray scan- various, possibly cultural reasons, thereby
focus is on the technology required to ners have: a threat image projection sys- undermining system performance to
provide computer-based training and how tem to allow regulators and security possibly dangerous levels. It may be ben-
this can be adequately delivered without providers to objectively evaluate detec- eficial to apply team resource manage-
actually practising on the security equip- tion performance. Such a system allows ment training (an article on team resource
ment to be used on the job. Also needed the virtual inclusion of a threat image management appears on page 15).
is development of a computer-basedtest into that of the scan- An element also
to assess different CBT packages, ned baggage image. found to be critical in
Another important component is relat- This example is cited other operational con-
ed to screener assist technology. This here not because of texts is increased
technology is available primarily a perceived need to awareness of the im-
because it can be readily programmed standardize this par- portance of shift man-
into software. However, only a few ticular technology, agement, Staff should
options may actually be used by opera- but because it repre- be assigned s h i s in
tors and it is not known whether auto- sents a good exam- uch a way that the
mated detection technology actually ple of where a global a p a c t of operator
improves system performance levels. design standard would fatigue is minimized.
Any potentid improvements will be criti- allow monitoring of Other elements reflect
cally dependent on the level of automat- performance levels which standards are
A framework for addressing human
ed "hits," false alarms, nuisance alarms over time and across factors issues related to a security
in place and how they
and the normalized difference between regions. Such exten- operation is comprised of four axes. are enforced, and how
hits and false alarm rate. This is because sive monitoringwould these standards mesh
the high number of false or nuisance lead to intervention when unacceptable into the policies, procedures and pro-
alarms can lead to a lowered confidence variances occur, Standards allow com- cesses in place within a particular orga-
in the technology. ConverseIy, a high parison, and the perfect place to apply nizational environment.
number of hits can lead to a false level of standards may be within the technologi- It is also important to pay close atten-
confidence and exclusive reliance on the cal elements. tion to the elements related to the opera-
technology. Only high hit rates com- The third dimension to the human fac- tional environment in order to avoid
bined with extremely low false alarm tars framework is one that does not have undoing what may have been tremen-
rates may actually provide the necessary as high a profile as the two described dous improvements elsewhere in the
conditions for improvement in system above. It delineates elements that com- system. Not enough has been done with
performance that still requires objective prise the organizational and operational respect to this aspect of human factors to
demonstration. environments. Elements include the be able to answer all the questions that
Also of importance is the ergonomic physical aspects of the working environ- have arisen except to acknowledge that
design of technology. Ergonomic consid- ment (e.g. noise, light, dust, tempera- more action is required in this area.
erations include but are not limited to: ture). Although personnel may be The fourth dimension of humanfactors for
the displays used in presenting infor- accurately selected, properly trained and aviation security comprises elements with
mation; equipped with the latest technology, opti- respect to critically important certification
the visual properties of the image; mal performance may not be possible in
- - co%tinued S -.--
T an $age 27
p u

the controls and menus required to certain physical environments. Dr. Vinceht i s a Senior Ergonomist at the Trans-
access the necessary image or informa- Training and field performance stan- portation Development Centre, Montreal, where he is
tion, including their layout; dards also need to be evaluated, and the currently responsible for multimodal human factors
research and development for Transport Canada.
how the interface is to be developed information utilized to provide job incen- This article IS an adaptation of a presentation given
by Dr. Vincent t o the ath Global Fl~ghtSafety and
to take into account that some operators tives. Increased motivation can be Human Factors Symposium in Sahtiago. Chile from 12
using the equipment will have had no achieved through improved wages, pos- to 15 April 1999.

JUNE 1999
Shift in safety paradigm is key to future
success in reducing air accidents
Safety approaches have evolved over the years, fiom a focus on individual performance to an emphasis
on systemic flaws. Yet,fiGrther evolution in safety strategy will be needed before the aviation community
can address the challenges of the coming years.

nology cockpit design. However, a 1996 oriented performance. This means, in


JEAN P A R I ~ S human factors report by the U.S. Fed- effect, that systems govern their own
DEDALE
S.A. eral Aviation Administration (FAA) and safety. The famous "Swiss cheese" model
(FRANCE) the European Joint Aviation Authorities postulated by James Reason indicates
UAA) was an indication of what to that the organizational environment in

T
EN years ago, a review of the expect from the evolution of the airwor- which personnel work is determined by
international standards and rec- thiness certification process. management decisions and processes
ommended practices as formu- During the past decade, there has that also determine the extent to which
lated in the ICAO annexes would not been a vast flow of ideas and exchanges the work context fosters or deters errors
have yielded much focus on human fac- of experience and perspectives between and violations (Figure 1).
tors. ICAO Annex 1, which deals with academics, regulators and safety practi- As for automation, several concerns
personnel licensing, was the only annex tioners. A consensus has emerged that have emerged. One is that automation is so
that included human factors training safety depends on teams more than indi- efficient and reliable it can lead to compla-
requirements. Annex 13, which address- viduals, and that team synergy can be cency, challengingflight crews' situational
es investigation and reporting on inci- learned. It is also acknowledged that awareness and eroding basic operating
dents and accidents, had an appendix operators are not autonomous (i.e. they skills which are required when automated
that described the standard accident work within the context of a larger sys- systems fail. However, the results of an
report format, but the only human fac- tem), and that systems govern their own Airbus Industrie study indicate that the
tors element related to individual train- safety. Operators are seen as risk man- introduction of new technologies has been
ing and medical aspects. agers, culture is viewed as a possible associated with safety improvements since
A similar review today would show issue, and errors themselves do not the 1950s, and that this relationship has
that crew resource management (CRM) cause accidents (instead it is the negative continued with the latest generation of
has been introduced into Annex 6, the consequences of errors that can lead to highly automated aircraft (see Figure 2).
accident report format has been amend- accidents or incidents). Lastly, automa- The notion that errors do not cause
ed to include references to organization- tion is neither a "silver bullet" nor a devil. accidents may be more provocative. The
al aspects of accidents, the revised Individual behaviour is determined by tendency is to believe that errors cause
ADREP 2000 version of the ICAO acci- many factors including the crew selec- accidents because most accident scenar-
denthncident database includes provi- tion process, training philosophy, cockpit ios include human error. But the psy-
sions for coding CRM issues and root procedures, cockpit design and working chological process of error, which is
causes. In addition, 14 digests address- conditions. Together with corporate, pro- essentially a deviation from the opera-
ing human factors applications to various fessional and even ethnographic cul- tor's intention, often is confused with the
domains have been produced, and the tures, these factors determine individual unexpected and, in some cases, unsafe
proceedings of various regional semi- behaviour and therefore govern safety- effect of erroneous actions within a sys-
nars and global symposia have been tem. The system's resistance to or
published. tolerance of errors is a more critical
As far as aircraft design is con- safety parameter than the number
cerned, neither Annex 8 (Airworthiness and the nature of errors.
of Aircraft) nor the two worldwide A second pivotal factor is the abili-
regulation references, the European ty of the error producers, the front-
JAR 25 and the U.S. FAR 25, have line personnel, to detect and evaluate
been changed. They remain inade- Figure 1. "Systems" govern their own safety. the potential consequences of their
quate for human factors in high-tech- (based on I. Reason's model) errors in order to correct or mitigate

20 ICAO JOURNAL
begin by definfndgits safety go&, 'Ih
god m y be ta hwe hwer accidents
regardlea of grawth or it muXd be fie
ellminationof amidtats mtirels if su& a
gaal irz wen podble. The hdustry must
alm 4e:brmin~how to keep pace w3th
the growth rjf &%ternmmpl&tyi, &nd
mt End n myto make an already safe

I Normative safety
'Total Quality Management Safety
Safety results from comprehe
and procedures
Good professionals adhere to
Good procedures and good pr
make a safe world

Ecological Safety
'Safety Through Adaptahon'
Av~at~on operat~onscannot be speclfled In thelr
ent~rety(In part because they Include humans)
Devlat~onsfrom nommal operation are both a
potentla1 threat to safety and a necess~tyfor
adaptation
Human operators are (up to now) the only
~telhgentcomponent of the system

JUNE 1999
with these challenges if it persists in its and to global interactions (the interna- acddent per two or three million flights,
current approach. This is because all tional aviation system). as currently displayed by the best oper-
safety strategies have an apogee, an Large and complex systems do not ators, the potential for improvement is so
asymptotic levd of safety they are unable obey linear causality. Rather, they are the limited that the normative approach will
to surpass. For example, the imposition scene of circular causality, with strong not achieve the desired outcome.
of rules and punishment can improve feedback and feedforward effects which Negative collateral effects, especially
safety when applied to an anarchical envi- can augment or inhibit the consequences those caused by human risk manage-
ronment, but that strategy peaks when of an action. Intrinsically stable, they ment, would no longer be of second
concealed errors and violations prevent a resist forces with a huge inertia or plas- order compared to the benefits of the
ticity and effect change by normative strategy. This means that the
leverage. However, they also normative and ecological approaches
can involve interactions that must be considered complementary.
are not obviously evident and Intelligent rather than blind compli-
which may trigger divergent ance to safety regulations must be reha-
processes and ripple effects. bilitated as a key safety issue. The
The functioning and the fail- boundary between what we "need to
ure modes of larger, more know" and what is "nice-to-know'knd
interconnected systems chal- the philosophical difference between
Fi$ure 2, Accident trends for different generations of lenge traditional h e a r models a&erence and interpretation must be re-
aircraft 1959-96. (Airbus industrie data) and necessitate the use of thought. Training will have to give oper-
more sophisticated systems ators enough knowledge far them to
better understanding of failure. Doing thimking. understand what they do and why they
more of what is already being done no Secondly, there is the growing com- do it at a relevant operational level.
longer yields better results, because neg- plexity of the system model. This initially Interface and automation design will
ative collateral effects reach the same was the SHEL model in which a system have to respect naturalistic human deci-
order of magnitude, was a set of interactions between main sion-making and risk perception mecha-
A shift in the safety paradigm, the fun- components. Then there was the afore- nisms, showing the risk and the margins
damental rules and principles that are mentioned Swiss cheese model which rather than hiding them.
believed to be the definition of and the linked organizationalstructure and hier- More generally, the front-line opera-
conditions for safety, is the key to the archy with real-time safety. These were tional realities of risk management also
future. A safety paradigm is seen in the followed by a tight coupling of systems must be accounted for and properly
principles underlying accident and inci- to their environment and the effects of understood. Current preventive strate-
dent investigation, in design philosophy opaque coupling between apparently gies cannot be proactive since they are
and concepts, in training philosophies, independent components of the system. based on feedback which is appropriated
and in the role of procedures, punish- That evolution represents a shift in the by safety specialists, investigators, ana-
ment of violators and the like. metaphor of systemic models from lysts and the like - all using engineer-
Until the early 1980s, individual pilot machines or technological solutions to ing design logic. Feedback should be
errors or violations were a key issue. natural or ecological solutions. The ecol- understood as a component of, and a
The CRM era followed, stressing crew ogy metaphor suggests that since sys- contributor to, organizational learning.
synergy. The 1990s have been the sys- tems are not totally stable, they are Rather than an external loop independ-
temic era, acknowledging the organiza- adaptable at the local, real-time level of ent from operations, it should be inte-
tional dimensions of safety. A careful individuals as well as at a global, collec- grated into daily production schemes.
review of this evohtion highlights two tive long-term level. The machine Scheduling should allow for debriefing
main directions in the way thinking has metaphor implies a normative approach - - continued
- an page 28
changed. to safety, ~ H k b ag reduction of variety Jean Paribs IS a founding member and Managing
D~rectorof D&daleSA, a consultancy that focuses on
One is that the segment of the system and a process which is inher- the human factors dimensions of safety in the aviation,
that has to be under surveillance for ently reactive. The ecology metaphor nuclear and maritme fields. Mr. Parks is a former air
acddent investigator, and has been a member of the
safety reasons has continuously expand- imp1ies a and learning ICAO Flight Safety and Human Factors Study Group
ed. mom individual interactions, it has t0 safety. since its ~nceptionin 1988, He holds a commercial pilot
l~censewith instrument, multi-engine, turboprop and
evolved through local interactions (one The normative has been the instructor ratings, and a helicopter private pilot license.
team, including cockpit crew, then cabin keystone of safety improvements for at This article is an adaptation of a paper presented
by Mr. P a r k to the 4th Global Flight Safety and
crew, ground staff and air traffic control) least the past O' years but it is nearing Human Factors Symposium held in Santrago, Chile
to organizational interactions (an airline) its apogee. Beyond a safety level of one from 12 to 15 ~pril1999.

ICAO JOURNAL
Human factors programme introduced in parallel
with CNSIATM systems implementation
Chile has moved decisively to establish a plan for applying human factors knowledge throughout its
aviation system. A key impetus has been a national project to introduce satellite-based technologies.

by the analysis took the form of a resist- cepts. For this purpose, a structured
ance to change from management who course has been acquired with simple
were concerned about the scale of equip- training aids which can be adapted to all
ment investment required by the new types of audiences.
technology and the necessity to divert No concept can be implemented effec-

u
resources for human factors training. tively without solid support from senior
NTIL fairly recently, human fac- Another challenge identified by the management, and for this reason the
tors was a discipline left to a few human factors analysis was the capacity of DGCA has conducted an awareness pro-
initiates who, although they had the world aviation community to produce gramme at the executive level of the
some knowledge of it, could not really be seamless CNS/ATM systems given the organization.
called experts. This was particularly evi- socio-economic, cultural and development The Escuela Tecnica Aeronautica (ETA),
dent as the concept of crew resource differencesthat exist among countries. the DGCA's institute for training opera-
management (CRM) evolved. Chile's analysis indicated that the tional professionals and technicians, has
With CRM, air carriers began to move greatest difficulty in implementing a inaugurated a 42-hour human factors
ahead with their own training progam- human factors programme arose from course in the final semester of its pro-
mes and, soon after,the subject came under the yap between theoretical knowledge gramme. This is supplemented by CRM
greater scrutiny as ICAO developed hu- and the practical application of that training using an air traffic control
man factors standards and recommend- knowledge. Clearly, it is hard to account (ATC) simulator.
ed practices that States could incorporate for human factors when introducing new A human factors course for non-oper-
into their regulaFory regimes. ICAO also technology, and Chile's Directorate ational personnel will begin in 2000 and
began to disseminate valuable informa- General of Civil Aviation (DGCA) has is scheduled to continue for five years.
tion in the form of a series of digests on designed an aggressive plan to ensure This course is designed to reach person-
different aspects of human factors. integration. nel who are neither senior management
The principal challenge for many The objective of the plan is to incorpo- or ETA graduates. A current trial pro-
administrations has been how to define rate human factors concepts as a manage- gramme provides practice and gives
the perceived need without a sound ment and flight safety tool throughout instructors a better command of the sub-
knowledge base. In Chile, this challenge the aviation system by 2010. Several ject. Also, a two-year plan has been
has been met in part by developing a DGCA personnel have been assigned for developed with the objective of inform-
matrix analysis that focused on threats training in the operation of new tech- ing all DGCA clerical officers about
to safety, opportunities, weaknesses and nologies. To be provided by the U.S. human factors.
strengths. This analysis has been used Federal Aviation Administration (FAA), A sub-programme is targeted at
to develop a human factors programme the training focuses on flight inspection, instructors certified by the DGCA to
in parallel with the country's implemen- design of instrument approaches based work in the private sector with airlines,
tation of communications, navigation, on the global navigation satellite system flying clubs and schools which provide
surveillance and air traffic management (GNSS), and wide area augmentation training for cabin crew, mechanics, dis-
(CNS/ATM) systems using satellite tech- system (WAAS) engineering and opera- patchers and other aviation personnel.
nology. The human factors programme tions. In addition, a CNS/ATM systems continued on page 28
has been kept as simple as possible - the manual for use by air traffic controllers Mr. Makr~novIS Ch~efo f the DGCA Flight Safety
goal being a paradigm shift which would and flight crews has been developed. Department and a Colonel (Ret.) In the Ch~leanAir
Force, where he has served i n a number o f safety-relat-
encourage acceptance of the human fac- The goal is to set up a cadre of highly ed posts.
tors discipline at all regulatory and oper- motivated professionals who will act Th~sart~cleIS an adaptation o f a paper presented
by Mr. Makrlnov at the 4th Global F l ~ g h Safety
t and
ational levels of Chile's aviation system. more as facilitators than as instructors in Human Factors Syrnpos~umat Sant~ago,Ch~lefrom 12
The main obstacle to safety identified disseminating awareness of the new con- t o 15 Aprd 1999

JUNE 1999
Human error a warning flag for regularors and managers,
Council President tells safety conference
Despite an impressive safety record, with a relatively low fatal the resources, investigate failures of the system and take
accident rate that has remained practically unchanged for remedial action, Dr. Kotaite explained.
over two decades, a further reduction in the rate of fatal air- "Human error should become a warning flag for regulators
craft accidents is possible as the industry moves into the and managers, a possible symptom that individual workers
21st century, ICAO Council President Dr. Assad Kotaite have been unable to achieve the system goals because of
informed participants at a safety conference in London in difficult working environments, flaws in policies and proce-
early June. dures, inadequate allocation of resources, or other deficien-
Innovative technology as well as human factors endeav- cies in the architecture of the system," Dr. Kotaite informed
ours have already contributed significantly to the aviation the participants - representatives of numerous govern-
safety record, but the most important safety strategy today ments, manufacturers and airlines.
centres on the relationship between management and safety, The importance of the role of management is reflected in
Dr. Kotaite said in opening the conference, organized by the current safety efforts, with three of the six key activities which
Royal Aeronautical Society, on 3 June. comprise the ICAO global aviation safety plan (GASP) orient-
The Council President stressed the role of management in ed on management.
enhancing safety. "Simply put," he stated, "managers play a The universal safety oversight audit programme, the flight
fundamental role in defining and sustaining the safety culture safety and human factors programme, and the controlled
of their organizations." flight into terrain (CFIT) programme, the Council President
Regulators and airline management define the environ- indicated, all emphasize organizational and managerial solu-
ment within which operational personnel conduct their tasks: tions to safety issues (see also "ICAO providing tools and
it is they who determine the policies and procedures, allocate leadership needed for enhancing safety worldwide", page 4).
The CFlT programme, Dr. Kotaite observed by way of
example, "moves away from the simplistic notion that CFlT
accidents are simply caused by substandard human perfor-
mance . . . CFlT accidents are truly systemic accidents." For
that reason, a substantial part of the ICAO message con-
cerning CFlT is directed at decision makers in the aviation
system.
The Council President concluded his address by observing
that current and future safety challenges will require innova-
tive approaches, not versions of old solutions that were suc-
cessful in the past. He noted that a rapidly changing world
demands adaptation.
"Human error is nothing new," Dr. Kotaite pointed out, "but
the massive introduction of advanced technology in contem-
porary aviation and its resulting interface with people brings
new challenges . . . requiring new solutions." 0

Y2K initiatives have already achieved


substantial progress, UN meeting told
ICAO presented a status report on Y2K readiness at the Unit-
ed Nations on 23 June 1999 which reaffirmed that safety
remains the top priority of the world aviation community.
The ICAO report, which also highlighted the Y2K initiatives
of the International Air Transport Association (IATA) and the
Airports Council International (ACI), pointed to substantial
progress in all critical areas of air transport and in the devel-
opment of contingency plans. Contracting States of ICAO are
responsible for the safety of their airspace, airline and airport
operations, and the provision of air navigation services.
Aircraft manufacturers and airlines have already indicated
that aircraft will continue to operate normally through the year
2000 date change and during other date changes which are

ICAO JOURNAL
considered critical to computer systems. While airports have The 265 member airlines of IATA have collectively spent an
developed Y2K contingency plans, travellers may experience estimated U.S. $2.3 billion to resolve Y2K problems. IATA has
some inconveniences during the Y2K date change as air- pursued an active campaign to increase industry awareness
ports in most cases are dependent on community services and monitor the progress of Y2K programmes at 2,000 air-
and utilities to ensure smooth operations. ports and over 180 air traffic service providers. IATA field
Another area of concerted activity is the provision of air work involves close cooperation with ICAO on regional con-
navigation services, which use a significant number of essen- tingency planning efforts, and with ACI in promoting local
tial computer-based systems. coordination between airports and airlines.
The ICAO Y2K action plan focuses on disseminating infor- ACI has provided its 1,350 member airports with specific
mation about the Y2K problem, raising the level of awareness assessment tools, such as a standard action checklist and a
within the international civil aviation industry, assessing the list of systems which may require scrutiny, as well as a refer-
progress of States in addressing the Y2K problem, and sup- ence definition of Y2K compliance.
porting States' efforts as well as those of air transport organi- During the remainder of 1999 and the early months of 2000,
zations. The plan encourages the development of national con- ICAO, IATA and ACI will continue to coordinate their respective
tingency plans while working through planning groups toward Y2K programmes in information gathering and dissemination,
the development of contingency plans at the regional level. site visits, problem solving and contingency planning, the UN
A report on the state of Y2K readiness will be available to meeting of national Y2K coordinators was informed. 0
ICAO Contracting States in August, when contingency plans
are expected to be finalized. The status report will be based
on information ICAO had received from States by 1 July. In
the meantime, information on Y2K compliance is being dis-
seminated to States as soon as it becomes available so that
national and regional contingency plans and other Y2K activ-
ities can be updated. ICAO will operate a global coordination
unit during the Y2K transition period, which will assist with
implementation of regional plans.
Year 2000 programmes at IATA and ACI, implemented in
cooperation with ICAO, have ensured the widest possible
scope of activity by the international aviation industry.

JUNE 1999
ICAO Council appointment A second source of information comes from the use of a ques-
Victor P. Kuranov has been appointed tionnaire concerned with attitudes and behaviours, which can be
Representative of the Russian Federa- administered before and after the TRM course. Often a more
tion on the Council of ICAO. His robust method of evaluating the changes in these attitudes and
appointment took effect on 14 May behaviours is to administer a third questionnaire, identical to the
1999. second, some four to six months after the TRM course.
A Doctor of Technical Sciences spe- A third and more rigorous evaluation comes from the corre-
cializing in navigation and air traffic lation of these attitudinal changes with observation or interviews
V k t o r E ! KuranOv management, Mr. Kuranov has, over the
(Russian Federation) of the same personnel to gauge meaningful behaviourial
course of his career, received training at
changes. From this methodology, measurable positive changes
a number of technical institutes in his
in interaction should be present following the course. Lastly, the
country, including graduate training at the State Civil Aviation
Scientific Research lnstitute in Moscow. His professional ultimate validation should be found in a correlation between the
background includes several years of service with the "Aeron- TRM programme and a decrease in the frequency of incidents
avigatsiya" State Scientific Research lnstitute in Moscow, within the system. The latter two methodologies are highly com-
most recently as its Executive Director. plex and take considerable time to apply. It is for this reason that
Mr. Kuranov has participated in the development of vari- the evaluation phase of the Eurocontrol TRM programme used
ous technical regulatory documents governing the former only the course evaluation and the monitoring of attitudinal and
Union of Soviet Socialist Republics and the Russian Federa- behaviourial changes as an assessment of its effectiveness.
tion, as well as in negotiations with other States related to the Safety questionnaire. The development of the air traffic con-
international civil aviation activities of his country. He has trol safety questionnaire (ATCSQ) was based on the work under-
also participated regularly in sessions of the ICAO Assembly
taken previously in developing flight crew resource manage-
and in the work of various ICAO technical bodies, including
ment and, in the medical field, operation room management. The
the ICAO Special Committee on Future Air Navigation Sys-
tems (FANS Phases I and II), the CNSIATM Implementation
TRM programme clearly defines seven areas of concern in its
Advisory Group (ALLPIRG) and the Global Navigation Satel- prototype course. Within these areas, items were identified and
lite Systems Panel (GNSSP). 0 became the basis of the ATCSQ.
The questionnaire consists of four main sections. The first
section concerns attitudes towards the quality of training, work-
Team resource management ing conditions and documentation, and the last section concerns
continued fiom page 17 demographic information.The second and third sections contain
is culturally dependent. Some ATC organizations have designed the main evaluative information and are concerned with atti-
specific courses for different ATC functions. tudes and the responses associated with behaviours.
Evaluation process. The objective of team resource man- The results of the evaluation of this questionnaire show that
agement is the use of all available resources (people, informa- the safety questionnaire is a robust, reliable instrument and is
tion and equipment), in order to achieve the safe and efficient informative for the purpose for which it was designed. A few
movement of air traffic. This objective is obviously ambitious questions indicated some bias and will therefore be reworded for
and, as in the case of its flight deck counterpart - crew future use. In terms of the results concerning the attitudes
resource management - there will be difficulties in the mea- towards the professional training and working environment, the
surement of its effectiveness. However, to ignore the challenge questionnaire clearly indicated acceptable satisfaction with most
of this evaluation would be foolish. of the areas of the TRM course, but some aspects concerned
Programme evaluation should not only provide information with operations and safety manuals, handling of emergency traf-
on the effects of the TRM course but should also provide direc- fic and feedback in daily operations, were not as positive.
tion for further initiatives. The most basic type of information The results were based on a small response sample (30 sub-
comes from participants' evaluations, usually collected by ques- jects) and could give a rather biased picture of the impact of the
tionnaire at the end of the TRM course. Positive reactions to the TRM course. The results did, nevertheless, illustrate a signifi-
course provide necessary but not sufficient evidence of impact. cant change within some of the subject domains. Most of these
While a positive reaction to the course is not sufficient in itself changes were in the anticipated direction; that is, the course
to indicate a positive s h i i in attitudes, a negative reaction to the had changed the attitudes of participants in favour of better and
course is an almost certain indication that positive behaviour more cooperative teamwork and more sympathetic team roles.
change is not going to occur. Overall the results showed that the questionnaire is sensitive
to changes in attitude and, with a larger sample and strict adher-
ence to data gathering, a more meaningful database could be
established in the future. This will not only strengthen the use of
such a questionnaire but will help individual countries to cus-
tomize their individual needs for TRM applications.
The outlook. After the assessment and evaluation of the testing
phase, the task force will issue recommendationsfor further irnple-
mentation of the TRM concept. This will include advanced imple-

ICAO JOURNAL
mentation plans for team resource management in all 37 ECAC
States and its incorporation into the selection, training (including
upgrade and refresher training) and licensing of ATC staff.
The initial implementation of the TRM concept is likely to go
through an evolutionary process, just as CRM did when it was
introduced. The target population will be extended to other
operational staff in air traffic services and air traffic manage-
ment. Other areas, such as incident and accident investigation,
also need to consider teamwork-related aspects in their evalua-
tions and reports.
The success of the development and introduction of automa-
tion in ATM will partly depend on enhancing and supporting the
teamwork culture, both in the control centre and on the flight
deck. The day when controllers and pilots will be trained in com-
bined CRM/TRM courses may not be far off. 0

continued from page 14


attempts to discover ways to make this system even safer. The
industry is accepting that challenge, both worldwide and
regionally.
Addressing the most serious safety problems - CFIT,
approach-and-landing and loss-of-controlaccidents - can pro- Human factors related to these elements need to be consid-
vide high leverage for accident prevention, and there are many ered along the certification axis because all of the axes need to
interventions available for use today, including those focusing provide for a seamless stream that contributes to maximizing
on human factors. security levels while keeping the human operator in the fore-
Although safety practitioners represent a very small portion front of any actions.
of the worldwide flight operations community, each individual The accompanying figure (see page 19) is a representation
can play a large role within his or her airline and geographic of the four axes that form a framework for addressing human
region. It's up to each to do his part and to carefully consider factors issues in the field of aviation security. Two of these axes,
and implement the appropriate safety interventions if the indus-
concerned with the human operator and with technology, have
try is to positively contribute to the worldwide safety story.
a user focus. It is here that almost all of the resources have
been expended. The other two axes, concentrating on the orga-
nizational and operational environments and on certification,
Security operations
have a mandate focus. These dimensions seem to have been
continuedfrompage 19
issues. It is necessary to take into account the process by which neglected with respect to human factors.
individuals and organizations are certified. What standards are The human operator and the organizational and operational
required, and are they in place? A third element concerns re- environment are both realitybased dimensions, while technology
certification.What period is optimal, and when is it necessary to and certification are objective-based.The first two dimensions
d e c e r an~ individual or organization? concern personnel selection, training, turnover, motivation and

I * Airline Pilot Qualification Program Certification after 10 months CPL/IR


More than 60% of Crossair pilots have trained at HORIZON = IS0 9001 certified Please contact: . I
-
-- -
-
-

HOTSwissO
FlightN
Academy Ltd
~ckerstrasse4, CH-8180 Bulach/Zurich
phone i l l - l B 6 2 0'7 07, fax 1 - 1 6 2 02 1
http://www.horizon-sfa.ch, info@horizon-sfa.ch E
Z
$ I
Where flying c a r e e r s t a k e off
JUNE 1999
wages, while the latter two are concerned with the objectives
that technology needs to meet and the certification of such
Safety strategy
continued from page 22
technology and operators. It would be problematic to ignore as well as individual and collective communication, and debrief-
any of these human factors aspects. ing tools and practices should be developed.
Summary and conclusion. Human factors has been rec- It has been said that safety is not a static victory, but a
ognized for several decades as a primary factor in aircraft cock- dynamic control of a permanent hazard. From this perspective,
pit design. Recently, awareness of the importance of human normative safety reaches its apogee when large, complex and
factors in all areas where humans interact with technology has tightly coupled systems are involved. Procedures and regula-
grown. This is certainly the case in aviation security, where the tions are needed, but they are not the ultimate solution as they
deployment of increasingly sophisticated technology heightens will not allow the aviation community to meet the challenges
the demands on the human operator. of the coming years. As long as humans are a part of the sys-
Demands on security personnel may have a negative impact tem, they are both a source of potential hazard and a form of
on the performance of the system (i.e. human operator plus protection.
technology). The objective of the system is for the human Human knowledge and adaptability are critical. And because
observer to interact with the technology and render a correct perception and intelligence are involved, causation processes
decision in, for example, accepting or rejecting luggage for cannot be a linear chain of independent links.
loading on board aircraft. Progress towards an even safer system implies organiza-
A framework that allows systematic evaluation of human fac- tional learning. Also, ironically, it means that errors, deviations,
tors research and interventions could contribute to minimizing incidents and accidents must be considered as a fundamental
system error and optimizing system performance. The frame- source of knowledge in the drive for increased safety. This
work is designed to allow airport operators and regulators to could well be the challenge of the next 10 years. 17
focus their efforts on human factors issues that may negatively
affect the performance of the aviation security system.
It is to our detriment if we focus our resources along only Human factors programme
one or two axes of activity. The application of a human factors continued from Page 23
perspective has revealed at least four dimensions where we The DGCA plans to require that candidates for license
need to apply resources in order to improve and maximize the renewal demonstrate a knowledge of human factors. This
level of security. It may be that the relatively neglected axes requirement will take effect in 2000 and continue until 2008, by
may provide for greater efficiencies in performance. Although which time all licenses then in force will have been subject to
the framework described here was developed with aviation renewal at least twice (the longest term of validity of a certifi-
security in mind, the various elements may be of potential use in cate held by technical aeronautical personnel is four years).
evaluating other areas that require human factors initiatives. The objective is to promote human factors awareness through-
out the DGCA and the industry it serves and supervises.
The human factors programme introduced in Chile is close-
ly tied to the plan to introduce the CNS/ATM systems. Chile's
CNS/ATM project has been progressing steadily since
December 1996, when the DGCA and the U.S. Trade and
Development Agency agreed to develop a feasibility study and
define a CNS/ATM systems architecture for Chile. Some of
these elements are now being incorporated into a national
implementation plan, which is in its final revision stage.
With expertise provided by the U.S. FAA, Chile has been
developing an air navigation system based on the global posi-
tioning system (GPS), a component of GNSS. In March 1998,
the two agencies agreed to facilitate training of DGCA personnel
in subjects related to satellite-based navigation. The FAA also
supplied three WAAS reference stations to the DGCA (they are
installed at Santiago, Balmaceda and Antofagasta). In
December 1998, it provided a Boeing 727 for landing demon-
strations at Santiago International Airport. The B-727 complet-
ed a series of Category I approaches using signals generated
by the WAAS stations and corrected by the FAA Technical
Center in Atlantic City. While the acceptable standard of accu-
racy is 7.6 metres, the Santiago trial revealed an accuracy as
precise as three metres.
To better evaluate WAAS technology, the DGCA has pur-

ICAO JOURNAL
chased two reference stations to supplement the three on loan.
The five stations are currently being used to transmit signals
through the Inmarsat I11 geostationary satellite, reducing GPS-
induced error to 7.6 metres (from 300 m).
1 POSTS VACANT
The DGCA has also acquired software which will allow Accounting Officer, Accounting Services Section, Finance
users to predict satellite availability along their routes, pro- Branch, Office of the Secretary General, Headquarters
viding better flight planning. To be installed at 23 facilities - Montreal. Ref. PC 99/21/P-4.
from Arica, near the northern border with Peru, all the way An advanced level post at Headquarters, Montreal. Essential
to Antarctica - the software enables satellite data to be pre- qualifications and experience: University degree or equivalent
sented to flight crews in a graphic format that can be under- academic qualifications in financial management, preferably
stood readily. with professional certification as Chartered Accountant or
Importantly, all these advances in introducing new technol- Certified Public Accountant; at least ten years' financial or
ogy in Chile's aviation system have been made in step with the accounting experience some of which with a national govern-
development of an effective human factors programme. While ment or international organization; experience in the utilization
of modern financial software packages; ability to develop and
the Chilean model may not be ideal for all countries, it strives to
maintain computerized financial information systems; experi-
adapt human factors to Latin American idiosyncracies and may
ence in the implementation of financial management/account-
be of interest to other administrations which need to incorpo- ing software in a client/server environment; experience in
rate similar training in their aviation systems. 17 preparing financial reports and statements; ability to analyse
and suggest solutions to accounting and financial problems;
experience in managing, planning and coordinating the work of
CRM study subordinate staff; ability to prepare clear, concise and accurate
continued fiom page 10 reports and correspondence; initiative, judgement, thorough-
changes in the nature and scope of technical training, schedul- ness and the ability to maintain harmonious working relation-
ing changes, revised rostering practices, and the establishment ships in a multinational environment. Desirable qualifications
or enhancement of a safety department. and experience: Knowledge of UN or ICAO functions and its
There are basic steps that every organization needs to follow organization.
to establish a proactive safety culture that is guided by the best Command of one of the following languages of the
Organization (Arabic, Chinese, English, French, Russian,
possible operational data. These include:
Spanish) is essential and a good working knowledge of one of
a non-punitive policy toward error;
the others is desirable.
commitment to taking action to reduce error- inducing con-
ditions;
data that show the nature and types of errors occurring;
training in error avoidance and management strategies for
crews;
training in evaluating and reinforcing threat recognition and
error; Aviation Organization is a specialized agency of the Unitec
management for instructors and evaluators; and Nations. Headquartered in Montreal, ICAO develops inter
national air transport standards and regulations and serve
fostering of trust. as the medium for cooperation in all fields of civil aviatio~
Trust is a critical element of a safety culture, since it is the among its 185 Contracting States.
lubricant that enables free communication. It is gained by
demonstrating a non-punitive attitude toward error (but not
deliberate violations) and showing in practice that safety con-
cerns are addressed. Data collection to support the safety cul-
ture must be ongoing and findings must be widely
disseminated. (One airline had its LOSA report bound, and
placed copies in every aircraft as well as every base for crews
Web S~te
to peruse.) http.//www lcao ~n
CRM training must make clear the penultimate goals of Sitatex: PAREUYA Facs~mile:011-5115-750974
threat recognition and error management. Ancillary benefits, REGIONAL OFF1 Cable ICAOREP, Pans
Facsrmle: 011-33-1-46418500
such as improved teamwork and morale, are important but not Asla and Pactfic 0 Telephone 011-33-1-46418585
Bangkok, Thatland
the driving force. Finally, instructors and check airmen need S~tatexBKKCAYA
special training in both evaluating and reinforcing the concepts Cable ICAOREP, Bangkok
Telex TH87969 ICAOBKK TH
and in relating them to specific behaviours. Facsmle 011-662-537-8199
Telephone 011-662-537-8189
If all of the needed steps are followed and management's
credibility is established, a true safety culture will emerge and
the contribution of CRM to safety will be recognized. 17

JUNE 1999
PROCEEDINGS OF THE FIRST LOSA WEEK

CATHAY CITY
HONG KONG
12 TO 14 MARCH 2001
Table of contents

Foreword

Objectives and agenda

Introduction

The Line Operations Safety Audit (LOSA)

Presentations
An introduction to LOSA
LOSA and Safety Culture
The phases of LOSA
Making the safety process better
The Air New Zealand 1998 LOSA
LOSA: a natural extension?

Short Course Observer Training


LOSA observation forms
Threat management worksheet
Error management worksheet
Crew interview document
LOSA error codebook
Threat and error management exercises

Threat and error management training

Conclusions

List of participants
FOREWORD

Safety of the civil aviation system is the major objective of the International Civil Aviation
Organization. Considerable progress has been made, but additional improvements are needed and
can be achieved. It has long been known that less than optimum human performance underlies the
vast majority of aviation accidents and incidents, indicating that any advance in this field can be
expected to have a significant impact on the improvement of aviation safety.

This was recognized by the ICAO Assembly, which in 1986 adopted Resolution A26-9 on Flight
Safety and Human Factors. As a follow-up to the Assembly Resolution, the Air Navigation
Commission formulated the following objective for the task:

To improve safety in aviation by making States more aware and responsive to the importance
of human factors in civil aviation operations through the provision of practical human factors
material and measures developed on the basis of experience in States, and by developing and
recommending appropriate amendments to existing materials in Annexes and other documents
with regard to the role of human factors in the present and future operational environments.
Special emphasis will be directed to the Human Factors issues that may influence the design,
transition and in-service use of the [future] ICAO CNS/ATM systems.

In November 1999, the Air Navigation Commission approved a second-phase Plan of Action on
the ICAOs Flight Safety and Human Factors Programme. The collection and analysis of Human
Factors safety data was discussed under the objective of continued integration of Human Factors
considerations into the aviation environment. Human Factors data accessed through existing
reporting schemes originate in accident/serious incident investigation reports, or incident
reporting systems. Such data record unsuccessful human performance (i.e., unmitigated
operational errors). To develop countermeasures to human error it is essential to expand the field
of observation, and access Human Factors data from normal operations. Examples of industry
attempts to capture such data are self-reporting schemes such as the Aviation Safety Action
Programme (ASAP), and flight data acquisition and analysis programmes such as the Flight
Operations Quality Assurance (FOQA) Programme. The Line Operations Safety Audit (LOSA)
was considered a natural extension of these attempts, since it permits recording successful human
performance (i.e., mitigated operational errors), and it leads to more complete conclusions to
develop countermeasures to human error.

LOSA started as a research project funded by FAA and developed by the University of Texas
Human Factors Research Project. It was first presented at the ICAO Fourth Global Flight Safety
and Human Factors Symposium held in Chile, in April 1999. Following the Symposium, ICAO
acquired an in-depth knowledge of the project, and this led to the development of a link between
the ICAO Flight Safety & Human Factors Programme and LOSA. A letter from the Secretary
General of ICAO was sent to the FAA Administrator in February 2000, informing FAA of
ICAOs commitment and support to LOSA, thus formalizing the link. The objective in linking the
FS&HF Programme and LOSA was to come up with an operational deliverable for States and
airlines, within a reasonable period of time.

Progress has been faster than expected, and the accelerated acceptance of LOSA observed during
2000 would suggest a short-term exponential growth of LOSA within the airline community. In
addition to a multiple-fold increase in the number of airlines practising LOSA, an ICAO LOSA
Manual was completed in November 2000, and is expected to be ready for publishing in
November 2001. The manual is a comprehensive source of information for States and airlines. It
was written observing a users-guide style, and it provides all the necessary background as well
as information about the basic tools in LOSA.

If the growth of LOSA materialises, two alternatives are being considered as milestones in the
continuation of the project: either a summit conference at ICAO Headquarters during 2002,
with a view to discuss broad-scale implementation issues; or dedicating the Fifth Global Flight
Safety and Human Factors Symposium, planned for 2003, as a LOSA implementation meeting.
The long-term objective is to give LOSA away, by providing States and airlines with the
necessary tools to implement and manage LOSA or a LOSA-like data collection system on their
own. LOSA ties with widely-implemented electronic data acquisition systems, as well as incident
reporting systems, so it should naturally flow into existing schemes within the airline industry.

With this in mind, ICAO organized, with the generous sponsorship of Cathay Pacific and with the
cooperation of the University of Texas Human Factors Research Project and Continental Airlines,
a three-day meeting on the Line Operational Safety Audit (LOSA) in Hong Kong, on 12-14
March 2001. The meeting was held at the Cathay Pacific complex in Hong Kong, known as
Cathay City. The first day included an introduction to LOSA, followed by a full-day LOSA
observer training. Participants had the opportunity to observe an abbreviated, short training
course, and thus form a complete picture of LOSA. The last day included Continentals threat and
error management training, followed by a strategy session.

Fifty-three participants representing fourteen airlines within the Asia Pacific region, the Hong
Kong Civil Aviation Department, the Civil Aviation Safety Authority (Australia), the Australian
Defence Force, Airbus, Boeing, the Association of Asia Pacific Airlines and IFALPA attended
the meeting.

The event was opened by Captain Henry Craig, Manger Training Boeing, Cathay Pacific, in
representation of Captain Rick Fry, General Manager Flying. The working language of the
meeting was English.

_______________
Objectives and Agenda

To provide a forum to introduce LOSA to the broadest possible airline audience within the
Asia/Pacific region;

To conduct LOSA observer training specific for Cathay, while allowing participant airlines to
observe Cathay-specific training to grasp a better understanding of LOSA;

To provide the basics of threat and error management training;

To discuss possibilities, constraints and resources required for the implementation of LOSA
at the airline-specific level;

To discuss the potential cooperation among airlines within the Asia/Pacific region in sharing
resources and technology as needed to implement LOSA; and

To discuss the potential contribution of non-airline organizations, including civil aviation


authorities, aircraft manufacturers and professional associations, to the worldwide
implementation of LOSA.

Monday, 12 March

1000 - 1200 Coordination meeting


LOSA Week Steering Committee

1300 - 1630 Welcoming remarks


Cathay Pacific

An introduction to LOSA
D. Maurino - ICAO
R. Helmreich - UT
J. Klinect - UT
B. Tesmer - Continental
C. Kriechbaum - Air New Zealand
S. Henderson - Ansett

1630 - 1700 Q&A session

Tuesday, 13 March

0900 - 1700 LOSA observer training sessions

J. Klinect - UT
Wednesday, 14 March

0900 - 1300 Threat and error management training


D. Gunther - Continental

1400 - 1700 LOSA implementation strategy discussion


All participants

1700 Conclusions and wrap up


LOSA Week Steering Committee

_______________
INTRODUCTION training interventions can be re-shaped and
reinforced based on successful performance,
This report introduces a programme for the that is to say, positive feedback. This is
management of human error in aviation indeed a first in aviation, since the industry
operations. The programme, known as has traditionally collected information on
LOSA, or Line Operations Safety Audit, is failed human performance, such as in
proposed as a critical organizational strategy accidents and incidents. Data collected
aimed developing countermeasures to through LOSA is proactive and can be
operational errors. LOSA is an immediately used to prevent adverse events.
organizational tool to identify threats to
LOSA is a mature concept, but its
safety, minimize the risks such threats may
implementation is still in a developing stage.
generate, and implement measures to
Although initially aimed at flight deck
manage human error in operational contexts.
operations because of a question of priorities
LOSA allows airlines to diagnose their
vis--vis limitations in resources, there is no
levels of resilience to systemic threats,
reason why the methodology could not be
operational risks and front-line personnel
applied to other aviation operational sectors,
errors, thus providing a principled, data-
including air traffic control, maintenance,
driven approach to prioritize and implement
cabin crew and dispatch. The initial research
actions to enhance safety.
and project definition was a joint endeavour
between the University of Texas at
LOSA is a programme that uses expert and Austin/Human Factors Research Project and
highly trained observers to collect data about Continental Airlines, with funding provided
flight crew behaviour and situational factors by the Federal Aviation Administration
on normal flights. The audits are conducted (FAA). In 1999, ICAO endorsed LOSA as
under strict non-jeopardy condition; the primary tool to develop countermeasures
therefore, flight crews are not at risk for to human error in aviation operations,
observed actions and errors. The observers developed an operational partnership with
record and code potential threats to safety the University of Texas and Continental
observed and how the threats are addressed Airlines, and made LOSA the central focus
during the flight. They record and code the of its Flight Safety and Human Factors
errors such threats generate, and how flight Programme for the period 2000-2004.
crews manage these errors. Lastly, observers
As of January 2000, the funded research
record and code specific behaviours that
project had conducted eight audits with a
have been known to be associated with
total of nearly 4,000 flights observed. These
accidents and incidents.
audits were conducted both within the
LOSA is linked to with Crew Resource United States and in international operations
Management (CRM) training. Since CRM is and involved two U.S. and one non-U.S.
essentially error management training for operator. The number of operators joining
operational personnel, data from LOSA LOSA has since January 2000 constantly
forms the basis for contemporary CRM increased and continues to increase, and
training re-focus and design. Data from includes major international operators from
LOSA also provides a real-time picture of different parts of the world and diverse
system operations that can guide cultures.
organizational strategies in regard to safety,
The role of ICAO in the LOSA programme
training and operations. A particular strength
is that of an enabling partner. ICAO will
of LOSA is that it identifies examples of
increase awareness of the international civil
superior performance that can be reinforced
aviation about the importance of LOSA, will
and used as models for training. In this way,
facilitate research in order to collect data as
necessary, and will act as cultural mediator
in the unavoidably sensitive aspects of data
collection, by aiming to integrate multi-
cultural observation teams. In line with these
objectives, this report is a first attempt at the
exchange information, and therefore to
increase awareness within international civil
aviation about LOSA. The report is a
generic introduction to the concept of
LOSA, the tools employed, the methodology
underlying the programme, and the potential
remedial actions to be undertaken based on
the data collected. A very important caveat
must be introduced at this point: the report is
not intended to convert readers in instant
expert observers and/or LOSA auditors. In
fact, it is strongly warned that LOSA should
not be attempted without formal training of
observers on the methodology, for the
following reasons.

Firstly, the forms presented in the report are


for practical illustration purposes
exclusively, since they are periodically
amended as a function of experience and
feedback from continuing audits. Secondly,
formal training in the methodology, in the
use of tools, and most important, in the
handling of the highly sensitive data
collected by the audits, is absolutely
essential. Thirdly, the proper structuring of
the data obtained from the audits is of
paramount importance. While LOSA
remains in its development stage, it is highly
desirable that LOSA training be coordinated
through the programmes founding partners.
As the methodology evolves and reaches
maturity, and broader industry partnerships
are developed, LOSA will be available
without restrictions to the total of the
international civil aviation community.

______________
THE LINE OPERATIONS SAFETY AUDIT (LOSA)

Robert L. Helmreich1, James R. Klinect, John A. Wilhelm & J. Bryan Sexton


University of Texas Human Factors Research Project
Department of Psychology
The University of Texas at Austin
Austin Texas

ABSTRACT 2. Incident reports. These are useful because


they provide insights into a far larger database of
The sources of data available to monitor system saves and near misses. They suffer, however,
operations and the safety culture in an airline are because of their voluntary nature and the resultant
described. In particular, the type of information fact that the actual baseline of occurence of
provided by the Line Operations Safety Audit (LOSA) various categories of events is unknown. Despite
methodology developed by the University of Texas efforts to assure pilots and other aviation
Human Factors Research Project is described. A model personnel of the non-jeopardy nature of reports
of Threat and Error Management derived from LOSA under initiatives such as the Aviation Safety
data is reviewed. The historical development of LOSA Action Programs (ASAP: FAA 2000), these
and its uses are presented along with representative programs do not elicit complete reporting. This is
data. The proactive uses of LOSA data for research, both because of embarassment at acknowledging
organizational safety initiatives, and curriculum error and because of suspicion about being
development are described. sanctioned. Nevertheless, programs such as ASAP
do provide invaluable information and allow
UNDERSTANDING SYSTEM SAFETY organizations to take needed safety action prior to
serious accidents and incidents.
An understanding of flight safety can only be gained
from valid, empirical data about normal operations. 3. Line Checks. Although required by civil
There are several sources of such data, each aviation regulators in most countries, line checks
incomplete. However, in combination they can provide generally lack in diagnosticity, espcially in the
a good understanding of the strengths and weaknesses United States, where grading is generally on a
of operations. Aside from proficiency checks of pass-fail basis and in many organizations fewer
technical competence, usually conducted in the than 1% are deemed unsatisfactory. As a result,
simulator, sources of data include: there is minimal diagnosticity obtained at
significant cost (one major airline cites a cost of
1. Accident investigation. Exhaustive analyses $1,000 per line check with no utility in obtained
of factors surrounding accidents have been a information). Pilots are certainly displaying their
primary source of safety information in aviation. best, not necessarily their normative behavior
However, accidents are infrequent events that during a line check.
usually reflect the concatenation of rare factors as
eloquently described by James Reason (1990) in 4. Flight Data Recorder monitoring (Flight
his classic Swiss cheese model. As a result, such Operational Quality Assurance: FOQA, FAA,
investigations probably do not uncover normative, 2001). Especially in new generation aircraft, it has
unsafe operational practices. become almost routine to utilize flight recorder
data to monitor exceedences in performance of the
aircraft and to use these data for safety analysis,

1
Research supporting this paper was supported by Federal Aviation Administration Grants 99-G-004, Robert Helmreich, Principal Investigator.
We wish to thank Captain Bruce Tesmer for his invaluable collaboration in all aspects of LOSA development and for suggesting the name.
without jeopardy to flight crews. While FOQA inspector riding on the jumpseat. The fact that
data provide essential information about what numerous instances of procedural and regulatory
happens in terms of deviations from organizational violations are observed attests to the achievement of
expectations, the data do not provide any insights trust with those observed.
into why the deviations occurred.
At a more macro level, the interview, survey, and
5. Normal Flight Monitoring LOSA. The observations provide both objective and subjective data
Line Operations Safety Audit (LOSA) was on strengths and weaknesses associated with
developed by the University of Texas Human professional and organizational culture, the National
Factors Research Project in conjunction with Airspace System, aircraft design, especially issues
major airlines in the United States as a means of related to automation, and the level of support
collecting normative data on crew performance provided to crews by ground operations, maintenance,
during line flights. The focus of this paper is to and dispatch.
describe the current state of LOSA and the
application of LOSA data for research, LOSA History
organizational safety initiatives, and training.
LOSA was first described here in 1999 at the 10th Thirteen LOSAs have been completed or are in
International Symposium (Helmreich, Klinect, and progress at this time. The initial five involved only
Wilhelm, 1999; Klinect, Wilhelm & Helmreich, assessment of crew performance on the CRM-related
1999), but its scope and acceptance have changed behavioral markers, technical proficiency, and overall
significantly and this will serve as an update as crew effectiveness. The significant shift to include
well as a discussion of the multiple uses of data recording of threats and errors and their avoidance and
collected. management was intitated in collaboration with
Captain Bruce Tesmer of Continental Airlines. At the
DEFINITION, SCOPE, and GOALS OF LOSA same time, the addition of the survey and/or interview
as an integral part of the data collection was finalized.
LOSA consists of a family of methodologies applied Eight LOSAs with the threat and error orientation have
to normal flight operations to assess their strengths and been completed in U.S. major and regional carriers and
weaknesses. At the heart of LOSA is the non-jeopardy, major international airlines.
systematic assessment from the jumpseat of operational
threats and cockpit crew errors and their management. Growing Support for LOSA
Tabulation of threats and errors is augmented by
assessment of CRM-related behaviors associated with Daniel Maurino of the International Civil Aviation
effective and ineffective flightdeck management Organization (ICAO) has been a strong supporter of
(behavioral markers: Helmreich & Merritt, 1998; normal process monitoring as represented by LOSA.
Helmreich, Wilhelm, Klinect, & Merritt, in press). In LOSA has been presented at meetings of ICAO, the
the future, plans are being developed to link LOSA International Air Transport Association, the Air
observations with FOQA data while still preserving the Transport Association, and the ICARUS Committee of
essential, non-jeopardy nature of the methodology. the Flight Safety Foundation. ICAO has named LOSA
its primary human factors initiative for 2000-2005 and
Current LOSA practice combines the observational is issuing a LOSA Handbook this year. Costa Periera,
data with structured interviews of crewmembers Secretary General of ICAO sent a formal letter to Jane
regarding safety issues and/or a survey of attitudes Garvey, FAA Administrator in June of 200 regarding
regarding safety practices, safety and organizational LOSA:
culture, and cockpit management using a specialized
version of the University of Texas Flightdeck [LOSA] acquires direct, first-hand data on
Management Attitudes Questionnaire (see Helmreich the successful recovery from errors by flight
& Merritt, 1998 for application of the FMAQ across crews during normal line flights. [It] is
organizations and national cultures). aimed at collecting data on successful
human performance; and this is indeed a
The key to obtaining useful data is the credible first in our industry, since aviation has
assurance to pilots that the observations are without traditionally collected data on failed human
jeopardy to them. With this trust a picture of flight performance, such as an accident or incident
operations quite different from that obtained by a investigation.
check airman conducting a line check or an FAA
LOSA concepts were presented at a regional human capacity, external stressors, poor group dynamics, and
factors conference in Mexico Citiy in February, 2001. cultural influences. Errors were noted in 64% of flights
An Asian LOSA Summit involving the major carriers observed. (4) manage aircraft deviations, which are
in the region was held in March, 2001, hosted by defined in the model as undesired aircraft states (for
Cathay Pacific in Hong Kong. This will be followed by example, wrong configurations, speed, heading, etc.).
a European LOSA Congress and a Middle Eastern Undesired aircraft states were found in 32% of flights.
meeting hosted by Emirates in Dubai. The goals of These safety tasks are shown in Figure 1.
these meeting are to demonstrate the methodology and
to discuss how to make LOSA widely available to
organizations while preserving the scientific integrity
of data collected and maintenance of a useful Error Avoidance

international LOSA database.

In discussions with senior FAA officials, the Threat Management

potential extensions of LOSA methodology have been Safety


Tasks
discussed. These include adapting parts of the
Error Management
methodology as a tool for FAA Air Carrier Inspectors
to use during enroute evaluations and applying it to
evaluate performance in Line Oriented Evaluations Undesired Aircraft State
Management
(LOE) as part of the Advanced Qualification Program
(AQP). Obviously, under jeopardy conditions such as a
route check by an FAA inspector, the observed
behavior is likely to be quite different from those Figure 1. Safety Tasks .
obtained during LOSA. It is only the classificatory
methodology that may be usefully adapted to the The distinction among safety tasks is important
formal evaluation setting. because different CRM practices (which can be defined
as threat and error countermeasures) are differentially
PLACING LOSA IN CONTEXT: THE THREAT associated with effective accomplishment of each of
AND ERROR MANAGEMENT MODEL the four tasks. When the cockpit crew has put an
aircraft is in an undesired state (for example, at the
The complex data developed through LOSA are best wrong speed or altitude), the primary task is recovery
understood in a model of threat and error in the from the undesired state. From LOSA data we have
aviation system that reflects not only external threats validated the importance of observable CRM skills in
and errors (for example, operational errors by air accomplishing these tasks. The necessary skills fall
traffic controllers, but also enables probing for latent into four groups: (1) Team Climate; (2) Planning; (3)
threats residing in the organization or system such as Task Execution; and (4) Review and Modify. Not
organizational and professional cultures, rostering surprisingly, team climate behaviors such as active
practices, design factors, etc. We presented the model leadership and establishing a team environment are
at the 10th Symposium (Helmreich Klinect, & cricial for all four safety tasks. Planning, in contrast, is
Wilhelm, 1999). Discussion of the components of the most related to error avoidance and threat management.
model is found in Helmreich, Wilhelm, Klinect, & Task Execution behaviors such as monitoring and
Merritt (in press). workload management are central to error
management. Review and Modify countermeasures,
For safe operations, in addition to the the technical which include evaluaton of plans, inquiry and
task of flying, crews must accomplish four safety tasks: assertiveness, are most relevant for threat management
(1) use proactive strategies to avoid committing errors. and undesired aircraft state management.
(2) manage operational complexity, which translates
into threat management. Across all operations The Types and Distribution of Crew Error
observed, threats were recorded in 79% of flights,
including not only environmental factors such as In developing the model of error management we
terrain, weather, and equipment malfunctions but also found empirically that all of the observed errors could
errors external to the cockpit (for example, by ATC). be classified into five types. These are:
(3) manage crew errors. Error is an inevitable result of 1. Procedural errors where crews are trying to
human limitations such as fatigue and other follow procedures but execute them
physiological factors, limited memory and processing
incorrectly (for example, making incorrect U.S. pilots are least accepting of the importance of
entries in the FMC). adhering to rules (Helmreich & Merritt (1998). The
2. Communication errors in which information is fact that such a low percentage of these violations were
improperly or incompletely communicated, consequential might tempt one to dismiss them as
witheld , or misunderstood. trivial, rather like driving a couple of miles over the
3. Proficiency errors where tasks are improperly speed limit on a deserted superhighway. However,
executed because of a lack of skill or additional analyses led to a different conclusion. We
knowledge. split the database into those crews that had at least one
4. Decision errors involving situations not violation and those that had none. We then compared
covered by procedure or regulation in which the number of other (non-violation) errors committed
crews take actions that unnecessarily increase by each group and the percentage of these other errors
risk. that resulted in undesired aircraft states or an additional
5. Intentional non-compliance when crews error. The results showed that crews with a violation
knowingly violate company policy or were almost twice as likely to make other types of
regulations. errors and that these other errors were twice as likely to
be consequential. Thus crews who violate put the
The distribution of errors across airlines completing organization at substantially greater risk, suggesting
a threat and error LOSA is shown in Figure 2, which that while an effective safety culture adopts a non-
gives the percentage of each error type and the punitive attitude towards inadvertent error, violations
percentage of each type which are consequential. By should not be tolerated.
consequential, we mean errors which result in an
undesired aircraft state. The Risky Phase

Non-compliance
Phase of Flight Threats Errors

Procedural
Pre-Departure / Taxi 30% 25%

Communication
Takeoff / Climb 22% 22%
Cruise 10% 10%
Proficiency
Descent / Approach / Land 36% 40%
Decision

0 20 40 60 80 1 00 Taxi / Park 2% 3%
% consequential % of all Errors

Figure 2. LOSA errors and their consequences Figure 3. Incidence of threat and error by phase of
flight
The highest percentage of errors involve intentional
non-compliance or violations, but only 6% of these led Threats and errors were also examined by phase of
to an undesired aircraft state. In contrast, only 5% of flight, with results that strongly confirmed the Flight
errors reflected a lack of proficiency, but more than Safety Foundation Approach and Landing Accident
60% of these were consequential. Similarly, decision Reduction initiative based on the high incidence of
errors were infrequent, but more than half of those that fatal accidents during this phase of flight (Flight Safety
occurred were consequential.2 Foundation, 1998). Our data, shown in Figure 3 on the
previous page, indicate that the highest number of both
Violations Matter threats and errors occur during descent, approach, and
landing. These findings in our initial LOSAs led to a
The high percentage of procedural non-compliance special focus on this aspect of flight operations that we
was surprising, although it is consistent with our data have called The Blue Box as shown in Figure 4. We
showing that of those surveyed in over 20 countries, have started to collect additional data to understand
better the dynamics that occur from the pre-descent
2 briefing through flare and touchdown.
Diferences between these numbers and those reported in earlier
publications are because we are continuously updated the database.
There results remain highly stable across organizations.
represents a start towards understanding the reasons
why crews fail to comply with procedures.

Safety initiatives. LOSA data provide organizations


with concrete data on line operations. Continental
Airlines, for example, has used the data to address
Brief TOD procedures and to provide guidance for crews
18000 ft.

4 regarding high threat operations. The data can also be


10000 ft.
Slow and
integrated with information from quick access
Configure Stabilized

FAF/OM
Approach
Bottom
recorders (FOQA: FAA 1999) to provide greater
Lines
insight into areas of risk. FOQA data were used in our
Flare / Touchdown most recent LOSA to pick airports that represent high
levels of operational threat.

Training and curriculum development. LOSA data


Figure 4. The Blue Box provide insights into areas in need of special training
(for example, one airline initiated new training on
Examining Blue Box data reveals that more captain leadership because this was an area identified
automation and decision errors occur duting this phase. as weak). Another important characteristic of LOSA
This is also the time when threats (including errors) data is the identification of superior performance.
associated with ATC are most difficult to manage. 28% Examples of outstanding behavior (instead of failures)
of errors occuring in the Blue Box resulted in can provide powerful learning.
undesired aircraft states, with the most frequent being
incorrect configurations, vertical deviations, and speed Continental Airlines under the direction of Captain
too high. Blue Box data provide guidance for Donald Gunther (Gunther, in press) has developed a
organizational actions to manage threat and error. new CRM Threat and Error Management course that is
being given to all pilots. The course is build directly on
USING LOSA DATA data from LOSA and, as a result, has high credibility
with pilots.
LOSA data have three major uses for research, for
organizational safety initiatives, and for the REFERENCES
development of training curricula.
Federal Aviation Administration. (2000). Aviation
Research. LOSA data are invaluable for research as Safety Action Programs. Washington, DC. Advisory
they capture the behavior of professionals performing Circular 120-66A.
challenging work in the real world. We have only
scratched the surface of the data for research as most Federal Aviation Administration. (2000). Flight
energy has been addressed to refining the methdology Operational Quality Assurance (FOQA) Notice of
and establishing the database. Several findings, Proposed Rulemaking. Washington, D.C.
however, have already emerged that are of both
theoretical and practical interest. Hines (1998), for Flight Safety Foundation: (1998). Killers in
example, found that under conditions of operational Aviation. Flight Safety Digest. Arlington, Va. Author.
complexity crew performance was significantly better
with the captain serving as pilot not flying while under Gunther, D. (in press). Threat and Error
more benign conditions it made no difference. Bryan Management Training. In Proceedings of the Eleventh
Sexton (Sexton & Klinect, in press) has found that International Symposium on Aviation Psychology, The
crews with more positive (and congruent) safety Ohio State University.
attitudes commit errors that are more likely to be
inconsequential, trap more errors, are less likely to Helmreich, R.L., Klinect, J.R., & Wilhelm, J.A.
commit sequences of errors, and less likely to make (1999). Models of threat, error, and CRM in flight
unstable approaches. This linkage between attitudes operations. In Proceedings of the Tenth International
and behavior is important both practically and in Symposium on Aviation Psychology (pp. 677-682).
addressing an old controversy in psychology about the Columbus, OH: The Ohio State University.
linkage between attitudes and behavior. This also
Helmreich, R.L., & Merritt, A.C. (1998). Culture at
work in aviation and medicine: National,
organizational, and professional influences. Aldershot,
U.K.: Ashgate

Helmreich, R.L., Wilhelm, J.A., Klinect, J.R., &


Merritt, A.C. (in press). Culture, error and Crew
Resource Management. In E. Salas, C.A. Bowers, & E.
Edens (Eds.), Applying resource management in
organizations: A guide for training professionals.
Princeton, NJ: Erlbaum.

Hines, W.E. (1998). Teams and Technology: Flight


Crew Performance in Standard and Automated
Aircraft. Unpublished doctoral dissertation. The
University of Texas at Austin.

Klinect, J.R., Wilhelm, J.A., & Helmreich, R.L.


(1999). Threat and error management: Data from line
operations safety audits. In Proceedings of the Tenth
International Symposium on Aviation Psychology (pp.
683-688). Columbus, OH: The Ohio State University.

Reason, J. (1990). Human error. New York:


Cambridge University Press.

Sexton, J.B. & Klinect, J.R. (in press).The link


between safety attitudes and observed performance in
flight operations. In Proceedings of the Eleventh
International Symposium on Aviation Psychology, The
Ohio State University.
PRESENTATIONS
An Introduction to
LOSA:
The Perspective of
ICAO
Captain Dan Maurino
Flight Safety and Human Factors - ICAO
First LOSA Week
Cathay City, Hong Kong
12-14 March 2001
The Ultra-Safe System
Fragile systems
Individual risk management
10-3 Accident investigation
Safe systems
Design
Regulations & procedures
10-5 Reporting systems

Ultra safe systems


Uncertain accident prediction
Reporting systems inconsistent
10-6 Political management
Common Sense Approach
ISO 9000/TQM safety
z Safety results from specifications
(comprehensive rules & procedures)
z Good professionals adhere to rules &
procedures
z Good professionals and good rules &
procedures ensure safety
The Pentium Illusion

GIGO
WYGIWYA
Data Does Not Mean Information
What About Human Error?
Design & manufacture
Management & supervision
Training & maintenance
Stakeholders
Operational
personnel

Errors reside
in the person
From Clear to Opaque Causality
Direct & linear Circular & random
Broadening the Scope

Design & manufacture


Management & supervision
Training & maintenance
Stakeholders
Operational
personnel
Errors reside
in the context
Unanticipated Disturbances

Operational
contexts
Warning! cannot be
To err is
human
entirely
Insert
pre-
carefully specified
Monitoring Normal Practice
Design & manufacture
Management & supervision
Training & maintenance
Stakeholders
Operational
Errors do not personnel

cause accidents
Operational Behaviours
Accomplishing the systems goals

Safety Production
Processes & Outcomes

Error:
Causes and
consequences
are not linear
in their
magnitude
Training Behaviours
Accomplishing training goals

Safety Production
Error: Once in A Million Departures

Flaps Checklist Unheeded


omitted failure warning

Error Deviation Amplification Degradation/


breakdown
Error: Quite Frequently

Flaps Checklist Effective


omitted works warning
LOSA

Error Deviation Amplification Normal


operation
Where Experts Dare
Crisis Operation Design

Expert operating Normative


zone operating zone
A Principled Approach
Consequential
Proficiency 69%
Operational Decision 51%

Communications 11%

Procedural
Procedura 23%
Violations 2%

0 10 20 30 40 50 60 70
Percentage of Errors
Violations and Open Systems
Accident Production
Higher objectives
Incident
Regulations
Violations Safe &
Risk efficient Technology
system People
performance
Training

Max System Output


The Role of the Regulator
Think about the spirit rather than the
letter of the law
Legislation
Decoration
Litigation

Aviation: a geriatric and


saturated system facing
significant threats
Getting the Best for the Buck
Intervention Systemic approach
efficiency Safety culture
Training
Error Management
Error Resistance

Regulations Flexible regulation

Engineering solutions Adaptive


technology

10-2 10-3 10-4 10-5 10-6 10-7


Risk of accident
The Ultra-Safe System: Epilogue
z Aviation cannot be entirely specified
Disturbances will occur; humans will
inevitably make errors
z Basic normative framework
z Flexible & adaptive implementation

z Deviation management

Danger: loss of control of the


deviation management process rather
that deviations themselves
The Line Operations Safety Audit
(LOSA) and Safety Culture
Robert L. Helmreich

The University of Texas Human Factors Research Project


The University of Texas at Austin

LOSA Summit
Cathay Pacific
Hong Kong
Human Factors
Research Project
Data for a Safety Culture
How do airlines monitor safety?
Accident investigation
Incident reports
Data slanted to events resulting from system and flight crew failures
Line checks
Data show crew proficiency and procedural knowledge
Flight Data Recorders FOQA
Data show what happened in terms of flight parameters
Observing normal flights Line Operations Safety Audit (LOSA)
Gives data on why things happen and how they are managed
Provides a more realistic baseline of safety
Line Operations Safety Audit (LOSA)
LOSA Jump seat observations of flight crew performance
during regular scheduled flights

Observers unobtrusive collecting data not participating in flight

Team of observers from different backgrounds


Line pilots / Union representatives
Check airmen
Safety and Training pilots
UT observers

All data are DE-IDENTIFIED and CONFIDENTIAL


Purpose of LOSA
Data provide a baseline of safety pertaining to:
1. Crew performance strengths and weaknesses
Proficiency
Decision-Making
CRM skills
Procedural compliance

2. System performance strengths and weaknesses


Culture
Airspace System airports and navigational Aids
Aircraft design / automation
Standards / Training / Safety / Maintenance
Crew support ATC, Cabin, Ground, and Dispatch
LOSA History 1992 - 2001

Threat and Error


Flight Management LOSA Management LOSA
Delta domestic Continental Latin America
Delta international Continental Express
TWA Gulfstream Express
American Air New Zealand
Continental Air Micronesia
USAir Continental
Delta
USAir
Cathay Pacific
Components of LOSA

Part 1. Flight crew survey


Part 2. Flight crew interview
Part 3. Descent and approach
Part 4. CRM Countermeasures
Part 5. Threat and error management
LOSA Data: Flight Crew
For each flight segment, observers collect data on:

General Flight Crew Flight Crew Performance


Information Behavioral markers
Demographics
Crew errors and violations
Attitudes / perceptions (FMAQ)
Undesired aircraft states
Safety interview comments
Technical data for approaches
Type and stability
Flight Description
Threat management
Observer narrative
Error management
Overt threats
Operational complexity Undesired state
management
The Safety Bottom Line

To fly safely, crews must

1 Avoid committing errors Error Avoidance

2 Manage operational complexity Threat Management


Safety
Bottom Line
3 Manage their own errors Error Management

4 Manage aircraft deviations Undesired Aircraft


State Management
Error Avoidance

1 Avoid committing errors Error Avoidance

Safety
Bottom Line
Error Avoidance
Complete error avoidance is impossible errors are
inevitable

LOSA data from 429 flight segments at five airlines


64% of the flights observed had at least one error

Must look for roots of error to strengthen system defenses


Human Limitations Lead to Error

Limited memory capacity


Limited processing capacity
multi-tasking capability
Limits imposed by stressors
tunnel vision
Limits imposed by fatigue and other physiological
factors
Poor group dynamics
Cultural influences
Threat Management

1 Avoid committing errors Error Avoidance

2 Manage operational complexity Threat Management


Safety
Bottom Line
Threat Management
Threats - Originate outside the flight crews influence but require active
management to prevent them from becoming consequential to safety
Adverse weather Operational time pressures
Terrain Non-normal operations
Traffic ATC command events / errors
Airport conditions Cabin events / errors
A/C malfunctions MX evens / errors
Automation events Dispatch events / errors
Communication events Ground crew events / errors

LOSA - 79% of the flights observed had at least one threat


Error Management

1 Avoid committing errors Error Avoidance

2 Manage operational complexity Threat Management


Safety
Bottom Line
3 Manage their own errors Error Management
Flight Crew Error Types
1. Intentional Noncompliance violations
ex) Performing a checklist from memory

2. Procedural Followed procedures but wrong execution


ex) Wrong altitude setting dialed into the MCP

3. Communication Missing information or misinterpretation


ex) Miscommunication with ATC

4. Proficiency Error due to a lack of knowledge


ex) Lack of knowledge with automation

5. Decision Discretionary decision that unnecessarily increased


risk
ex) Unnecessary navigation through adverse weather
Error Frequencies and Outcomes

Violations

Procedural

Communication

Proficiency

Decision

0 20 40 60 80 100
Percent Consequential Percent Frequency
The Importance of Violations
Airlines cannot allow violations to normalize

Why?
Violations cultivate complacency and a disregard of rules

Crews that commit at least one intentional noncompliance error are


twice as likely to:
Commit unintentional errors (Procedural, Communication ..)

Commit consequential errors that lead to additional errors or


undesired aircraft states
Error Management Results
Error Responses Most errors are undetected
64% Undetected
31% Detected and effectively managed
5% Detected and mismanaged

Error Outcomes Most errors are inconsequential


Inconsequential 77%
Additional Errors 6% (Error Chains)
Undesired Aircraft States 17%
Undesired Aircraft State Management

1 Avoid committing errors Error Avoidance

2 Manage operational complexity Threat Management


Safety
Bottom Line
3 Manage their own errors Error Management

4 Manage aircraft deviations Undesired Aircraft


State Management
Undesired Aircraft States Resulting
from Threat and Error
Aircraft deviations at edge of the safety envelope
Incorrect aircraft
Unstable approach
configurations
Long or hard landing
Vertical deviations -
altitude Wrong taxiway / runway
Lateral deviations - Wrong airport
heading Wrong Country
Speed deviations

LOSA 32% of the flights had an undesired aircraft state


The Riskiest Phase

Phase of Flight Threats by Phase Errors by Phase

Pre-Departure / Taxi 30% 25%

Takeoff / Climb 22% 22%

Cruise 10% 10%

Descent / Approach / Land 36% 40%

Taxi / Park 2% 3%

Descent / approach / land phase also contains the most


variability in crew performance and consequential errors
Safety Culture:
Survey Data and Error Management

Human Factors
Research Project
Flt Ops listens

My safety sugg acted upon

Mgmt not compromise safety


Safety
Culture
Encouraged to report unsafe

I know safety channels

Chief Pilot availability

FMAQ survey items form index of Safety Culture


Pilot Perceptions of Safety Culture

# Undesired States per Segment


1.2
1.1
1 1.13
0.9
0.8
0.7
Scale 0-100

0.6
0.8
0.5
0.4
0.3
0.2
0.1
0 0.19
Low Middle High
Pilot Perceptions of Safety Culture

% Errors that were Trapped


100
90
80
70
60
Scale 0-100

50
40
57.5%
30
20
10 23.6%
0 14.3%
Low Middle High
A Model of Threat and Error Management

Human Factors
Research Project
Purpose and Uses
The Threat and Error Management model was derived from
LOSA data and guides further refinement.
The model is being used by airlines as a framework for
analysis of incident and ASAP data
Critical Definition
Latent Threats are aspects of the system that predispose
the commission of errors or can lead to undesired aircraft
states
ATC practices
Scheduling practices that result in fatigue
Organizational, national, professional culture
etc
Latent System Organizational Professional
Threats
Expected Unexpected Events External
External Error
Error
Unexpected Events External Error
External Events and Risks and
and Risks
Threats and Risks
Risks

Violations - Communication
Procedural - Proficiency
Operational Decision

Threat Recognition Error Management


CRM and Error Avoidance
Behaviors
Behaviors Behaviors

Undesired Additional
Outcomes Inconsequential Error
Aircraft State

CRM Undesired State


Behaviors Management
Behaviors

Recovery to Additional
Safe
Final Flight Safe Flight Error
Outcome
Incident
Accident
LOSA and CRM

Human Factors
Research Project
Twenty Years of Change
Since 1981, CRM has evolved steadily from early courses that
were derived from corporate management and focused on
individual styles and skills

By 1998, the focus of the fifth generation had shifted to error


management

In 2001, sixth generation has become Threat and Error


Management
CRM skills defined as threat and error countermeasures
CRM
Problem Little data supporting CRM curriculum content
How do flightcrews detect and manage error in normal operations?
What are effective and ineffective error management strategies?
What magnitude does CRM play in error management?

LOSA - provides critical data

Continental Airlines use of LOSA data


Recognized need for revisiting basic CRM due to decay in acceptance
1996 Error Management (5th generation CRM) all pilots
2001 Threat and Error Management (6th Generation CRM) all pilots
An Expanded Definition of CRM
Sixth Generation CRM

Error Avoidance

Threat Management

CRM Skills

Error Management

Undesired Aircraft State


Management
Characteristics of Threat and Error
Management CRM
Based on data from airlines own experience
LOSA, FOQA, ASAP

Allows crews to assess threats and their management

Examines effective and ineffective error management

Operational focus avoids games and irrelevant activities

Gives crews specific threat and error countermeasures


LOSA and 6th Generation CRM
LOSA validates the role CRM plays in safety
Error Avoidance
Threat Management
Error Management
Undesired Aircraft State Management

Four groups of countermeasures (CRM skills) show up in LOSA


reflecting both superior and below standard crew performance
1. Team building
2. Planning
3. Execution
4. Review / Modify
Using LOSA Error Data for CRM Design
and Organizational Interventions

Violations - suggest poor procedures, weak captain leadership and/or a


culture of non-compliance
Procedural errors - may indicate poor workload management and/or poor
procedures
Communications errors - may reflect inadequate CRM (monitoring and
challenging) or complacency

Proficiency errors - suggest pressures to train and/or need for higher


standards and better monitoring by check airmen

Decision errors - may indicate need for more CRM training on expert
decision making and risk assessment
When LOSA Succeeds

Top management recognizes the need for data

Resources are provided for the process

Commitment to use the data for change

Pilots support the process


When LOSA is Inappropriate

LOSA will not work in a culture of blame


and punish

LOSA should not be undertaken without the


cooperation of the regulatory authority
LOSA Meetings 2000-2001

Australian Aviation Psychology Symposium,


Melbourne,
November, 2001
International Aviation Psychology Symposium
Ohio State University, March
Asian LOSA Summit, Hong Kong, March
ICAO Human Factors, Sta Cruz, Bolivia, April
European LOSA Summit, Zurich, TBD
Middle Easter LOSA Summit, Dubai, TBD
Latin American LOSA Summit, Panama, TBD
Summary

LOSA has proved to be a valuable safety


initiative

LOSA facilitates CRM curriculum


development

LOSA is still under development


It will evolve just as CRM has
University of Texas
Human Factors
Research Project

www.psy.utexas.edu/psy/helmreich/nasaut.htm
The Phases of LOSA
James Klinect
LOSA Week, Cathay Pacific, Hong Kong
March 12-14
Phases of LOSA
I. Planning
II. Observer Training / Recalibration
III. Observations
IV. Data Entry / Cleaning / Analysis
V. Final Report
VI. Communication of Results
Phase I
Planning
Planning Committee
First - Form a LOSA steering committee with all
major stakeholders represented
Flight operations
Safety
Training
Standards
Union

Aim is to give everyone ownership of the project


Planning and Education
Large part of the planning phase is education

Education to the line pilots - letters


Introduction letter three months before the project
Reminder letter on the week of project initiation
All letters should be signed by management and union
representatives

All forms of communication must emphasize:


LOSA is a non-jeopardy collection of safety data
All data are de-identified and confidential
Planning and Commitment
After education, need to get commitment from
regulatory, management, and union parties

Regulatory - Commitment to brief and maintain


the integrity of a non-jeopardy process

Management Commitment to use the data and


make changes if necessary

Union Commitment to support LOSA and offer


a quality check to the process
The Shotgun LOSA Approach
After education and gaining commitments, need to
establish LOSA goals

Shotgun LOSA it is only a start


Number of observations is dependent upon airline size
Want to observe as many different crews as possible
Provides a first baseline for threat and error management

For fleet comparisons, 30-50 observations per fleet


is enough to make confident conclusions
Observer Team
Composition of the observer team
Diversity is key all observers are biased to something

Line check airmen make sure most of their


observations are done outside of their own fleet

When selecting observers, think about someone


that will take the time do a good job
Many observations takes 30 min. to an hour to write up
Phase II
Observer Training / Recalibration
Observer Training
Two solid days

Typical schedule
Introduction to LOSA
How to be a fly on the wall observer
Observation form training and calibration
Wrap-up with logistics

End of training should mark the beginning of


LOSA observations
Dont allow lag time between training and observations
Recalibration
Recalibration is critical for data quality assurance

What is recalibration?
After 3 to 5 observations, time is set aside to go over
observer questions or concerns
Can be done individually or in a group

Once recalibration is complete, the observer is set


free to conduct the rest of their observations
Phase III
Observations
Observations
Scheduling - Make sure the observer has a game
plan for their observations
Example)
Observe 20 segments
10 long haul 777 and 10 regional A320
Make one out and back to SFO (LOSA highlighted airport)

Encourage observers to complete their forms


electronically allows for e-mail and copies

Instruct observers to send all of their observations


to a trusted site to maintain confidentiality
Phase IV
Data Entry / Cleaning / Analysis
Data Entry
All data are entered into Microsoft Access and
analyzed with a statistical software package (SPSS)

Have a process in place to organize data entry


Assign each returned form with a unique ID number

Keep a log of the ID numbers for effective tracking


and monitoring of LOSA
Also include identifiers such as aircraft type and city pairs
Data Cleaning
What is data cleaning?
Process of examining all threats, errors, and undesired
aircraft states to ensure correctness and coding
accuracy

Data cleaning takes time


Most recent data cleaning for 372 observations
180 pages of narratives (8 pt font, single-spaced)
1100 threats
730 errors
120 undesired aircraft states
Roundtables
The best way to clean is to divide and conquer
One person to go over the narratives to extract
unlogged threats, errors, and undesired aircraft states
Another person covers the accuracy of threats
Another person on errors and undesired aircraft states

Bring all problems to Roundtables for discussion


Keep the group small LOSA planning committee

The Final Roundtable jointly validates the


dataset for analysis
Data Analysis: Most General
Observation form data
Demographics overview
Threat exposure with management rates
Errors committed with management rates
Undesired aircraft states with management rates
Above/Below Good Threat and Error Countermeasure ratings

Crew interview data


Breakdown by fleet and code comments by theme

Survey data
Number of respondents who agreed / disagreed with each item
Phase V
Final Report
Final Report
The final report provides the most general
diagnostic data of strengths and weaknesses

Limitation The final report only scratches the


surface on why things occurred

Many answers to the why questions requires a


coding analysis of the narratives hard work
The focus of future research hopes to ease this process
Phase VI
Communication of Results
Communication of Results
Executive briefing
Give top-level management the first look
Have recommendations ready on how to use the data to
incorporate change if necessary

Fleet manager debriefing


Provide an in-depth analysis that are fleet specific
Good place to form committees for change

Communication to the line


Provide information about the results and how they are
being to used to improve the system
LOSA
Making The SafetyProcess
Better

Line Operations Safety Audits


Improving The Safety Margin
What has your company done to improve your safety
margin in the last five years?
Manuals Updated ATC Arr/App/Dep Help revise
Training Enhanced
Airport Issues Resolve
Equipment Upgraded
Maintenance Increased Staff
New Procedures - Added
Ground Towing Super-Tug
Old Procedures Out
Checklists Modified New Flight Sim Buy & Install
New Aircraft Purchased Flight Following Upgrade
Dispatch Improved WX, Aircraft and ground
NOTAMS, Flt Plan communications equipment
New systems Windshear, Web-site Install Info for crews
(E)GPWS Continuing Safety Program -
Old aircraft - Retired Develop
Why Are These Safety
Improvements Required?
Because Things

CHANGE!
NOT ETOPS CERTIFIED
Flies Good, but the visual needs work.
Instruments on the first blind flight
There was no FMS
CHANGES in an organization happen:

1). Through its PEOPLE

or
2). Through a TRAUMA (Accident)

Either Way, Change Will


Happen!
SYSTEM CHANGES
Affect Your Line Operations
Things are happening, on the line, that increase
your risk of a major accident
- and -
You do not know about them!
Why are things happening?
System Changes
Create Threats (Overt & Latent)
Stimulate Crew Errors
Why dont we know about these
things
Change limits your Big Picture
Crews are operating unsupervised
Crews do not report they fear punishment
Crews normal performance is different from
Check ride performance
Management has difficulty screening out
valid reported crew concerns from over-
reported crew complaints
Why we need know about these
things

These unknown things (error chains &


undesired aircraft states) - are the
precursors to accidents / incidents

LOSA provides data that identify these


accident/incident precursors
Line Operations Safety Audit
(LOSA)

Providing Data
To Better Manage CHANGES
LOSA - Gives You:

Safety data on what pilots really do

Data on what good crews do right

Buy-in by employees

Proactive approach

A plan for continuous improvement


Employee Buy-In & Impact on
Safety
Observations show reality

Previous encountered events

Crews believe in management when


management makes improvements
Results From Being Proactive
With LOSA
Results:
Better policies & procedures
Better crew compliance
Fewer unstable approaches
Better risk assessment of operations at all
levels
Targets for safety resources
A Continuous Improvement Plan
LOSA - A Focus Safety Tool

LOSA A Baseline Safety Tool

LOSA A Rotating Safety Tool

LOSA A Safety Change Measurement Tool


Past LOSAs

1996 836 segments (Entire System)

1997 COEX

1998 Latin American

1999 Pacific

2000 Entire System


LOSA 2000

There were five parts to LOSA 2000:


Part 1. Flight Crew Survey
Part 2. Flight Crew Interview
Part 3. Stabilized Approaches
Part 4. CRM Counter-Measures
Part 5. Threat and Error
Survey Data
Organizational Climate
Safety Culture
Training/Checking
Working Together
Stress
Organizational Climate items

Pilot morale is
high

I'm proud to work


for CO

I like my job
Organizational Climate at CO Across Time
100
90
80
70 82%
60 72%
Scale 0-100

50 63%
40
30
20
10
0

1996 1999 2000


CO CO CO

Organizational Climate Across Airlines


100
90
80
70
60
Scale 0-100

50
40
30
20
10
0

a1 1 2 3 4 000 5
Asi
A A e 1 A er 1 A A
US US ro p US A m US CO
2 US
Eu S
CO Safety Culture over time
Safety Culture Items
100
90
80
Pilots trust senior mgmt at CO 70

S cale 0-100
60
50
68%
40 53% 57%
30
Mgmt never compromises safety for profits
20
10
0
1996 1999 2000
Mgmt - Flt Ops listen & care about our concerns

Safety Culture Across Airlines


100
My safety suggestions would be acted upon 90
80
S cale 0-100 70
60
50
I'm encouraged to report unsafe conditions 40
30
20
10
0
Senior mgmt at CO is doing a good job
er 1 1 er 2 2 er 1 er 3 00 0
Am pe pe Am O2
N E u ro N Am E u ro S N Am C
Satisfaction with Training and Checking

Ground School

Checklists

Quality of initial training

Sim-based training

Flight Stds & Training overall

Fairness of checking
Improvement in Teamwork and Cooperation since 1996

Ramp Personnel CO 1996


CO 2000

Scheduling

Gate Agents

Flight Attendants

Dispatch

Maintenance

Other cockpit crew

High Satisfaction with the Quaility of Teamwork and Cooperation


Realistic Appraisal of Stress Items
I am more likely to make errors in emergencies

Personal problems affect my performance

I let others know when I'm becoming overloaded

I am less effective when stressed/fatigued

Situation awareness suffers when I am fatigued


Interview Topics
Safety concerns: Other topics
Operations Training
ATC Communications
Dispatch Stable Approaches
Checklists
SOP
Interview Responses By Category
350 320
300
262
N um ber Of R esponses

250

200 173
156
150
102
100

50
11
0
ATC and Checklists Flight Dispatch SOP Other Areas
Airports Operations
Response Rates - ATC & Airports:
By Fleet Type
2.50
2.20

2.00
1.76
Responses per Crew

1.50 1.41
1.33 1.30

1.00

0.50

0.00

B737 B757 B777 DC-10 MD-80


Responses by Percent

0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
20%

Ground Delay

9%
Flow/Reroute/Hold

6%
Clnc/Chgs/Rwy swap
8%

Comm/ Language
18%

Slamdunk/Spacing
16%

Arrival/Dept.Procedures
19%

Rwy / Twy
5%

Congstn/Danger
14%
ATC & Airports - Response Categories

Misc
5%
Airports With Serious Concerns : Traffic, Procedures,
Airport Layout (52 of 120)

14 13

12

10 9
8
8
6
6 5

4 3 3
2
2 1 1 1

0
SNA DCA EWR LAX IAH ATL, SFO, STL, GDL, MDW, UIO,
LGA, CLE, JAX, PDV, CDG, TGU,
SAN LAS Rome AUS, CDC, SJO,
MXP MEX Milan
Non-Conforming (Unstable)
Approaches
LOS A: No n-Co nfo rming Appro ac he s
Outs ide Re quire d
1000 ft o r 500 ft Windo w

-49%
-59% -59%

1996 - ALL 1998 - L.A. 1999 - P ACIFIC 2000 - ALL


NCA: Type Of Approach
80%
80%

60%

40%
13%
20% 7%

0%
Visual Non-Prec Prec

NCA: Pilot Flying

100%
80% 47% 53%
60%
40%
20%
0%
Capt FO
NCA: Number Of Parameters Out-Of-Limits
@ The Window - Per Approach
47%
50%
40%
30% 20% 20%
20% 13%
10% 0 0
0%
ONE TWO THREE FOUR FIVE SIX

NCA: Parameters Not Met @


The Window
73%
80%

60% 47%
40%
40%
33%

20% 13%
0%
Speed Vert Dev Unspool Vert/Spd Lat Dev
NCAs Affected By ATC (47%)

47%
50%
40% 27%
30% 20%
20%
10%
0%
App Chng Rwy Chng Slam Dnk

NCA - Crew Response At The Window

Force a Ldg Go - Around


Other Targeted Successes
Checklist protocol errors
Coordination with AQP
Captain Leadership performance
Intentional Non-Compliance errors
FOQA validation/coordination
Validated our change process
THE AIR NEW ZEALAND 1998
LINE ORIENTATED SAFETY
AUDIT

Captain Chris Kriechbaum


CRM Programme Manager (NZ)
US

600 pilots, 1600 cabin crew


All Boeing fleets, 7 x 747, 13 x 767, 16 x 737
97 sectors audited
No QAR data
WHY?

Pro-active approach to safety


Needed facts not opinions
Needed to establish where our programme sat
with the rest of the international community
Down under inferiority complex??
THE EXPERIENCE

Enjoyable

Management commitment initially a challenge

Dont underestimate the administrative


support required
THE RESULTS

Generally extremely positive

Automation issues were a dominant theme

Recommendations for procedure changes

Culture issues
THE BENEFITS
Established a baseline
Elevated CRM profile with in the Company
Credibility in International Aviation Community
Removed the scatter gun effect of training
Combined Training
Targeted Error Management re- focus
Updated Command Course
SOP changes
Just Culture
Value for money
Justification of Capital and other Expenditure.
WHERE TO FROM HERE ?

A continuous
STANDARDISATION
cycle of
improvement
TRAINING SAFETY
L.O.S.A.

O.O.R.s
Q.A.R.s
LOSA
A Natural Extension?

Simon Henderson
Ansett Australia
Operations and Standards

nt
c ie
ffi
e
nd
a
f e
S a
Operations and Standards

Assumption
Operations
Real world Check
performance is Data
accurately reflected
by check
performance
Tension and Conflict
Collected Data

Training Need
Tension and Conflict
Collected Data

nt
e
em
ag
an
M
i sk
R

Training Need
Traditional Data From?
Inc k
ide ec
n ts Ch
Operations
Safety
Observe Examine
Data
Reports

? ?
Additional Data From?
Inc k
ide ec
n ts Ch
Operations
Safety
Observe Examine
Data
Reports
Aud
it
Q AR
? ? LO
SA
Data Management

What do you need to do?


Evaluate the training system ability to deliver the product
needed
Capture compliance related information
Capture/measure individual standards
Capture/measure the reliability of the evaluators in the
training system
Additional Data From?

Inc k
ide ec
n ts Ch
Operations
Safety
Observe Examine
Data
Reports
Aud
it
AR
Q LO
RISK SA
MANAGEMENT
Risk Management
Crew Capability
Availability
and Demand

Risk
Likelihood and Consequence
Risk Management

From LOSA database. Likelihood based on


Threat
LOSA observations, QAR, etc. Likelihood

Consequences based on SME input, LOSA


Threat
consequentials, Safety etc. Consequence

Capacity based on Situation. From


Crew
Standards database, LOSA, etc. Capacity

Demand based on the Threat. From


Crew
Standards Database, incident review etc. Demand
Match Data to the Training
Need
Collected Data

nt
c ie
ffi
e
nd
a
f e
S a

Training Need
Captain Simon Henderson
Flight Training Project Manager
Ansett Australia
Simon_Henderson@ansett.com.au
SHORT COURSE
OBSERVER TRAINING
Short Course Observer Training
LOSA Week, Cathay Pacific, Hong Kong
March 12-14, 2001

The University of Texas Human Factors


Research Project
Reasons for a Short Course
Not really training it is more informational

Provides you with an in-depth understanding on the type


of data collected through a LOSA

An opportunity to exchange ideas to improve the current


process

Lets keep it interactive ask questions anytime


Conducting a LOSA Observation
Observation Tools / Instruments
1. LOSA Observation Rating Form 9.0
Demographics
Narratives
Threat and error countermeasures (CRM behavioral markers)
Descent / Approach / Land technical data
Threat and error management
Crew interview

2. LOSA Code Book 9.0

3. Pocket Notepad and Quick Reference Card


Observation Guidelines
Goal #1: - Strive to be a fly on the wall observer
Main issue when does an observer speak up?
Think of yourself as a jump seat rider on another airline
Avoid filling out the observation form in the cockpit take notes
Dont attempt to debrief unless asked
Keep all data de-identified and confidential

If observers fail to be flies on the wall LOSA will fail


Observation Time
For flights under three hours, stay in the cockpit

For flights over three hours, only observe:


Predeparture / Taxi-out
Takeoff / Climb
45 minutes after TOC (Good time to conduct the interview)
45 minutes before TOD
Descent / Approach / Land
Taxi-in
The Captain Brief
Ask for permission to ride and observe

Stress that data is de-identified and confidential

Ask if you can take notes on a notepad

Determine the best time to conduct the crew interview

Set up a schedule for time away from the cockpit

Inform the Lead Flight Attendant


LOSA Observation Form
Walk-Through
Demographics
(Page 1)
Demographics Top Three
1. Observer ID stay consistent

2. Dont forget the observation number - start with #1

3. Collect demographics as at the start of the crew interview


Narratives and Threat and Error
Countermeasures
(Pages 2-7)
Narratives Top Three
1. Narratives should provide a context - what and how
1. What did the crew do well?

2. What did the crew do poorly?

3. How did the crew perform when confronted with threats, errors,
or other significant event?

2. Keep the narrative concise and simple

3. When in doubt, write in the narrative


Countermeasures Top Three
1. Only rate with confidence - Dont have to rate everything

2. Another way to think about the ratings..


1 = Poor Major contributor to failing a check ride
2 = Marginal Would require a debrief
3 = Good Standard performance
4 = Outstanding Training tape quality

3. If you give a 1,2, or 4 rating, justify it in the narrative


Descent / Approach / Land
Technical Worksheet
(Page 4)
The Most Dangerous Phase
Data from the Descent / Approach / Land Phase
Most accidents worldwide (ALAR)

Most threats

Most mismanaged threats

Most crew errors

Most mismanaged crew errors

Most variability in threat and error countermeasure ratings


The Blue Box

Brief TOD
Transition

4 Altitude

10000 ft.
Slow and
Configure Stabilized
Approach
FAF/OM Bottom
Lines

Flare / Touchdown
Descent/Approach/Land Top Three
1. Fill out the technical data worksheet first and use The
Blue Box landmarks in your narrative

2. If the approach was unstable, laundry list the


contributing factors

3. We defer tell us what is going on in The Blue Box


Threat and Error Management
Threat and Error Management
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Error

Crew Error
Responses

Error Outcome

Error Induced
Incident or
Accident
Threat Management Worksheet
(Page 8)
Threats Defined
Threat Events or errors that originate outside the influence of the
flight crew but require their attention to maintain safety

1. Environmental Threats 4. Crew Support Threats


Adverse WX Dispatch events / errors
Terrain Ground events / errors
Airport Conditions MX events / errors
Heavy traffic / TCAS events
5. Operational Threats
Time Pressures
2. ATC Threats
Irregular Operations
Command events / errors
Radio congestion / poor
Language difficulties reception

3. Aircraft Threats 6. Cabin Threats


Malfunctions Cabin events / FA errors
Automation events
Threat Management Top Three
1. If a threat induced a crew error, enter NO for
effectively managed

2. See more than five threats? staple another worksheet


and renumber

3. When in doubt, make sure you include answers the


following somewhere on the worksheet
1. What was the threat?
2. How did the crew respond to the threat?
3. What was the outcome of the threat?
Error Management Worksheet
(Pages 9-11)
Error Management Worksheet
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Error

Crew Error
Responses

Error Outcome

Error Induced
Incident or
Accident
Crew Error Types
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Intentional
Noncompliance
Procedural
Communication
Proficiency
Decision

Crew Error
Responses
Error Induced
Incident or
Error Outcome Accident
Error Types Defined
1. Intentional Noncompliance intentional deviations from SOP
Ex) Performing a checklist from memory

2. Procedural Followed procedures with incorrect execution


Ex) Wrong attitude setting dialed

3. Communication Missing information or misinterpretation


Ex) Miscommunication with ATC

4. Proficiency Error due to the lack of knowledge or skill


Ex) Lack of knowledge with automation

5. Decision Crew decision unbounded by procedures that


unnecessarily increased risk
Ex) Unnecessary navigation through adverse weather
START
Was the error associated with a: YES
- Miscommunication Communication Error
- Misinterpretation
- Failure to communicate Go to Page 5
pertinent information
NO

Was the error associated with a: YES


- Lack of knowledge Proficiency Error
- Lack of stick and rudder
proficiency Go to Page 5

NO
Was the error associated with a:
- Decision that increased risk in YES Operational Decision Error
which there were no written
procedures to follow Go to Page 6
NO

Was the error associated with a: Intentional Noncompliance


- Intention not to follow written YES
Error
regulations or company
procedures Go to Page 2

NO
Procedural Error

Go to Page 3 and 4
Crew Error Responses
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Error

Detected & Action


Detected & Ignored
Undetected

Error Induced
Error Outcome Incident or
Accident
Crew Error Responses Defined
Detected and Action (Trap or Exacerbate)
Error is detected
Crew did something to manage or mismanage the error

Detected and Ignored (Fail to respond)


Error is detected or willingly committed
Crew intentionally elected not to manage the error

Undetected (Fail to respond)


Error is undetected
No error management takes place
Error Outcomes
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Error

Error Management

Undesired Aircraft
State

Undesired Aircraft Error Induced


State Management Incident or
Accident
Undesired Aircraft States Defined
Undesired aircraft states A state in which the aircraft is placed in
a compromised situation that increases risk to safety
Lateral deviation Unstable approach
Vertical deviation Abrupt aircraft control
Speed too high Long landing
Speed too low Firm landing
Incorrect aircraft configurations Wrong taxiway / ramp
Flight controls
Wrong runway
Systems
Fuel Runway incursion
Automation
Undesired Aircraft State Responses
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Error

Crew Error
Responses

Undesired Aircraft
State

Detected & Action Error Induced


Detected & Ignore Incident or
Undetected Accident
Threat and Error Management
Threats

Threat Induced
Inconsequential Threat Management Incident or
Accident

Crew Error

Crew Error
Responses

Undesired Aircraft
State

Crew Undesired Error Induced


Aircraft State Incident or
Responses Accident
Error Management Top Three
1. Dont spend too much time on the codes it is not an
exact science

2. See more than five errors? staple another worksheet


and renumber

3. When in doubt, make sure you include answers the


following somewhere on the worksheet
1. What was the crew error?
2. How did the crew respond to the error?
3. What was the outcome of the error?
LOSA Observation Form
EXAMPLE

Observer Information
Observer ID (Employee number) 3059

Observation Number #1

Crew Observation Number


(e.g., 1 of 2 indicates segment one for a crew that you observed across two segments) 1 Of 1

Flight Demographics
City Pairs (e.g., PIT-CLT) PIT - LAX
A/C Type (e.g., 737-300) B-757
Pilot flying (Check one) CA FO X

Time from Pushback to Gate Arrival (Hours:Minutes) 4:55


Late Departure?
(Yes or No) Yes How late?
(Hours:Minutes)

Crew Demographics
CA FO SO/FE Relief 1 Relief 2
Base PIT PIT
Years experience for all airlines 35 5
Years in position for this A/C 7 1 month
Years in automated A/C
(FMC with VNAV and LNAV) 12 1 month

First LEG the crew has EVER flown together

Crew Familiarity First DAY the crew has EVER flown together
(Check one)

Crew has flown together before X

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 1
Predeparture / Taxi-Out
Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did the crew perform
Narrative when confronted with threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings.
The CA established a great team climate positive with open communication. However, he seemed to be in a rush
and not very detail oriented. The FO, who was relatively new to the A/C, tried to keep up but fell behind at
times. The CA did not help the cause by interrupting the FO with casual conversation (marginal workload
management).

All checklists were rushed and poorly executed. The CA was also lax verifying paperwork. This sub-par behavior
contributed to an undetected error - the FO failed to set his airspeed bugs for T/O (poor monitor/cross-
check). The Before Takeoff Checklist should have caught the error, but the crew unintentionally skipped over
that item. During the takeoff roll, the FO noticed the error and said, Missed that one.

The Captains brief was interactive but not very thorough (marginal SOP briefing). He failed to note the
closure of the final 2000 of their departing runway (28R) due to construction. Taxiways B7 and B8 at the end of
the runway were also out. The crew was marked poor in contingency management because there were no plans
in place on how to deal with this threat in the case of a rejected takeoff. Lucky it was a long runway.

1 2 3 4
Poor Marginal Good Outstanding
Observed performance had safety Observed performance was barely Observed performance was Observed performance was truly
implications adequate effective noteworthy

Planning Behavioral Markers Rating

2
The required briefing was interactive and - Concise, not rushed, and met SOP requirements
SOP BRIEFING
operationally thorough - Bottom lines were established

3
Operational plans and decisions were - Shared understanding about plans - Everybody on
PLANS STATED
communicated and acknowledged the same page

3
Roles and responsibilities were defined for - Workload assignments were communicated and
WORKLOAD ASSIGNMENT
normal and non-normal situations acknowledged

1
CONTINGENCY Crew members developed effective strategies to - Threats and their consequences were anticipated
MANAGEMENT manage threats to safety - Used all available resources to manage threats

Execution Behavioral Markers Rating

1
Crew members actively monitored and cross- - Aircraft position, settings, and crew actions were
MONITOR / CROSS-CHECK
checked systems and other crew members verified

2
WORKLOAD Operational tasks were prioritized and properly - Avoided task fixation
MANAGEMENT managed to handle primary flight duties - Did not allow work overload

3
Crew members remained alert of the
VIGILANCE - Crew members maintained situational awareness
environment and position of the aircraft
- Automation setup was briefed to other members
AUTOMATION Automation was properly managed to balance
- Effective recovery techniques from automation
MANAGEMENT situational and/or workload requirements
anomalies

Review / Modify Behavioral Markers Rating


Existing plans were reviewed and modified when - Crew decisions and actions were openly analyzed
EVALUATION OF PLANS
necessary to make sure the existing plan was the best plan

3
Crew members asked questions to investigate - Crew members not afraid to express a lack of
INQUIRY
and/or clarify current plans of action knowledge - Nothing taken for granted attitude
Crew members stated critical information and/or
ASSERTIVENESS - Crew members spoke up without hesitation
solutions with appropriate persistence

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 2
Takeoff / Climb
Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did the crew perform
Narrative when confronted with threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings.
Normal takeoff besides one error. As the crew started to clean up the aircraft, the FO called flaps up before
the flap retraction speed. The CA trapped the error and did not retract the flaps until the proper speed.

After passing 10000 all the way up to the TOC, the CA and FO failed to cross-verify multiple altitude changes.
There was no intention on part of the CA to verify. In addition, since it happened multiple times, the observer
coded it as an intentional noncompliance.

1 2 3 4
Poor Marginal Good Outstanding
Observed performance had safety Observed performance was barely Observed performance was Observed performance was truly
implications adequate effective noteworthy

Execution Behavioral Markers Rating

1
Crew members actively monitored and cross- - Aircraft position, settings, and crew actions were
MONITOR / CROSS-CHECK
checked systems and other crew members verified

3
WORKLOAD Operational tasks were prioritized and properly - Avoided task fixation
MANAGEMENT managed to handle primary flight duties - Did not allow work overload

2
Crew members remained alert of the
VIGILANCE - Crew members maintained situational awareness
environment and position of the aircraft
- Automation setup was briefed to other members
AUTOMATION Automation was properly managed to balance
- Effective recovery techniques from automation
MANAGEMENT situational and/or workload requirements
anomalies

Review / Modify Behavioral Markers Rating


Existing plans were reviewed and modified when - Crew decisions and actions were openly analyzed
EVALUATION OF PLANS
necessary to make sure the existing plan was the best plan
Crew members asked questions to investigate - Crew members not afraid to express a lack of
INQUIRY
and/or clarify current plans of action knowledge - Nothing taken for granted attitude
Crew members stated critical information and/or
ASSERTIVENESS - Crew members spoke up without hesitation
solutions with appropriate persistence

Cruise
Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did the crew perform
Narrative when confronted with threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings.
Routine no comments

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 3
Descent / Approach / Land Technical Worksheet

Descent (Above 10,000 ft.)


1 Was the approach briefed before the TOD? (Yes / No) Yes

2 Did the crew begin the descent before or at the FMS TOD? (Yes / No) Yes

Did the aircraft get significantly above/below the FMS or


3
standard path? (Yes / No) No
If Yes, explain in the narrative the cause and whether the
crew tried to regain the path.

Approach and Land (Below 10,000 ft.)


Instrument backup on Yes X
Visual X visual approach?
(Check One) No

4 Approach flown?
(Check one) Type of precision
Precision
approach

Type of nonprecision
Nonprecision
approach

5 Approach: Hand flown or Automation flown? Hand-flown

Did the aircraft get significantly above/below a desirable


6
descent path? (Yes / No) Yes
If Yes, explain in the narrative the cause and whether the
crew tried to regain the path.

Close to or at minimum maneuvering


speed
During flap extension, flaps
X
Close to or at the maximum flap
7 were generally extended: extension speed
(Check one)
Above maximum flap extension speed
(If this happens, be sure to describe in the narrative)

8 Weather (Check One) VMC X IMC

Stabilized Approach Parameters 1500 AFE 1000 AFE 500 AFE


Target airspeed between 5 and +15 Yes Yes Yes

Vertical speed 1000 fpm Yes Yes Yes


9
Engines spooled Yes Yes Yes
Landing configuration
Yes Yes Yes
(Final flaps / gear down)
On proper flight path (G/S and localizer) Yes Yes Yes

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 4
Descent / Approach / Land The Blue Box

Briefing TOD

Transition

4
Altitude

10000 ft.

Slow and configure

FAF / OM Stabilized Approach


Bottom Lines

Flare / Touchdown

Think blue box. Describe significant events from the TOD to landing using the picture above to define landmarks. Talk about
Narrative how the crew performed when confronted with threats and crew errors. Also, be sure to justify your behavioral ratings.

Briefing to TOD The CA and FO did a nice job with the approach brief, which was completed by the TOD. Much
better than their takeoff brief. They expected runway 25L from the Civet Arrival for a straight-in visual
approach. Jepp charts were out, contingencies talked about, and everything was by the book. The FO asked a
lot of questions and the CA was patient and helpful. Nicely done!

10000 to slowing and configuring ATC cleared the crew to 25L, but at 8000, ATC changed us to the Mitts
Arrival for runway 24R due to a slow moving A/C on 25L. The CA changed the arrival and approach in the FMC
and tuned the radios. As soon as everything was clean, ATC called back and told the crew they could either land
on 25L or 24R at their discretion. Since time was a factor, the crew discussed and decided to stick with the
approach into 24R. The crew was flexible and the CA did a nice job assigning workload. He directed the FO fly
the plane while he checked everything over one more time.

The crew was also better monitors and cross checkers. However, their execution of checklists was still a little
sloppy late and rushed.

The crew did a nice job staying vigilant with heavy traffic in the area used ATC and TCAS effectively.

Bottom lines to Flare / Touchdown The approach was stable, but the FO let the airplane slip left, which
resulted in landing left of centerline. Since the FO was new to this aircraft (1 month flying time), the observer
chalked it up to a lack of stick and rudder proficiency. .

Taxi-in The crew did a great job navigating taxiways and crossing the active 24L runway. Good vigilance and
teamwork.

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 5
Descent / Approach / Land
1 2 3 4
Poor Marginal Good Outstanding
Observed performance had safety Observed performance was barely Observed performance was Observed performance was truly
implications adequate effective noteworthy

Planning Behavioral Markers Rating

4
The required briefing was interactive and - Concise, not rushed, and met SOP requirements
SOP BRIEFING
operationally thorough - Bottom lines were established

4
Operational plans and decisions were - Shared understanding about plans - Everybody on
PLANS STATED
communicated and acknowledged the same page

4
Roles and responsibilities were defined for - Workload assignments were communicated and
WORKLOAD ASSIGNMENT
normal and non-normal situations acknowledged

3
CONTINGENCY Crew members developed effective strategies to - Threats and their consequences were anticipated
MANAGEMENT manage threats to safety - Used all available resources to manage threats

Execution Behavioral Markers Rating

2
Crew members actively monitored and cross- - Aircraft position, settings, and crew actions were
MONITOR / CROSS-CHECK
checked systems and other crew members verified

3
WORKLOAD Operational tasks were prioritized and properly - Avoided task fixation
MANAGEMENT managed to handle primary flight duties - Did not allow work overload

3
Crew members remained alert of the
VIGILANCE - Crew members maintained situational awareness
environment and position of the aircraft
- Automation setup was briefed to other members
3
AUTOMATION Automation was properly managed to balance
- Effective recovery techniques from automation
MANAGEMENT situational and/or workload requirements
anomalies

Review / Modify Behavioral Markers Rating

4
Existing plans were reviewed and modified when - Crew decisions and actions were openly analyzed
EVALUATION OF PLANS
necessary to make sure the existing plan was the best plan

3
Crew members asked questions to investigate - Crew members not afraid to express a lack of
INQUIRY
and/or clarify current plans of action knowledge - Nothing taken for granted attitude
Crew members stated critical information and/or
ASSERTIVENESS - Crew members spoke up without hesitation
solutions with appropriate persistence

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 6
Overall Flight

Narrative This narrative should include your overall impressions of the crew.

Overall, the crew did a marginal job with planning and review/modify plans during predeparture. However, during
the descent/approach/land phase, it was excellent. Their execution behaviors were marginal to good for the
entire flight.

While the takeoff brief was marginal, the CA made an outstanding approach brief. Open communication was not
a problem. Good flow of information when the flights complexity increased with the late runway change. They
really stepped it up.

The big knock against this crew involved checklists, cross verifications, and all monitoring in general. They were
a little too complacent during low workload periods (e.g., No altitude verifications during climb). The CA set a
poor example in this regard.

During predeparture, the CA introduced an unnecessary element of being rushed, which compromised workload
management. However, his decisiveness and coordination in the descent/approach/land phase kept his leadership
from being marked marginal.

1 2 3 4
Poor Marginal Good Outstanding
Observed performance had safety Observed performance was barely Observed performance was Observed performance was truly
implications adequate effective noteworthy

Overall Behavioral Markers Rating


4
COMMUNICATION Environment for open communication was - Good cross talk flow of information was fluid,
ENVIRONMENT established and maintained clear, and direct

3
Captain showed leadership and coordinated flight - In command, decisive, and encouraged crew
LEADERSHIP
deck activities participation

Yes Rating
Did you observe a flight attendant briefing on No
the first leg of the pairing? (Check one)
X
No opportunity to
observe

CA FO
Contribution to Crew Effectiveness 2 3

Rating

3
Overall Crew Effectiveness

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 7
Threat Management Worksheet
Threats Events or errors that originate outside the influence of the flightcrew but require
active crew management to maintain safety.
Threat Description Threat Management
Phase of
Threat ID

Flight
Effectively
Threat 1 Predepart/Taxi managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

Threat mismanaged CA failed to talk about the construction


Runway and taxiway construction on
T1 4 1 No and closures in his brief. No plans in place in the event of a
their departing runway (final 2000)
RTO.
Late ATC runway change - changed
Threat managed CA reprogrammed the FMC, handled the
T2 runway to 24R from 25L due to a slow 50 4 Yes radios, and placed emphasis on the FO to fly the aircraft.
moving aircraft on 25L
ATC called back and told the crew that Threat managed CA asked for the FOs preference. They
T3 it was at their discretion to land on 24R 50 4 Yes mutually decided to continue the approach into 24R because it
or 25L was already in the box.

Threat managed The crew closely monitored the traffic


T4 Heavy congestion going into LAX 3 4 Yes with the help of ATC and TCAS.

T __

Threat Codes
Departure / Arrival Threats Operational Threats Cabin Threats Crew Support Threats
1 Adverse weather / turbulence / IMC 30 Operational time pressure delays, 40 Cabin event / distraction / interruption 80 MX event
2 Terrain OTP, late arriving pilot or aircraft 41 Flight attendant error 81 MX error
3 Traffic Air or ground congestion, TCAS warnings 31 Missed approach 82 Ground handling event
4 Airport construction, signage, ground conditions 32 Flight diversion ATC Threats 83 Ground crew error
5 TCAS RA/TA 33 Unfamiliar airport 50 ATC command challenging clearances, late changes 84 Dispatch/ paperwork event
34 Other non-normal operation events 51 ATC error 85 Dispatch / paperwork error
Aircraft Threats max gross wt. T/O, rejected T/O 52 ATC language difficulty 86 Crew scheduling event
20 Aircraft malfunction 53 ATC non-standard phraseology 87 Manuals / charts incomplete /
21 Automation event or anomaly 54 ATC radio congestion incorrect
22 Communication event - radios, ATIS, ACARS 55 Similar call signs
99 Other Threats
The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 8
Error Management Worksheet
Error Description Error Response / Outcome
Phase of Error Type
Error Crew Error Error
Error ID

flight
1 Intentional Code Who Who Response Outcome
Describe the crew error and associated 1 Predepart/Taxi Noncompliance
committed detected
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use 1 Detected & Action 1 Inconsequential
3 Cruise 3 Communication Code the error? the error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E1 FO failed to set his airspeed bugs. 1 2 211 2 7 3 3

Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired State Response Aircraft State


Who
threat? Aircraft Outcome
How did the crew manage or mismanage the error? detected
State 1 Detected & Action
the state? 2 Detected & Ignored
(If Yes, enter Code 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E1 No Error chain to E2

Who Committed /
Undesired Aircraft State Codes
Detected Codes
Flightcrew Other people Configuration States Aircraft Handling States All Phases Approach / Landing States
1 CA 8 ATC 1 Incorrect A/C configuration - flight controls, brakes, 40 Vertical deviation 80 Deviation above G/S or FMS path
2 FO 9 Flight attendant thrust reversers, landing gear- 41 Lateral deviation 81 Deviation below G/S or FMS path
3 SO / FE 10 Dispatch 2 Incorrect A/C configuration systems (fuel,
4 Relief Officer 11 Ground electrical, hydraulics, pneumatics, air- 42 Unnecessary WX penetration 82 Unstable approach
5 Jumpseat Rider 12 MX conditioning, pressurization, instrumentation) 43 Unauthorized airspace penetration 83 Continued landing - unstable approach
3 Incorrect A/C configuration automation
6 All crew Aircraft 4 Incorrect A/C configuration - engines 44 Speed too high 84 Firm landing
members 20 Aircraft 45 Speed too low 85 Floated landing
systems Ground States 86 Landing off C/L
7 Nobody 20 Proceeding towards wrong runway 46 Abrupt aircraft control (attitude) 87 Long landing outside TDZ
21 Runway incursion 47 Excessive banking
99 Other 48 Operation outside A/C limitations
22 Proceeding towards wrong taxiway / ramp 99 Other Undesired States
23 Taxiway / ramp incursion
24 Wrong gate
The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 9
Error Management Worksheet
Error Description Error Response / Outcome
Phase of Error Type
Error Crew Error Error
Error ID

flight Who
1 Intentional Code Who Response Outcome
Describe the crew error and associated 1 Predepart/Taxi Noncompliance detected
committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
3 Cruise 3 Communication Code the error? 2 Detected & Ignored 2 Undesired state
error?
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi 5 Decision

In running the Before Takeoff Checklist,


E2 1 2 200 2 7 3 1
the FO skipped the takeoff data item.

FO called flaps up prior to the flap


E3 2 2 299 2 1 1 1
retraction speed.

Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

Errors mismanaged The bug error should have been caught with the
Before Takeoff Checklist, but the FO unintentionally skipped that
E2 No item. All checklists during this phase were poorly executed. The FO
caught the error during the takeoff roll.

Error managed CA saw that the aircraft was not at the proper
E3 No speed and waited to retract the flaps. Good monitoring in this case.

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 10
Error Management Worksheet
Error Description Error Response / Outcome
Phase of Error Type
Error Crew Error Error
Error ID

flight Who
1 Intentional Code Who Response Outcome
Describe the crew error and associated 1 Predepart/Taxi Noncompliance detected
committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
3 Cruise 3 Communication Code the error? 2 Detected & Ignored 2 Undesired state
error?
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi 5 Decision

CA and FO failed to verify multiple


E4 2 1 140 1 6 2 1
altitude changes.

FO, who was new to the aircraft, let it slip


a little to the left during the final
E5 4 4 402 2 6 1 2
approach. Resulted in landing left of the
centerline.

Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E4 No No error management intentional error

Error mismanaged FO tried to correct but still landed left of the


E5 No centerline. Approach was stable and they made the first high-speed 86 6 1 1
taxiway. The CA did not verbalize the deviation during the approach.

The University of Texas Human Factors Research Project - LOSA Observation Form Version 9.0 11
52

LOSA Crew Interview

1) Training
a) Is there a difference in how you were trained, and how things really go in line
operations?
b) If so, why?

2) Standardization
a) How standardized are other crews that you fly with?
b) If there is a lack of standardization, what do you think is the reason(s) for
procedural non-compliance?

3) Automation
a) What are the biggest automation gotchas for this airplane?

4) Overall safety improvements concerns and suggestions for improvement


a) Flight Ops
b) Dispatch
c) Airports and ATC
d) SOPs
LOSA Error Code Book 9.0

The University of Texas Human Factors Research Project

Klinect, J. R, Wilhelm, J. A., & Helmreich, R. L


Error Type Decision Tree

START
Was the error associated with a:
- Miscommunication YES
Communication Error
- Misinterpretation
- Failure to communicate Go to Page 5
pertinent information

NO

Was the error associated with a:


- Lack of knowledge YES Proficiency Error
- Lack of stick and rudder
proficiency Go to Page 5

NO

Was the error associated with a:


- Decision that increased risk in YES Operational Decision Error
which there were no written
procedures to follow Go to Page 6

NO

Was the error associated with a:


Intentional
- Intention not to follow written YES Noncompliance Error
regulations or company
procedures
Go to Page 2

NO
Procedural Error

Go to Page 3 and 4

University of Texas Human Factors Research Project Version 9.0 10/1//00 1


Intentional Noncompliance Error Codes
Sterile Cockpit Errors 142 Failure to cross-verify altimeter settings
100 Sterile cockpit violation
Hard Warning Errors
Callout Errors 160 Failure to respond to GPWS warnings
104 Omitted takeoff callouts (i.e., V-speeds) 161 Failure to respond to TCAS warnings
105 Omitted climb or descent callouts 162 Failure to respond to overspeed warning
106 Omitted approach callouts
Briefing Errors
Crew to ATC Errors 170 Omitted takeoff briefing
109 Altitude deviation without ATC clearance 171 Omitted approach briefing
110 Course or heading deviation without ATC clearance 172 Omitted flight attendant briefing (only for the first
(deviation more than 20 degrees) flight of a trip or crew change)
111 Use of nonstandard ATC phraseology 173 Omitted engine-out briefing
112 Omitted position report to ATC
113 Omitted non-radar environment report to ATC 179 Intentional failure to arm spoilers
114 Omitted call signs to ATC
Approach Errors
Checklist Errors 180 Failure to execute a go-around after passing
120 Checklist performed from memory procedural bottom lines of an unstable approach
121 Completed checklist not called complete 181 Speed deviation without ATC clearance
122 Checklist not performed to completion 183 Intentionally flying below the G/S
123 Use of nonstandard checklist protocol (i.e., use of 184 PF makes own flight control settings
nonstandard responses)
124 Omitted checklist
125 Self-performed checklist no challenge or response Automation and Instrument Setting Errors
126 Omitted abnormal checklist 185 PF makes own MCP changes
127 Self initiated checklist not called for by PF 186 PF makes own FMC changes
128 Self initiated checklist not called for by CA 187 Failure to set altitude alerter
129 Checklist performed late or at wrong time 189 Setting altimeters before the transition altitude
190 Using equipment placarded inoperative
Cross-Verification Errors
140 Failure to cross-verify MCP / altitude alerter Other Noncompliance Errors
changes 195 Taxi-in or out without a wing walker
196 A/C operation with unresolved MEL item
141 Failure to cross-verify FMC/CDU changes before 199 Other noncompliance errors not listed in the
execution code book

University of Texas Human Factors Research Project Version 9.0 10/1//00 2


Procedural Error Codes
Checklist Errors 245 Wrong MCP navigation select setting
200 Missed checklist item (NAV/GPS/ILS/VOR switch)
201 Wrong checklist performed 246 PF makes own MCP changes
202 Checklist performed late or at the wrong time 247 Wrong MCP setting on the auto-throttle switch
203 Forgot to call for checklist
206 Wrong response to a challenge on a checklist (i.e., item Flight Management Computer / Control Display Unit
not checked that was responded to as checked) Errors
207 Completed checklist not called complete 249 Failure to cross-verify FMC/CDU changes / position
209 Omitted checklist 250 Wrong waypoint / route settings entered into the FMC
233 Omitted abnormal checklist 251 Failure to execute a FMC mode when needed
252 Wrong mode executed in the FMC
Primary Instrument or Panel Errors 253 Wrong mode left engaged in the FMC
210 Wrong altimeter settings 254 Wrong present position entered into the FMC
211 Wrong bug settings (i.e., airspeed or altimeter) 255 Wrong weights / balance calcs entered into the FMC
212 Failure to set altitude alerter 256 Wrong speed setting entered into the FMC
213 Failure to cross-verify altimeter settings 257 PF makes own FMC changes
214 Failure to cross-verify altitude alerter 258 Wrong FMC format for input
205 Wrong approach selected in the FMC
Lever and Switch Errors 204 Other wrong CDU entries / settings
215 Failure to extend the flaps on schedule
216 Failure to retract the flaps on schedule 259 Wrong nav radio frequency
217 Wrong display switch setting
218 Failure to leave thrust reversers extended Radio Errors
219 Failure to lower the landing gear on schedule 260 Wrong ATIS frequency dialed
220 Failure to bring up the landing gear on schedule 261 Wrong ATC frequency dialed
221 Failure to extend the speed brakes on landing 262 Wrong squawk
222 Failure to retract the speed brakes
223 Failure to engage thrust reversers on landing Documentation Errors
224 Failure to retract thrust reversers after landing 263 Wrong ATIS information recorded
225 Failure to turn on the landing lights 264 Wrong runway information recorded
226 Wrong fuel switch setting 265 Wrong V-speeds recorded
227 Failure to turn on TCAS 266 Wrong weights and balance information recorded
228 Failure to turn on the fasten seat belt sign 267 Wrong fuel information recorded
229 Failure to arm spoilers 268 Missed items on the documentation (flight plan,
230 Failure to turn on the A/C packs (no pressurization) NOTAMS, or dispatch release)
231 Wrong panel setup for an engine start 269 Misinterpreted items on the documentation (flight
278 Wrong power settings for T/O plan, NOTAMS, or dispatch release)
279 Wrong autobrake setting 270 Wrong time calculated in the flight plan
232 Other incorrect switch or lever settings 271 Wrong clearance recorded

Mode Control Panel Errors


234 Failure to cross-verify MCP / altitude alerter changes Procedural Errors Continued
235 Wrong MCP altitude setting dialed
236 Wrong MCP vertical speed setting dialed on Next Page
237 Wrong MCP speed setting dialed
238 Wrong MCP course setting dialed
239 Wrong MCP heading setting dialed
240 Wrong setting on the MCP autopilot or FD switch
241 Wrong MCP mode executed
242 Wrong MCP mode left engaged
243 Manual control while a MCP mode is engaged
244 Failure to execute a MCP mode when needed

University of Texas Human Factors Research Project Version 9.0 10/1//00 3


Callout Errors Hard Warning Errors
275 Omitted takeoff callouts (i.e., V-speeds) 293 Failure to respond to GPWS warnings
276 Omitted climb or descent callouts 294 Failure to respond to TCAS warnings
277 Omitted approach callouts

Job Sequence Errors Briefing Errors


280 Executing the correct job procedures out of sequence 272 Incomplete flight attendant briefing
273 Incomplete cruise briefing
Handling Errors 274 Incomplete approach briefing
281 Unintentional lateral deviation 295 Omitted takeoff briefing
282 Unintentional vertical deviation 296 Omitted approach briefing
286 Unintentional speed deviation 297 Omitted flight attendant briefing
298 Omitted engine-out briefing
Ground Navigation Errors
283 Attempting or actually turning down the wrong runway Other Procedural Errors
284 Attempting or actually turning down the wrong ramp / 299 Other procedural errors not listed in the code
taxiway / gate book
287 Attempting or actually lining up for the incorrect runway
288 Attempting or actually lining up off C/L
289 Failure to execute a go-around after passing procedural
bottom lines of an unstable approach
290 Missed runway
291 Missed taxiway
292 Missed gate

University of Texas Human Factors Research Project Version 9.0 10/1//00 4


Communication Error Codes
Crew to ATC Errors 320 Wrong taxiway communicated
300 Wrong readbacks or callbacks to ATC 321 Wrong runway communicated
301 Missed ATC calls 322 Wrong takeoff callouts communicated
302 Omitted call signs to ATC 323 Wrong climb and descent callouts communicated
303 Failure to give readbacks or callbacks to ATC 324 Wrong approach callouts communicated
305 Omitted position report to ATC 325 Wrong gate assignment communicated
306 Omitted non-radar environment report to ATC 335 Crew miscommunication that lead to a
307 Misinterpretation of ATC instructions misinterpretation
309 Crew omitted ATC call 336 Wrong engine out procedures stated
310 Missed instruction to hold short
Other Communication Errors
350 Misinterpretation of ATIS
399 Other communication errors not listed in the
Crew to Crew Errors code book
319 Wrong airport communicated

Proficiency Error Codes


400 Lack of systems knowledge
401 Lack of automation knowledge
402 Lack of stick and rudder proficiency
403 Lack of knowledge to properly contact ATC
404 Lack of procedural knowledge
405 Lack of weather knowledge
406 Lack of knowledge of standard ATC phraseology
407 Lack of knowledge to contact company (i.e., gate assignments)

499 Other knowledge or proficiency based errors not listed in the code book

University of Texas Human Factors Research Project Version 9.0 10/1//00 5


Operational Decision Error Codes
Descent and Approach Errors
500 Failure to execute a go-around before reaching procedural bottom-lines
501 Unnecessary low maneuver on approach
502 Approach deviation (lateral or vertical) by choice
503 Decision to start the descent late
520 Operating at the edge of the performance envelope (no buffer for error)

Navigation Errors
510 Navigation through known bad weather that unnecessarily increased risk (i.e., thunderstorms or wind shear)
512 Decision to navigate to the wrong assigned altitude
513 Decision to navigate on the incorrect heading or course
514 Decision to navigate without ground clearance

521 Speed too high for operating environment

ATC Errors
530 Accepting instructions from ATC that unnecessarily increased risk
531 Making a request to ATC that unnecessarily increased risk
532 Failure to verify ATC instructions
533 Altitude deviation without ATC notification
534 Course or heading deviation without ATC clearance
535 Accepting a visual in nonvisual conditions

Crew Interaction Errors


540 Non-essential conversation at inappropriate times

Automation Errors
550 FMC over-reliance used at inappropriate times
551 FMC under-reliance not used when needed
552 Heads down FMC operation
553 Discretionary omission of FMC data (e.g., winds)

Instrument Errors
560 Lack of weather radar use

Checklist Errors
570 Failure to complete a checklist in a timely manner (i.e., after takeoff checklist)

Paperwork Errors
590 Failure to cross-verify documentation or paperwork

Other Operational Decision Errors


599 Other operational decision errors not listed in the code book

University of Texas Human Factors Research Project Version 9.0 10/1//00 6


Threat and Error Management Worksheet Codes

Threat Codes
Departure / Arrival Threats Operational Threats Cabin Threats Crew Support Threats
1 Adverse weather / turbulence / IMC 30 Operational time pressure delays, 40 Cabin event / distraction / interruption 80 MX event
2 Terrain OTP, late arriving pilot or aircraft 41 Flight attendant error 81 MX error
3 Traffic Air or ground congestion, TCAS warnings 31 Missed approach 82 Ground handling event
4 Airport construction, signage, ground conditions 32 Flight diversion ATC Threats 83 Ground crew error
5 TCAS RA/TA 33 Unfamiliar airport 50 ATC command challenging clearances, late changes 84 Dispatch/ paperwork event
34 Other non-normal operation events 51 ATC error 85 Dispatch / paperwork error
Aircraft Threats max gross wt. T/O, rejected T/O 52 ATC language difficulty 86 Crew scheduling event
20 Aircraft malfunction 53 ATC non-standard phraseology 87 Manuals / charts incomplete /
21 Automation event or anomaly 54 ATC radio congestion incorrect
22 Communication event - radios, ATIS, ACARS 55 Similar call signs
99 Other Threats

Who Committed /
Undesired Aircraft State Codes
Detected Codes
Flightcrew Other people Configuration States Aircraft Handling States All Phases Approach / Landing States
1 CA 8 ATC 1 Incorrect A/C configuration - flight controls, brakes, 40 Vertical deviation 80 Deviation above G/S or FMS path
2 FO 9 Flight attendant thrust reversers, landing gear- 41 Lateral deviation 81 Deviation below G/S or FMS path
3 SO / FE 10 Dispatch 2 Incorrect A/C configuration systems (fuel,
4 Relief Officer 11 Ground electrical, hydraulics, pneumatics, air-conditioning, 42 Unnecessary WX penetration 82 Unstable approach
5 Jumpseat Rider 12 MX pressurization, instrumentation) 43 Unauthorized airspace penetration 83 Continued landing - unstable approach
3 Incorrect A/C configuration automation
6 All crew Aircraft 4 Incorrect A/C configuration - engines 44 Speed too high 84 Firm landing
members 20 Aircraft 45 Speed too low 85 Floated landing
systems 86 Landing off C/L
7 Nobody
Ground States 46 Abrupt aircraft control (attitude) 87 Long landing outside TDZ
20 Proceeding towards wrong runway
47 Excessive banking
99 Other 21 Runway incursion
48 Operation outside A/C limitations
22 Proceeding towards wrong taxiway / ramp 99 Other Undesired States
23 Taxiway / ramp incursion
24 Wrong gate

University of Texas Human Factors Research Project Version 9.0 10/1//00 7


THREAT AND ERROR MANAGEMENT
EXERCISES
Threat and Error Management Exercises
Version 9.0

Klinect, J.R., Wilhelm, J.A., & Helmreich, R.L.

The University of Texas Human Factors Research Project1

March 7th, 2001

1 This research was supported by the Federal Aviation Administration Grant Number 99-G-004, Robert L. Helmreich, PhD, Principal Investigator.
Threat and Error Management Worksheet Codes

Threat Codes
Departure / Arrival Threats Operational Threats Cabin Threats Crew Support Threats
1 Adverse weather / turbulence / IMC 30 Operational time pressure delays, 40 Cabin event / distraction / interruption 80 MX event
2 Terrain OTP, late arriving pilot or aircraft 41 Flight attendant error 81 MX error
3 Traffic Air or ground congestion, TCAS warnings 31 Missed approach 82 Ground handling event
4 Airport construction, signage, ground conditions 32 Flight diversion ATC Threats 83 Ground crew error
5 TCAS RA/TA 33 Unfamiliar airport 50 ATC command challenging clearances, late changes 84 Dispatch/ paperwork event
34 Other non-normal operation events 51 ATC error 85 Dispatch / paperwork error
Aircraft Threats max gross wt. T/O, rejected T/O 52 ATC language difficulty 86 Crew scheduling event
20 Aircraft malfunctions 53 ATC non-standard phraseology 87 Manuals / charts incomplete /
21 Automation event or anomaly 54 ATC radio congestion incorrect
22 Communication event - radios, ATIS, ACARS 55 Similar call signs
99 Other Threats

Who Committed /
Undesired Aircraft State Codes
Detected Codes
Flightcrew Other people Configuration States Aircraft Handling States All Phases Approach / Landing States
1 CA 8 ATC 1 Incorrect A/C configuration - flight controls, brakes, 40 Vertical deviation 80 Deviation above G/S or FMS path
2 FO 9 Flight attendant thrust reversers, landing gear- 41 Lateral deviation 81 Deviation below G/S or FMS path
3 SO / FE 10 Dispatch 2 Incorrect A/C configuration systems (fuel,
4 Relief Officer 11 Ground electrical, hydraulics, pneumatics, air- 42 Unnecessary WX penetration 82 Unstable approach
5 Jumpseat Rider 12 MX conditioning, pressurization, instrumentation) 43 Unauthorized airspace penetration 83 Continued landing - unstable approach
3 Incorrect A/C configuration automation
6 All crew Aircraft 4 Incorrect A/C configuration - engines 44 Speed too high 84 Firm landing
members 20 Aircraft 45 Speed too low 85 Floated landing
systems Ground States 86 Landing off C/L
7 Nobody 20 Proceeding towards wrong runway 46 Abrupt aircraft control (attitude) 87 Long landing outside TDZ
21 Runway incursion 47 Excessive banking
99 Other 48 Operation outside A/C limitations
22 Proceeding towards wrong taxiway / ramp 99 Other Undesired States
23 Taxiway / ramp incursion
24 Wrong gate

The University of Texas Human Factors Research Project Version 9.0 2


Exercise One
Predeparture / Taxi-out The Captain requested an extra 3000 pounds of fuel for possible weather at the destination. After ground confirmed the fueling, the
First Officer alerted the Captain that they were 2000 pounds off. The Captain radioed back to ground and the remaining 2000 pounds was loaded.

Exercise Two
Predeparture / Taxi-out - While taxiing to the assigned runway, the First Officer performed the entire Taxi Checklist from memory. The Captain noticed and
ignored it. In the end, everything was set correctly.

Exercise Three
Takeoff / Climb - During climb and cleared to FL270, the First Officer heard the clearance but dialed 230 instead of 270 in the MCP. The Captain caught the
error on cross-verification.

Descent / Approach / Land During the approach, the crew had a hard time understanding ATC accents. At one point, the First Officer misunderstood a
controller and thought they had cleared them direct to the MAMAS waypoint when in actuality, they were only cleared to the closer OREGON waypoint. Before the
First Officer started to reprogram the FMC, the Captain stated he had heard something else. They called ATC back and verified the proper route.

Exercise Four
Descent / Approach / Land During a 30-degree bank on a visual approach, the First Officer allowed the aircraft to get 15 knots below minimum maneuvering
speed. The low speed had to be pointed out by the observer. The FO said thanks and immediately increased the speed. The observer noted that this was most
likely due to a lack of stick and rudder proficiency.

Exercise Five
Takeoff / Climb Before T/O, the Captain looked at the radar and noticed a red asymmetrical cell passing over the airport. In order to get to the destination
before the imposed curfew, the crew elected to takeoff when they could have waited a few minutes for it to pass over. The takeoff was a little rough but they got
through it with no significant problems. They arrived at the destination15 minutes early.
The University of Texas Human Factors Research Project Version 9.0 3
Exercise Six
Predeparture / Taxi-out and Takeoff During the after start flow, the FO forgot to turn on the packs to pressurize the aircraft. He would have caught it with the
After Start Checklist, but he skipped that item. At 8000 feet into climb, both pilots noticed that the plane was not pressurized. The First Officer promptly corrected
the error.

Exercise Seven
Takeoff / Climb During climb, the crew discovered that a fuel pump low pressure light was illuminated. All crewmembers were heads down working the problem
when ATC instructed them climb to FL350. The First Officer (PNF) read back FL350 but failed to make the change in the MCP. The crew also failed to point and
shoot the altitude change in the MCP. After a few moments, ATC noticed that the aircraft remained at FL270 and called the crew to correct the problem.

Exercise Eight
Descent / Approach / Land On the downwind leg of a visual approach at 6000ft., ATC asked them if they could make an immediate turn for the outer marker
and maintain 180 knots as they crossed over the marker. If they could, they would be number two for landing. They were already early, but the Captain accepted
the challenge without asking the First Officer (PF). The First Officer was noticeably reluctant at first, but he said nothing, made the close-in turn, and extended the
speed brakes to begin the approach. The FO was really pushed but he crossed over the marker on speed. At 1800, the Captain noticed that the speed brakes
were left extended. The First Officer corrected the problem and continued a stabilized approach.

Graduation Exercise
Descent / Approach / Land At 1500 feet during a precision approach in IMC weather, the Captain noticed he was more than a dot high on the glide slope. The
Captain corrected but he got a vertical speed call from the First Officer. At 1000, they had a sink rate of approximately 1500 fpm (Stabilized approach criteria
states vertical speed 1000 fpm at 1000 for a precision approach). The FO asked the Captain if he wanted to execute a go-around but he ignored him and
continued the approach. The result was an unstable approach with a long landing outside the touchdown zone.

The University of Texas Human Factors Research Project Version 9.0 4


Exercise One
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 5


Exercise Two
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 6


Exercise Three
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 7


Exercise Four
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 8


Exercise Five
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 9


Exercise Six
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 10


Exercise Seven
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 11


Exercise Eight
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Crew Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 12


Graduation Exercise
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __

E __

The University of Texas Human Factors Research Project Version 9.0 13


Threat and Error Management Worksheet Exercise Solutions

The University of Texas Human Factors Research Project Version 9.0 14


Solution: Exercise One
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

1 Threat managed Crew asked ground to load an


T1 WX at the destination Adverse 1 Yes
WX
extra 3000 lbs.
83
Incorrect fueling Threat managed FO caught the ground crew
T2 Ground 1 Yes
(2000 lbs. short) Crew error on a cross-check.
Error

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

E __ NO ERRORS
Crew Error Management Undesired Aircraft State
Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E __ NO ERRORS

The University of Texas Human Factors Research Project Version 9.0 15


Solution: Exercise Two
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __ NO THREATS

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

120
FO ran the Taxi Checklist from 2 1
E1 1 1 Checklist 2 1
memory from FO CA
memory

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

No management intentional noncompliance error was


E1 No
detected and ignored by the Captain.

E __

The University of Texas Human Factors Research Project Version 9.0 16


Solution: Exercise Three
Threat Description Threat Management
Phase of Flight
Threat ID

Effectively
Threat 1 Predepart/Taxi managed?
Describe the threat 2 Takeoff/Climb How did the crew manage or mismanage the threat?
Code 3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

Difficulty understanding ATC 52


T1 4 No Threat mismanaged Misinterpretation by FO.
accents ATC language

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Crew Error Error
Error ID

Flight
Error Code Who Response Outcome
1 Intentional Who
Describe the crew error and 1 Predepart/Taxi detected
Noncompliance committed
associated undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
Code Book the error?
3 Cruise 3 Communication error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

FO dialed an incorrect MCP 235 2 1


E1 2 2 Wrong MCP 1 1
altitude altitude dialed
FO CA

307 2 1
E2 FO misinterpreted a clearance 4 3 Misinterpret
FO CA
1 1
ATC instruct
Crew Error Management Undesired Aircraft State
Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E1 No Error managed CA caught the error on a cross-check.

Error managed Captain trapped the error by staying in


E2 Yes T1
the ATC communication loop good monitoring.

The University of Texas Human Factors Research Project Version 9.0 17


Solution: Exercise Four
Threat Description Threat Management
Phase of
Threat ID

Flight Effectively
Threat 1 Predepart/Taxi
managed?
Describe the threat How did the crew manage or mismanage the threat?
Code 2 Takeoff/Climb
3 Cruise (Yes / No)
4 Des/App/Land
5 Taxi-in

T __ NO THREATS

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Crew Error Error
Error ID

Flight Error Who Response Outcome


1 Intentional Code Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural the 1 Detected & Action 1 Inconsequential
Use the error?
3 Cruise 3 Communication Code Book error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

FO exhibited a lack of stick and 402


rudder proficiency Failed to Lack of 2 7
E1 4 4 stick and 3 2
maintain speed during a 30- rudder
FO Nobody

degree bank turn. proficiency

E __

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

No management Both the error and undesired aircraft 45 5


E1 No Jumpseat 3 1
state went undetected poor vigilance. Speed too low
Rider

E __

The University of Texas Human Factors Research Project Version 9.0 18


Solution: Exercise Five
Threat Description Threat Management
Threat ID

Phase of Effectively
Threat Flight managed?
Describe the threat How did the crew manage or mismanage the threat?
Code
1 Predepart/Taxi (Yes / No)
2 Takeoff/Climb

Time pressure did not want to 30


T1 Op 1 No Threat mismanaged Crew got in a hurry
miss the curfew in DCA Pressure

1 Threat mismanaged Crew unnecessarily flew


T2 Red asymmetrical weather cell Adverse 2 No
WX
through the adverse WX

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

510
Poor decision to fly through WX Decision
E1 when they could have waited a few 2 5 to fly 6 6 2 2
into All Crew All Crew
minutes for it to pass over. adverse
WX
Crew Error Management Undesired Aircraft State
Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

Error mismanaged Crew knew they made a risky decision. 42 6


E1 Yes T1 Penetration 2 1
They wanted to avoid further delay. adverse WX
All Crew

The University of Texas Human Factors Research Project Version 9.0 19


Solution: Exercise Six
Threat Description Threat Management
Threat ID

Phase of Effectively
Threat Flight managed?
Describe the threat How did the crew manage or mismanage the threat?
Code
1 Predepart/Taxi (Yes / No)
2 Takeoff/Climb

T __ NO THREATS

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Error Crew Error Error
Error ID

Flight
Code Who Response Outcome
1 Intentional Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural Use the 1 Detected & Action 1 Inconsequential
the error?
3 Cruise 3 Communication Code error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency Book 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

230
FO forgot to turn on the packs 2 7
E1 1 2 Fail to 3 3
during his flows turn on FO Nobody
packs

200 5
Crew missed the pack item on the 7
E2 1 2 Missed
All 3 2
Before Start Checklist checklist
crewmembers
Nobody
item
Crew Error Management Undesired Aircraft State
Associated Crew Undesired Undesired
Error ID

with a Who State Response Aircraft State


Undesired
threat? detected Outcome
How did the crew manage or mismanage the error? Aircraft
the 1 Detected & Action
State Code
(If Yes, enter state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E1 No Error Chain E2
2
Errors mismanaged Simple miss on the checklist 6
E2 No Incorrect A/C 1 1
became consequential. Resulted in an unpressurized A/C config - All Crew
pressurization

The University of Texas Human Factors Research Project Version 9.0 20


Solution: Exercise Seven
Threat Description Threat Management
Threat ID

Phase of Effectively
Threat Flight managed?
Describe the threat How did the crew manage or mismanage the threat?
Code
1 Predepart/Taxi (Yes / No)
2 Takeoff/Climb

20 Threat mismanaged Crew got distracted from


T1 Fuel pump low pressure light A/C 2 No
malfunction
their flight duties

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Crew Error Error
Error ID

Flight Error Who Response Outcome


1 Intentional Code Who
Describe the crew error and associated 1 Predepart/Taxi detected
Noncompliance committed
undesired aircraft states 2 Takeoff/Climb 2 Procedural the 1 Detected & Action 1 Inconsequential
Use the error?
3 Cruise 3 Communication Code Book error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

235 2 7
E1 Failed to dial MCP altitude 2 2 Wrong MCP
FO Nobody
3 3
altitude

Failure to perform altitude 234 6 7


E2 2 2 Failure to 3 2
awareness procedures cross-verify
All Crew Nobody

Crew Error Management Undesired Aircraft State


Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

E1 Yes T1 Error chain E2

Errors mismanaged Crew devoted their attention to the 40 8


E2 Yes T1 Vertical 3 1
fuel pump problem with no one left to fly the airplane deviation
ATC

The University of Texas Human Factors Research Project Version 9.0 21


Solution: Exercise Eight
Threat Description Threat Management
Threat ID

Phase of Effectively
Threat Flight managed?
Describe the threat How did the crew manage or mismanage the threat?
Code
4 Des/App/Land (Yes / No)

ATC challenged the crew to 50 Threat mismanaged CA accepted the clearance


T1 ATC 4 No
accept a tough-to-meet clearance command
without discussing it with the FO (PF)

Crew Error Description Crew Error Response / Outcome


Phase of Error Type
Crew Error Error
Error ID

Flight Error Who Response Outcome


1 Intentional Code Who
Describe the crew error and 1 Predepart/Taxi detected
Noncompliance committed
associated undesired aircraft states 2 Takeoff/Climb 2 Procedural the 1 Detected & Action 1 Inconsequential
Use the error?
3 Cruise 3 Communication Code Book error? 2 Detected & Ignored 2 Undesired state
4 Des/App/Land 4 Proficiency 3 Undetected 3 Additional error
5 Taxi-in 5 Decision

Decision to accept a risky ATC 530 1 2


E1 4 5 Accept risky 2 3
clearance ATC instruct
CA FO

222
FO left the speed brakes 2 1
E2 4 2 Failure to 1 1
extended at 1800 retract speed FO CA
brakes
Crew Error Management Undesired Aircraft State
Associated Crew Undesired Undesired
Error ID

with a Undesired Who State Response Aircraft State


threat? Aircraft detected Outcome
How did the crew manage or mismanage the error?
State the 1 Detected & Action
(If Yes, enter Code state? 2 Detected & Ignored 1 Inconsequential
Threat ID) 3 Undetected 2 Additional error

Error mismanaged CA accepted a clearance that


E1 Yes T1 unnecessarily pushed the FO. In response, the FO said
nothing and made the close-in turn
E2 Yes T1 Error managed CA caught the extended speed brakes

The University of Texas Human Factors Research Project Version 9.0 22


Solution: Graduation Exercise
Threat Description Threat Management
Threat ID

Phase of Effectively
Threat Flight managed?
Describe the threat How did the crew manage or mismanage the threat?
Code
4 Des/App/Land (Yes / No)

1 Threat mismanaged Partially contributed to the


T1 IMC WX Adverse 4 No
WX
unstable approach

Crew Error Description Crew Error Response / Outcome


Crew Error Error
Error ID

Phase of Error Type Error Who Response Outcome


Flight Code Who
Describe the crew error and detected
committed
associated undesired aircraft states 1 Intentional the 1 Detected & Action 1 Inconsequential
Noncompliance Use the error?
4 Des/App/Land Code Book error? 2 Detected & Ignored 2 Undesired state
2 Procedural 3 Undetected 3 Additional error

282
CA got more than a dot high on 1 1
E1 4 2 Unintentional 1 2
the glideslope vertical CA CA
deviation

180
Crew intentionally did not Failure to 6 6
E2 4 1 execute a 2 2
execute a go-around missed
All Crew All Crew
approach
Crew Error Management Undesired Aircraft State
Crew Undesired Undesired
Error ID

Undesired Who State Response Aircraft State


Associated
Aircraft detected Outcome
with a How did the crew manage or mismanage the error?
State the 1 Detected & Action
threat?
Code state? 2 Detected & Ignored 1 Inconsequential
3 Undetected 2 Additional error

Error mismanaged CA tried to correct the glideslope


82 6
E1 Yes T1 deviation at 1500 but ended up with a 1500 fpm sink rate Unstable
All Crew
1 2
at 1000. Unstable approach. approach

Error mismanaged Crew failed to execute a go-around 88 6


E2 Yes T1 1 1
and ended up landing long outside the TDZ. Long landing All Crew

The University of Texas Human Factors Research Project Version 9.0 23


THREAT AND ERROR MANAGEMENT
TRAINING
THREAT and ERROR MANAGEMENT TRAINING

Captain Don Gunther, Continental Airlines


Manager Human Factors
Captain Bruce Tesmer, Continental Airlines
Safety, Manager Crew Performance

ABSTRACT on the realization that all crew errors cannot, and will
not be avoided. Therefore, crews must use all means to
Line Operational Safety Audit (LOSA) data have three successfully resolve the error(s) to reduce or eliminate
major uses for research, for organizational safety the consequences.
initiatives, and for the development of training .
curricula1. The development and format of an Error T&EM

Management (EM) course is reviewed and how the ERROR MANAGEMENT


data from a follow on LOSA confirms the effect of the
training. From the follow-on LOSA, data is then used
to develop the new Threat & Error Management
(T&EM) course.

1996 LOSA

In 1996 Continental Airlines conducted a system-wide


Line Operation Safety Audit (LOSA). LOSA is a non- 3/4/01 HF Team 12
Human Factors Team3/4/01 Dec 2000
jeopardy observation of line crews during normal
operations. LOSA is designed to identify overt and
latent threats, crew errors, and aggregate crew The course introduced how SOPs, checklists, etc.
performance on how threats/errors are managed (use of (Resist) are important guards against possible errors
counter-measures). LOSA also identifies the links and how the use of CRM counter-measure skills
between crew errors, threats and other errors, as they (Resolve) can be practiced and improved to more
become the precursors to accidents/incidents. It is far effectively manage error. Additionally, all Check
more important to use LOSA data to affect system Airmen were trained in identifying crew error and to
changes than policing individual crews or pilots. Based reward effective error management when it was
on the 1996 LOSA results, Continental focused on present and not just focus on a crews lack of success
crew error and the countermeasures necessary to avoid, in trying to avoid all error. The CRM markers and
trap and mitigate errors before they became error management were also imbedded in training
consequential. Working with the University of Texas courses where crews are held accountable not only for
Human Factors Research Project Team, Continental their technical skills but also their skills at using CRM
developed an Error Management course that differed markers as effective error counter-measures.
from previous CRM programs.
The operations safety change process that began with
Error Management LOSA 1996 was about to complete a full cycle: 1) use
LOSA to collect error data in line operations, 2). Flight
Starting in 1997, all Continental pilots began attending Standards & Training use LOSA analysis to make
a new CRM course named Error Management. changes, 3). Line pilots receive training on the changes,
Pilots gained an understanding of human (pilot) error, 4). Conduct a follow-on LOSA to measure the
error avoidance, error-trapping and mitigating the effectiveness of the changes. It has always been
consequence of error. A major focus of the course was difficult to collect hard data on the value of CRM
courses and their effect on crew performance.
However, the system-wide LOSA 2000, conducted
1 after completion of the Error Management Course,
System Safety and Threat and Error Management: The
Line Operational Safety Audit (LOSA). provided measurement against the base line data
11th International Symposium on Aviation Psychology collected in LOSA 1996.
March 7, 2001 Columbus, OH
Robert L. Helmreich, James R. Klinect, John A. Wilhelm & J.
Bryan Sexton
successfully and reduce crew errors associated with
LOSA 2000 these threats.

The Line Operational Safety Audit (LOSA) 2000, Threat & Error T&EM

when compared to the 1996 LOSA, showed that the Management


CAL pilots not only accepted the principals of Error MAINTANCE PROBLEM
WEATHER ALTERNATE
LOW VISIBILITY TAXI
DE - ICING

Management but had incorporated them into their


everyday operation on the line. LOSA 2000 showed a WRONG TURN

sizeable improvement in the areas of checklist usage, a ALTITUDE DEVIATION

70% reduction in non-conforming (not meeting


stabilized approach criteria) approaches and an
increase in overall crew performance. While
conducting LOSA 2000 observations, the observers
saw another area of crew performance that needed to 3/8/01 HF Team 36
Dec 2000
be addressed in addition to Error Management. The Human Factors Team3/8/01

new area of crew performance concerned threat


recognition and threat management. The LOSA 2000 Threats = Red Flags: In reviewing several accidents
data and analysis concerning threats led to the and incidents over the last several years, it became
development of the current 2001 Threat and Error apparent that where there were several threats not
Management CRM course. properly managed. There were also crew errors that
together, played a significant role in the mishap.
Threat & Error Management Threats must be identified and assessed as Red Flag
warnings. When crews successfully recognize and
Based on the premise that crew errors occur on normal acknowledge threats as red flags, they are in a better
daily flights, a question was formed from pilot position to manage that threat so it becomes
discussions during the Error Management training. The inconsequential. Accident/incident crews typically do
question is: what defines a normal flight? The not recognize all the threats, or their severity, and
answer decided upon, defined a normal flight as one accumulate red flags, which invites crew error. Crews
having no threats. This would be considered a pristine are most vulnerable when they acquire several threats,
flight, requiring no crew effort to change anything from (red flags) and have employed no strategies to manage
the plan, through the execution, of flying from them. Accumulation of red flags places the crew at the
departure to destination. While this is unusual, it does edge or corner of the operating envelope where time
happen. That being the case, the LOSA 2000 team and options are limited. To effectively manage threats,
defined a threat as anything that requires a they must be identified, then assessed, and then
crewmembers time/attention/action above and beyond countered. Identification of threats comes through
the tasks of a pristine flight. LOSA 2000 observers many system alerting methods i.e. aircraft system
were trained to observe the threats crews faced and alerting lights, bells, horns, voice, and devices such as
how they were managed. These were external threats GPWS, TCAS, winds hear etc. Those devices may also
(weather, maintenance, passenger problems, show the best course of action to counter the threat.
operational pressures, distractions/interruptions, ATC However, there are not systems and devices to detect,
errors /language/communications problems, etc.) that assess, and counter all threats. Effective use of CRM
were not crew errors but came from external sources counter-measures provides primary threat detection and
and increased the potential for error, if not managed management for threats not having system alerts, and
properly. What surfaced during these observations provides backup for those that do. Abundant examples
were the strategies good pilots use to effectively exist showing that failure to employ CRM counter-
manage threats. These strategies, pro-active in nature, measures was the last option for detecting a threat that
were sometimes personal techniques that pilots have ended in a CFIT accident with the GPWS warning
developed over time in order to effectively operate in sounding for 30 seconds or more with no corrective
todays complex environment. Other strategies that are action taken by the crew .One of the primary counter-
common and used routinely have developed into measures to managing threats effectively is Flight Deck
procedures (SOPs). An example would be the use of a Leadership. The attributes of Flight Deck Leadership
pre-departure briefing to review a power loss on take- (setting the example, planning ahead, initiative,
off. This briefing prepares the crew to make a better fostering communications, etc.) are the foundation for
decision, should a loss of power occur when risk is effective strategies to manage the threats crews face
high and decision time is minimal. Crews that every day.
effectively use strategies manage external threats
. Effective Red Flag (threat) management reduces
the complexity of the operating environment,
decreasing the potential of crew error. Strategies,
whether personal or SOP, need to be employed
consistently so threats can be more easily recognized
and managed. Interactive communications, vigilance,
monitoring & challenging are full time strategies the
assist crews in identifying Red Flags. These
countermeasures arent developed at the time of the
event but are developed and deployed pro-actively. In
academic terms it means, Get your stuff together
before the stuff hits the fan, or words to that effect.
The environment we operate in has only become more
complex over the last few years and will continue to
present our pilots with an increased number of
challenges. As a pilot group we must raise the bar
and accept that outstanding performance today will
only be standard in the future. We must improve
ourselves to reach that new above standard
performance level. Threat & Error Management
training is one of the means pilots can use to reach an
improved level of performance that will enable them to
deal with the increased challenges of maintaining a
safe operation. The idea behind all CRM courses is to
define the best practices in applying threat and error
management counter-measures to reduce or eliminate
the consequence of threats and errors, which are the
precursors of accidents and incidents. Safer operations
can be had by imbedding the best practices of our
pilots into our training and everyday operation. Pilots
learn many of their positive traits from the sharing of
ideas and experiences with their peers and then
applying them to their own operational philosophy.
The Threat & Error Management course is a means
by which the pilots can prepare for the future and be
part of the team successfully building a threat and error
management culture.
IMPLEMENTATION STRATEGY SESSION
115

Conclusions experience, and taking into consideration the


typical size, fleet composition and route
In the last session of the Second LOSA Week, structure of Latin American airlines
participants engaged in an open forum discussion (approximately 40 aircraft and 500 pilots), it
with the following objectives: was estimated that about 100 observation
segments would be necessary to ensure
To discuss possibilities, constraints and statistical validity. Based upon this, the figure
resources required for the implementation of of (US) $50 000/60 000 was provided. It is
LOSA at the airline-specific level; exceedingly important to introduce the strong
caveat that such figure was a rough estimate,
C To discuss the potential cooperation among and may substantially differ in specific airline
airlines within the Ibero-American community cases. The case of a mini-LOSA was
in sharing resources and technology as needed discussed. In such case, costs would
to implement LOSA; significantly vary from the figure provided.

C To discuss the potential integration of an 3. An issue specific to the region was discussed
Ibero-American LOSA steering committee; at large. Some airlines in Latin America are
and relatively small, in some cases less than 10
aircraft and 100 pilots. In such cases,
C To discuss the potential contribution of non- maintaining confidentiality and objectivity of
airline organizations, including civil aviation the observations may be difficult if not
authorities, aircraft manufacturers and impossible. As solution to this problem, the
professional associations, to the worldwide possibility of inter-carrier cooperation was
implementation of LOSA. considered. Smaller airlines could pool
resources, and observers from airlines other
that the one being observed could conduct the
The main points debated and conclusions actual observation. The exchange of observers
arrived to thereafter follow. de-personalizes and thus removes subjectivity
from the observations. The data could be then
1. Questions about language and culture were sent to the central data processing centre at
raised early in the discussions. LOSA is the University of Texas. In this way, by
essentially a data acquisition tool, and some pooling observations which would only be
participants felt that, within Latin contexts, identifiable by the University of Texas, a
the means to obtain data and, most important, volume of data large enough could be
the management of the information such data obtained, while ensuring confidentiality.
generates, may substantially differ from the
way data is acquired and information 4. In line with the perceived need to pool
managed in Anglo-Saxon contexts. Extreme resources to overcome financial constraints,
perceptions were expressed, including a view the contribution of regional associations, such
that Latin systems are designed in such a way as AITAL, was considered. However, in
that they can only function by violating the regard to LOSA, the Pan American Aviation
norms. On the other hand, other participants Safety Team (PAAST) was considered a more
felt that rather than allowing culture to stand appropriate vehicle. One of the mandates of
in the way of LOSA, culture should be used to PAAST is to identify and facilitate the
facilitate LOSA, by leveraging upon the implementation of safety tools in Latin
strengths of Latin culture while trying to America. Thus, supporting and facilitating the
neutralise potential weaknesses regarding implementation of LOSA within Latin
information management. America would perfectly fit within this
mandate. Nevertheless, regardless of the
2. The costs of implementing LOSA were participation of regional associations, there
discussed. Based upon existing industry was consensus within the group that a joint
116
effort by a pool of airlines within the region, but the methodology to implement it, LOSA,
with proper initial support, could provide all is not equally so. This, however, is not the
the resources necessary to deploy and conduct case with TREM. Participants felt that TREM
LOSA, with the exception of data should immediately be initiated within the
management. It was felt that data management Ibero-American airline community; and that
should be conducted by the University of the right way to proceed is to plant the seed of
Texas. While the idea of engaging local Latin LOSA though TREM. Therefore, it was
American universities in the LOSA project agreed that the organization of a TREM
was briefly entertained, it was discarded since training the trainers workshop in the region
no participant could identify any university in should be an immediate priority. In this way,
Latin America that conducts Human Factors airlines could count on a core group of TREM
research. course designers/facilitators to move forward
in the implementation of TREM and thus
5. The contribution of the Colegio Profesional pave the way to LOSA.
de Aviadores of Spain was considered.
Representatives of Spanish airlines strongly 10. In conclusion, the following points summarise
felt that LOSA could first be introduced the discussion as well as immediate avenues
within the Spanish context through the of action:
participation of the Colegio.
C There was agreement as to the
6. The participation of pilots professional relevance of the concept of normal
associations was considered essential. operations monitoring to aviation
Without unions support, LOSA was simply safety;
considered not viable.
C LOSA could be applied within the
7. It was considered very important that the Latin American context, although its
concept of normal operations monitoring methodology might need some
becomes an ICAO standard. However, many specific adaptation;
participants felt that it was equally important
to initiate and sustain an aggressive campaign, C The Colegio Profesional de Pilotos
within each airline and country in Latin Aviadores seems to be the most
America, to lobby for the concept. Such appropriate organization to launch
campaign should be a personal endeavour by LOSA in Spain;
all those individuals who believe in the
potential of LOSA as a safety tool. C TREM is immediately applicable
within the Ibero-American airline
8. Participants expressed regret that, community and is considered the
unfortunately, very few civil aviation vehicle to introduce LOSA in Latin
authorities from the region attended the America; and
meeting. The few civil aviation authorities
present at the meeting endorsed LOSA C ICAO should organize, in a very near
without reservations. future and as a matter of urgent
priority, a training the trainers
9. The discussion then moved on towards Threat workshop for Latin American airlines
and Error Management Training (TREM), in order to qualify TREM course
presented as a re-conceptualization of CRM. designers and facilitators.
There was absolute consensus of opinion that
TREM is the vehicle to introduce LOSA
within the Ibero-American airline community.
The line of reasoning followed was that the _______________
concept of normal operations monitoring is
valid and acceptable across the community,
LIST OF PARTICIPANTS
SURNAME GIVEN NAMES AIRLINE COMPANY ADDRESS
Anca Jose Ansett Ansett Australia, Building 186, Cnr, Service & Grants Rds. Melbourne Airport 3045, Australia joey_anca@ansett.com.au
Antoni Eugene Callistus SIA Singapore Changi Airport, Singapore 819643 SIN-T2-01A geneam@pacific.net.sg
Ballantyne Thomas Orient Aviation Orient Aviation, P.O Box 2109, Carlingford Court, Carlingford NSW 2118, Australia tomball@ozemail.com.au
Bent John GECAT GE Capital Aviation Training (HK) 6/F, Flight Training Centre, Cathay Pacific City, 10 Scenic Road, Hong Kong john.bent@gecat.com
Burdekin Susan Gay ADF Sue Burdenkin Bsc CPL, 25 Goldner Circuit, Melba ACT 2615, Australia rob.sue@dynamite.com.au
Chou Jyh Shyan China Airlines China Airlines, No.3 Alley 123, Lane 405, Tung Hwa N.Rd. Taipei, Taiwan, R.O.C. jyh-hsyan_chou@email.china-airlines.com
Duan Frank (Weiping) China Northen Airlines China Northern Airlines, 3-1 Xiao He Yan Rd. Sheyang 110043 China frankduan_99@sina.com
Edkins Graham Derek Qantas Qantas Airways Ltd, Flight Safety Department MA9 Qantas Jet Base, 203 Coward St. Mascot 2020 Australia gedkins@qantas.com.au
Evers Owen James China Airlines China Airlines, No.3 Alley 123, Lane 405, Tung Hwa N.Rd. Taipei, Taiwan, R.O.C. o-james_evers@email.china-airlines.com
Fabresse Anne Airbus Airbus Industrie B28 Building, 5 Rue Gabriel Clerc, BP33, S1707 Blagnac Cedex, France
Ghandour Malik Nassib China Airlines China Airlines - San Francisco International Airport mghandour@earthlink.net
Greenfield Peter William Emirates Emirates, Operations Centre (FC 256) P.O. Box 92, Dubai - U.A.E. peter.greenfield@emirates.com
Greeves Brian IFALPA IFALPA, M10 Scenic Villas, 26 Scenic Villa Drive, Pok Fu Lam, Hong Kong bgreeve@attglobal.net
Guang Ren Su Xiamen Xiamen Airlines, 22 Dailiao Road, Xiamen, China
Gunther Donald L. Continental Continental Airlines frigate1@ix.netcom.com
Hankins Frank Moris Boeing Boeing China Inc. Tower A, 16/F, Pacific Century Place No.2A, Worker's Stadium Rd North Chaoyang District, Beijing, 10027 frank.m.hankins@boeing.com
Harris Dayle Miguel EVA EVA Air, 15th Floor, Flight Safety Dept., c/o Danny Ho, 376 Hsin-nan Rd., Section 1, Luchu, Taoyuan Hsien, Taiwan dale.harris@attglodoal.net
Hawke Mike Qantas Qantas Airways Ltd, Qantas Centre Building C/3, 203 Coward Street, Mascot NSW 2020 Australia mjhawke@optushome.com.au
Helmreich Robert U of T The University of Texas Crew Research Project, 1609 Shoal Creek Blvd., Austin, TX78701-1022, USA helmreich@psy.utexas.edu
Henderson Simon Thornton Ansett Ansett Australia, Building 186, Cnr, Service & Grants Rds. Melbourne Airport 3045, Australia henders@melbpc.org.au
Hori Hiroto ANA ANA, 3-3-2, Haneda Airport Ota-ku, Tokyo 144-0041, Japan
Hsu Chung-cheng Steven EVA EVA Air, 15th Floor, Flight Safety Dept., c/o Danny Ho, 376 Hsin-nan Rd., Section 1, Luchu, Taoyuan Hsien, Taiwan Stevenhsu@mail.evaair.com.tw
Jennings Herschel Todd EVA EVA Air, 15th Floor, Flight Safety Dept., c/o Danny Ho, 376 Hsin-nan Rd., Section 1, Luchu, Taoyuan Hsien, Taiwan todd@jennings.net
Jeremica Vernon Benedict Boeing The Boeing Company, P.O. Box 3707 MC14-HA, Seattle, WA 98124-2207 vernon.b.jeremica@boeing.com
Jian Li China Northen Airlines China Northen Airlines, DongTa Airport, Shenyang China
Jingchen Tang Xiamen Xiamen Airlines, 22 Dailiao Road, Xiamen, China
Johnson Robert Boeing Boeing China Inc. Tower A, 16/F, Pacific Century Place, No.2A, Worker's Stadium Road, North Chaoyang District, Beijing, 100027 robert.e.johnson8@boeing.com
Keith Leroy Allen AAPA AAPA, 9th Flor, Kompleks Antarabangsa Jalan Sultan Ismail 50250 Kuala Lumpur, Malaysia lakeith@aapa.org.my
Klinect James U of T The University of Texas Crew Research Project, 1609 Shoal Creek Blvd., Austin, TX78701-1022, USA klinect@mail.utexas.edu
Kriechbaum Christopher Hugh Air NZ Air New Zealand Ltd, Private Bag 92007, Aukland 1, new Zealand chris.kriechbaum@airnz.co.nz
Kuei Cherng China Airlines China Airlines, Taipei C.K.S. International Airport kuei744@m4.is.net.tw
Lima Frederico EVA EVA Air, 15th Floor, Flight Safety Dept., c/o Danny Ho, 376 Hsin-nan Rd., Section 1, Luchu, Taoyuan Hsien, Taiwan fredericolima@attglobal.net
Markson Greg Cathay Pacific Rm. 5.1, FTC, Cathay Pacific City, 8 Scenic Rd., Lantau, Hong Kong greg_markson@cathaypacific.com
Maurino Daniel ICAO 999 University St., montreal, Quebec H3C 5H7, Canada Cmaurino@icao.int
Morii Tamaki JAL Japan Airlines, West Passenger Terminal 3-2, Haneda Airport 3 chome, Ota-ku, Tokyo 144-0041 tamaki.morri@jal.co.jp
Peacock Gerard Rene SIA Singapore Airlines Ltd, SIA Training Centre Flt. Ops Div., Flt. Crew Training, SIN STC 03-C, Airmail Transit Centre. PO BOX 501 Singapore 918101 gerard_peacock@singaporeair.com.sg
Quay Chew Eng SIA Singapore Airlines Ltd, SIA Training Centre Flt. Ops Div., Flt. Crew Training, SIN STC 03-C, Airmail Transit Centre. PO BOX 501 Singapore 918101 CE_Quay@singaporeair.com.sg
Radzi Azmi Bin Mohd Malaysian Malaysian Airlines arhfam@tm.net.my
Rodgers Michael CASA CASA, Baillieu House, 71 Northbourne Ave PO Box 2005, Canberra ACT 2601 rodgers_m@casa.gov.au
Snelgar Robin Anthony SIA Singapore Airlines Ltd, SIA Training Centre Flt. Ops Div., Flt. Crew Training, SIN STC 03-C, Airmail Transit Centre. PO BOX 501 Singapore 918101
Soh Tommy Tian Seng SIA Singapore Airlines Ltd, SIA Training Centre Flt. Ops Div., Flt. Crew Training, SIN STC 03-C, Airmail Transit Centre. PO BOX 501 Singapore 918101 tommysoh@magix.com.sg
Liangtian REN Xiamen Xiamen Airlines, 22 Dailiao Road, Xiamen, China
Sydiongco Jim EVA EVA Air, 15th Floor, Flight Safety Dept., c/o Danny Ho, 376 Hsin-nan Rd., Section 1, Luchu, Taoyuan Hsien, Taiwan
Tai Chi Hung Mandarin Mandarin Airlines, Shun Shan Airport, Taipei, Taiwan, R.O.C.
Tan Charlie SIA Singapore Airlines Ltd, SIA Training Centre Flt. Ops Div., Flt. Crew Training, SIN STC 03-C, Airmail Transit Centre. PO BOX 501 Singapore 918101 charlie_tan@singaporeair.com.sg
Tesmer Bruce Continental Continental Airlines Btesme@coair.com
Topfer Ian Qantas Qantas Airways Ltd, Qantas Centre Building C/2, 203 Coward Street, Mascot 2020 Australia itopfer@qantas.com.au
Tsay Chiou Yueh China Airlines China Airlines, No.3 Alley 123, Lane 405, Tung Hwa N.Rd. Taipei, Taiwan, R.O.C. chiou-yueh_tsay@email.china-airlines.com
Xiaogang Peng Xiamen Xiamen Airlines, 22 Dailiao Rd., Xiamen, China
Yeh William EVA EVA Air, 15th Floor, Flight Safety Dept., c/o Danny Ho, 376 Hsin-nan Rd., Section 1, Luchu, Taoyuan Hsien, Taiwan
Ying Yuan (Ms.) China Eastern Airlines China Eastern Airlines, Shanxi Branch, Flight Operations Dept.,Tai Yu Rd. 32 030031, Taiyuan Shanxi, China
Zheng Han China Eastern Airlines China Eastern Airlines, Shanxi Branch, Flight Operations Dept.,Tai Yu Rd. 32 030031, Taiyuan Shanxi, China
PROCEEDINGS OF THE FIRST ICAO-IATA
LOSA & TEM CONFERENCE

DUBLIN
IRELAND
5 TO 7 NOVEMBER 2003
TABLE OF CONTENTS

Agenda ................................................................................................................................... (ii)

The Line Operations Safety Audit (LOSA) ................................................................................. 1

Presentations

Improving Safety in an Ultra Safe System


$ Monitoring Normal Operations: the Perspective of ICAO ........................................... 11
$ Collecting Safety Data from Normal Operations: the Perspective of IATA ................ 13

The Safety Change Process


$ Managing Threat and Errors in Airline Operations ..................................................... 17
$ The Ten Operating Characteristics of Line Operations Safety Audits ........................ 25
$ The Safety Change Process Following Line Operations Safety Audits .................... 26

Collecting Safety Data from Normal Operations


$ Cathay Pacific Line Operations Safety Audit: Training for Safety .............................. 32
$ Air New Zealand: The experience of a Arepeat@ LOSA ................................................... 35
$ The Braathens experience ........................................................................................ 38
$ The Futura experience .............................................................................................. 43
$ Collecting Safety Data from in Service Occurrences and Learning from them .......... 47
$ Boeing Safety Approach: The Value of Collaboration ................................................ 54
$ Learning about automation from Line Operations Safety Audits ................................ 60

Workshops

Training Instructors on Threat and Error Management (TEM) ................................................. 75

Training Flight Crew on Threat and Error Management (TEM)


$ TEM General .............................................................................................................. 84
$ TEM: Improving Flight Crew Monitoring and Cross-Checking .................................. 92
$ TEM: Automation and Security .................................................................................. 99

List of participants .................................................................................................................. 115


Agenda

Tuesday, 04 November

1900 2100 Registration, collection of badges and reception

Wednesday, 05 November

0800 0830 Late registration and collection of badges

Opening & welcome

0830 Conference Chairman


Captain Denis Coughlan
Director of Training
Aer Lingus

Opening remarks

0835 0855 Mr. Eamonn Brennan, Chief Executive of the Irish Aviation Authority
Mr. Willie Welsh, Chief Executive - Aer Lingus
Ms. Jill Sladen, Manager Safety - IATA
Capt. Dan Maurino, Coordinator, Flight Safety and Human Factors - ICAO

Improving Safety in an Ultra Safe System

0900 0920 Monitoring Normal Operations: the Perspective of ICAO


Capt. Dan Maurino - ICAO

0920 - 0940 Collecting Safety Data from Normal Operations: the Perspective of IATA
Ms. Jill Sladen - IATA

The Safety Change Process

0940 1020 Managing Threat and Errors in Airline Operations


Prof. Robert Helmreich - UT

1020 1050 Coffee-break

1050 - 1120 The Ten Operating Characteristics of Line Operations Safety Audits
Mr. James Klinect UT/TLC

1120- 1150 The Safety Change Process Following Line Operations Safety Audits
Capt. Don Gunther - Continental
Collecting Safety Data from Normal Operations

1150 1220 Cathay Pacific Line Operations Safety Audit: Training for Safety
Capt. Henry Craig Cathay Pacific

1220 1235 Q&A session

1235 - 1345 Lunch

1345 1415 Air New Zealand: the experience of a repeat LOSA


Capt. Chris Kriechbaum Air New Zealand

1415 1445 The Braathens experience


Capt. Morten Ydalus Braathens

1445 - 1515 The Futura experience


Capt. Robert Aran - Futura

1515 1545 Coffee-break

1545 1615 Collecting Safety Data from In Service Occurences and Learning from Them
Mr. Jean-Jacques Speyer - Airbus

1615 1645 Boeing Safety Approach: The Value of Collaboration


Dr. Curt Graeber Boeing

1645 - 1715 Learning about automation from Line Operations Safety Audits
Capt. Carlos Arroyo Landero - IFALPA

1715 1730 Q&A session

Thursday, 06 November

Workshops

Training Instructors on Threat and Error Management (TEM)

0900 - 1215 Mr. James Klinect & Capt. Pat Murray - UT/TLC

1215 1330 Lunch break


Training Flight Crew on Threat and Error Management (TEM)

1330 1700 WORKSHOP

TEM: General
Mike Bombala - IATA

TEM: Improving flight crew monitoring and cross-checking


Capt. Robert Sumwalt US Airways/ALPA

Friday, 07 November

0900 1230 TEM: Automation and Security


Capt. Don Gunther - Continental

1230 Conference conclusions and wrap up


Ten Years of Change Crew Resource Management 1989-1999

Robert L. Helmreich1
University of Texas Team Research Project
The University of Texas at Austin

Since the end of the 1980s Crew The definition of CRM is


Resource Management (CRM) has grown misunderstood. Although CRM programmes
globally, has come under some attack, and is were clearly rooted in efforts to reduce pilot
finding its place as an essential component of a error accidents, over the years understanding of
safety culture by focusing on error and threat the goals of programmes has faded, perhaps in
management. CRM training is now mandated for part because of the extension of the training to
pilots of all member states of ICAO and airlines flight attendants and other personnel. An
in member states of the European Union will be indication of this misunderstanding can been
required to evaluate individual proficiency in seen in the work of scientists in the United States
CRM-related behaviours. CRM training is also who recently defined CRM as Instructional
being extended to other components of the strategies that seek to improve teamwork in the
aviation system, including flight attendants, cockpit. While effective teamwork is clearly
flight dispatchers, and maintenance personnel. In important, it is not the primary goal of CRM
terms of its application, CRM is certainly a training. The following is a more accurate
success story. Considerable evidence has also representation of the latest, 1999 model CRM
been accumulated to show the effectiveness of programmes: CRM is a subset of the Human
CRM training in changing behaviours and Factors discipline, which is concerned with
attitudes (e.g., Helmreich & Foushee, 1993). human-machine and human-human interfaces,
and the integration of human operators and
The situation, however, is not entirely
maintainers within an overall system. CRM
positive. There are three areas of contention. 1)
consists of the application of human factors
CRM has been attacked for failing to eliminate
knowledge to the special case of flight crews and
pilot error and accidents; 2) There is confusion
their interactions with each other, with other
among some members of the research and
groups, and with the technology in the system.
operational communities about the definition of
Broadly defined, CRM consists of the effective
CRM; 3) It is claimed by some critics that CRM
utilization of all available human, informational,
issues are too subjective to be evaluated fairly,
and equipment resources toward the goal of safe
thus subjecting pilots to the risk of
and efficient flight. More specifically, it is the
discrimination; and 4) There is confusion about
active process employed by crewmembers to
the role of CRM programmes within
identify existing and potential threats and to
organisations. Before turning to the substance of
develop, communicate, and implement plans and
the discussion, it is necessary to address each of
actions to avoid or mitigate perceived threats.
these issues.
CRM also supports the avoidance, management,
Has CRM has failed in its mission? and mitigation of human errors. The secondary
The contention that CRM has failed its mission benefits of effective CRM programmes are
because accidents still occur is based on a improved morale and enhanced efficiency of
profound misunderstanding of human operations.
capabilities and limitations. By their very nature,
CRM issues are subjective and
humans are inevitably prone to errors and no
cannot be evaluated. Fear of inequitable
training, however sophisticated or intensive, can
evaluation is certainly a legitimate concern of
change human nature. Humans will make errors
pilots whose livelihoods may be threatened.
and accidents and incidents will occur in
However, contemporary CRM programmes
complex systems.
focus on specific and well-defined behaviours.
Those behaviours chosen for evaluation in the

1
Research reported here was conducted with the support of FAA Grants 92-G-017 and 99-G-004, Robert Helmreich, Principal
Investigator. Thanks are due to all of the participating airlines and their personnel who made the project possible. Particular credit is
due Captain Sharon Jones, James Klinect, Captain Bruce Tesmer, and John Wilhelm for data analysis and development of the
conceptual models.
European Union (which are closely related to the and other members of the British Empire define
Behavioral Markers defined by our research the other end of the continuum with much lower
group) are objective and observable. concern for rules and written procedures.
What is the role of CRM in the Looking at the cockpit environment,
organisation? Some have argued that CRM national culture influences how juniors relate to
should ultimately disappear as it becomes fully seniors, including their willingness to speak up
integrated into technical training. We once with critical information. It is demonstrated in
supported this notion, but with hindsight we now the way information is shared. Through Rules
realise that it is and should be a separate aspect and Order, culture influences adherence to SOPs.
of training. CRM falls at the interface between We have also found, unexpectedly, that culture is
safety departments, flight training, and flight strongly associated with liking for automation
operations. CRM programmes represent ongoing and attitudes about the appropriate use of
training driven by objective data reflecting automation.
operational issues. CRM is not a one-time
Organisational culture. Organisations
intervention, but rather a critical and continuing
can function within a national culture or can
component of a safety culture.
extend across national boundaries. An
Cultures the Missing Element organisations culture demonstrates its attitudes
and policies about human error, the openness of
Early CRM programmes and
communications between management and
investigations of human error in accidents
flightcrew, and the level of trust between
viewed the cockpit as an isolated universe. With
flightcrew and senior management.
growing sophistication, we now understand that
Organisational culture also influences norms
flight operations are part of a complex system
regarding adherence to regulations and SOPs. Of
that is heavily influenced by cultures. There are
great importance, the organisational culture
three intersecting cultures that surround every
determines the level of commitment to safety and
flight crew national, organisational, and
the strength of a safety culture.
professional (Helmreich & Merritt, 1998).
Hofstede (1980) defines culture very aptly as Professional culture. Many
the software of the mind. More technically, professions such as aviation have strong cultures
culture consists of the shared norms, values, and and develop their own norms and values along
practices associated with a nation, organisation, with recognisable physical characteristics such as
or profession. We shall not be concerned with all uniforms or badges. The positive aspects of the
the facets of national culture, but those aspects professional culture are shown in strong
that may influence behaviour in the cockpit. motivation to do well and a in a high level of
professional pride. There is also a negative
National culture. Two related
component that is manifested in a sense of
dimension of national culture identified by
personal invulnerability. In our research we have
Hofstede (1980) have particular relevance for
found that the majority of pilots of all nations
aviation, Individualism-Collectivism and Power
agree that their decision making is a good in
Distance (PD). Those from individualistic, low
emergencies as normal situations, that their
PD cultures tend to focus on the self, autonomy,
performance is not effected by personal
and personal gain while those from collectivist,
problems, that they do not make more errors
high PD cultures show great concern for the
under high stress, and that they can leave behind
group and harmonious relationships and
personal problems. While the positive aspects of
deference to leaders. Another relevant dimension
professional culture undoubtedly contribute to
has been labelled Rules and Order (Helmreich &
aviations splendid safety record, the macho
Merritt, 1998). Those high on this attribute
attitude of invulnerability can lead to risk taking,
believe that rules should not be broken, that
failure to rely on fellow crewmembers, and error.
written procedures are needed for all situations,
and that strict time limits should be observed. CRM as Defence Built on Data
This dimension, which is conceptually similar to
CRM programmes provide a primary
Hofstedes Uncertainty Avoidance has proved to
line of defence against the threats to safety that
be one on which there are large and highly
abound in the aviation system and against human
significant cultural differences. At the high end,
error and its consequences. Todays CRM
Taiwan and many Asian culture are most rule
training is based on accurate data about the
oriented while the United States, Great Britain,
strengths and weaknesses of an organisation. threat and error in aviation that is shown below
Building on detailed knowledge of current safety in Figure 1.
issues, organisations can take appropriate
remedial actions, which will include topics in External Threats Expected Unexpected
CRM. There are five critical sources of data, Events and Risks Events and Risks
External Error

each of which illuminates a different aspect of


flight operations. They are: 1) Data from formal
evaluations of performance in training and on the
line; 2) Incident reports from non-punitive Internal Threats Crew-Based
systems that encourage open communication; 3) Errors

Surveys of flightcrew perceptions of safety and


human factors; 4) Quick Access Recorders
Threat Recognition Error Detection
(QAR) that provide information on the conduct CRM Behaviors and Error Avoidance
Behaviors
and Response
Behaviors
of flight. (It should be noted that these data
provide a reliable indication of what happens but
not why things happen; and 5) Line Operations
Safety Audits (LOSA). I will focus on what we Outcomes
Additional
Recovery to
have learned from LOSA. A Safe Flight
A Safe Flight Error

The nature and value of LOSA. Line


Operations Safety audits are programmes that Incidents / Accident
use expert observers to collect data about crew
behaviour and situational factors on normal
flights. They are conducted under strict non-
jeopardy conditions, meaning that no crews are
at risk for observed actions.2 Observers code Figure 1. A model of threat and error in aviation
observed threats to safety and how they are As the model indicates, risk comes from both
addressed, errors and their management, and expected and unexpected threats. Expected
specific behaviours that have been associated threats include such factors as terrain, predicted
with accidents and incidents (and that form the weather, and airport conditions while those
basis for contemporary CRM training). Data unexpected include ATC commands, system
from LOSA provide a valid picture of system malfunctions, and operational pressures. Risk
operations that can guide organisational strategy can also be increased by errors made outside the
in safety, operations, and training. A particular cockpit, for example, by ATC, maintenance, and
strength of LOSA is that the process identifies dispatch. External threats are countered by the
examples of superior performance that can be defences provided by CRM behaviours. When
reinforced and used as models for training. Data successful, these lead to a safe flight. In addition
collected in LOSA are proactive and can be used to external errors, the crew themselves may err
immediately to prevent adverse events. The and again CRM behaviours stand as the last line
University of Texas project has participated in of defence. If the defences are successful, error is
eight audits with a total of nearly 4,000 flights managed and there is recovery to a safe flight, If
observed. In this paper, data from the three most the defences are breached, they may result in
recent audits, which include threat recognition additional error or an accident or incident. In our
and error management, are discussed. These three airline database, 72% of all flights faced
three LOSA projects were conducted both in the one or more threats, with a range of 0 to 11. The
U.S. and in international operations and involved average was two threats per flight. The most
two U.S. and one non-U.S. carrier. common threats encountered were difficult
A Model of Threat and Error terrain on 58% of flights, adverse weather on
28%, aircraft malfunctions on 15%, unusual
Data are most valuable when they fit within a ATC commands on 11%, external errors
theoretical or conceptual framework. Our (including ATC, maintenance, ground handlers,
research group has developed a general model of etc.) on 8%, and operational pressures on 8%.
Threats occurred most frequently in the descent,
2
In practice, members of the University of Texas project approach and landing phase (40%).
have trained observers from participating airlines and also
serve as observers. Their presence across all organisations
allows us to make valid cross-airline comparisons.
A Model of Crew-Based Error country would be classified as an undesired state.
3) Additional error the response to error, as
Errors made within the cockpit have
we have noted, can result in an additional error
received the most attention from safety
that again initiates the cycle of response.
investigations and crew error has been
implicated in around two-thirds of air crashes Undesired states can be mitigated by
Helmreich & Foushee (1993).3 Our analyses of crew action, for example, recognition of an
error have led us to reclassify and redefine error unstable approach and a decision to initiate a go-
in the aviation context. Operationally, flightcrew around). Response to the undesired state may
error is defined as action or inaction that leads to also result in another error. Finally, in the worst
deviation from crew or organisational intentions case, the result may be an accident or incident.
or expectations. Our definition classifies five
The model aids analysis of all aspects of
types of error: 1) Intentional noncompliance
error, response, and outcome. The failure or
errors or violations of SOPs or regulations.
success of defences such as CRM behaviours can
Examples include omitting required briefings or
also be evaluated. Errors thus classified can be
checklists; 2) Procedural errors in which the
used not only to guide organisational response
intention is correct but the execution flawed; 3)
but also as scenarios for training, either in
Communication errors that occur when
classroom or LOFT. The full error management
information is incorrectly transmitted or
model is shown graphically in Figure 2.
interpreted. Examples include incorrect readback
to ATC or communicating wrong course to the Intentional Noncompliance
other pilot; 4) Proficiency errors that indicate a Procedural
Error
lack of knowledge or stick and rudder skill; and Types Communication
Proficiency
5) Operational decision errors in which crews
Operational Decision
make a discretionary decision that unnecessarily
increases risk. Examples include extreme
manoeuvres on approach, choosing to fly into Trap
Error Responses Exacerbate
adverse weather, or over-reliance on automation. Fail to Respond

Response to error and error


outcomes. Three responses to crew error are
Inconsequential Undesired State Additional Error
identified: 1) Trap the error is detected and
managed before it becomes consequential or
leads to additional error; 2) Exacerbate the
Undesired
error is detected but the crews action or inaction State Mitigate Additional Error
leads to a negative outcome; 3) Fail to respond Responses

the crew fails to react to the error either Crew-Based


Accident/Incident
because it is undetected or is ignored.
Definition and classification of errors
and responses are based on the observable
process without consideration of the outcome. Figure 2. A model of flightcrew error.
There are three possible outcomes: 1)
Inconsequential the error has no effect on the
safe completion of the flight. This is the modal
outcome, a fact that illustrative of the robust P r o f ic ie n c y

nature of the aviation system; 2) Undesired O p e r a tio n a l

state the error results in the aircraft being in a D e c is io n

C o m m u n ic a t io n s
condition that increases risk. This includes
incorrect vertical or lateral navigation, unstable P ro c e d u ra l

approaches, low fuel state, and hard or otherwise V io la t io n s

improper landings. A landing on the wrong 0 10 20 30 40 50 60

runway, at the wrong airport, or in the wrong P e rc e n ta g e o f E rro rs

3
Early investigations tended to focus on the crew as the sole
causal factor. Today, of course, we realize that almost all Figure 3. Distribution of error types
accidents are system accidents as discussed by Helmreich &
Foushee (1993) and Reason (1998)
Error Results from LOSA
Examination of the aggregate data from A B C
the first three LOSAs in which error was
T h re a ts p e r s e g m e n t 3 .3 2 .5 .4
measured is instructive. Errors were committed
by 73% of the crews observed. The range of E rro rs p e r se g m e n t .8 6 1 .9 2 .5

errors on any flight was from zero to fourteen, % c o n s e q u e n t ia l 18% 25% 7%


with an average of two per flight. As Figure 3
illustrates, the most frequent type of error in the
data from the three airline study was intentional
non-compliance (or violation) followed by
procedural. The high percentage of procedural
non-compliance is alarming and will be Figure 4. Threats and errors in three airlines.
discussed later. Procedural errors doubtless have
multiple causes. They can reflect the inherent
limitations of humans accomplishing difficult
tasks, often under high workload conditions or I n t e n tio n a l O p e ra tio n a l
P ro c e d u r a l C o m m u n ic a t io n P ro fic ie n c y
N o n c o m p lia n c e D e c is io n
they may be an indication that procedures A ll
themselves are sub-optimal. Of all the errors F le e ts 33% 39% 12% 9% 7%
A d v te c h
observed, 18% were trapped, 5% were 1 40% 31% 15% 8% 7%
exacerbated, and 77% elicited no response. A d v te c h
2 30% 44% 19% 4% 4%
Errors differ by types in whether they become Conv1
3 c re w 17% 55% 9% 11% 9%
consequential. While proficiency and operational
Conv2
decision errors are least common, they are more 3 c re w 53% 20% 10% 10% 8%

likely to become consequential.


Of particular significance was the fact
that there were very large differences in threat, Figure 5. Percentages of error types within fleets
error, and percent becoming consequential in one airline.
between fleets within airlines and between
airlines as shown in Figures 4 and 5. By Phase of flight is also strongly
consequential, we mean resulting in an additional associated with the occurrence of errors and their
error or an undesired aircraft state. This finding consequences, as shown in Figure 6. Consistent
indicates the importance of airlines determining with analyses of world wide approach ad landing
the status of their own operations rather than accidents (Khatwa & Helmreich, 1998), the
assuming that their organisation conforms to highest percentage of errors occurs during this
some industry standard. Variability can result phase of flight and the highest percentage
from differences in the operating environment as becoming consequential. Clearly, special
well as from differing organisational cultures and attention should be directed toward enhancing
subcultures. Note in particular the range of performance in this phase.
variables which became consequential,
terminating in an undesired state. There is P h a s e o f F lig h t E rro rs C o n s e q u e n tia l
certainly differential risk in the three P re flig h t/T a xi 23% 8%
organisations. There are some common factors in
T a k e o ff/C lim b 24% 12%
the three airlines, but the data that follow are
shown primarily to show the type of information C ru is e 12% 15%

that can be obrained and its utility for safety. D e s c e n t/A p p ro a c h / 41% 23%
L a n d in g

Figure 6. Percent of errors and their


consequences by phase of flight.
Specific errors. Although a wide array unwieldy, and inefficient operating environment
of error classifications was observed, some major that invites violations (Reason, 1997).
problem areas emerged. Earlier audits pointed to
Although many violations may be
the appropriate use of automated systems as an
committed with the good intention of increasing
industry-wide problem (Helmreich, Hines, &
operational efficiency, organisations cannot and
Wilhelm, 1997; Sherman, Helmreich, & Merritt,
should not tolerate disregard for established
1997). Consistent with these findings, the most
procedures. There are several compelling reasons
frequent classification of error in LOSA involved
for this. One is, of course, that standardisation of
the operation of automated systems (mode
operations cannot be achieved with idiosyncratic
control panel and flight management computer.
adherence to procedures. There is also
Errors included wrong settings, wrong modes,
compelling evidence for the threat to safety
and failure to verify settings, along with
associated with violations. First, a Flight Safety
numerous others. Overall, these accounted for
Foundation analysis of global approach and
31% of all errors. The second highest
landing accidents found that more than 40%
classification (24%) was checklist errors such as
involved violations of SOPs (Khatwa &
non-standard terminology, procedural errors,
Helmreich, 1998). Second, analysis of LOSA
performance from memory, and failure to use
data indicate that those who commit intentional
required challenge and response methods. The
non-compliance errors are more likely to commit
third highest category consisted of sterile cockpit
other types of errors. Thus, it can be concluded
violations, accounting for 13%. Fourth highest at
that violations are associated with greater risk in
8% were ATC-related crew errors such as missed
operations. Further analyses may give us greater
calls, omitted information, and accepting
insight into the nature of this relationship.
instructions that unnecessarily increase risk (i.e.,
slam dunk approaches). The fifth highest CRM as Countermeasures
category (5%) consisted of briefing errors,
failure to conduct required briefings or leaving One of the most informative aspects of
out required information. The remainder of LOSA data is the ability to link threat
recognition and error management with the
errors fell into a variety of categories.
specific behavioural markers that form the core
Violations Matter of CRM. These emerge very clearly in observer
ratings provide a clear portrait of the actions
We were dismayed by the high
taken by effective crews. Those who deal
proportion of intentional non-compliance errors
proactively with threat and error exhibit the
found in the data. Several points regarding these
following behaviours:
violations should be considered. First, as we
have noted, there were very large differences Active Captain leadership
between airlines and between fleets within Briefing known threats
airlines. Hence, one cannot generalise from these Asking questions,
data about the general frequency of violations in speaking up
the global aviation system. This point is further Decisions made and
emphasised by the fact that the three carriers reviewed
included in the study all came from countries Operational plans clearly
that scored very low on commitment to rules, as communicated
measured by our Rules and Order scale Preparing/planning for
(Helmreich & Merritt, 1998). It would be threats
incorrect to assume that pilots from other
Distributing workload and
cultures, especially those high on in adherence to
tasks
procedures would be equally cavalier in
Preparation and planning -
disregarding formal rules. On the other hand, the
staying ahead of the curve
universal pilot belief in personal invulnerability
may foster a disregard for rules. The fact that Vigilance --monitoring
many rules are broken does not imply that and challenging
violating pilots have a death wish or have
contempt for formal requirements. One must also Leadership is an overarching behaviour that
consider the possibility that the proliferation of governs interaction on the flightdeck. Although
regulations may have created a contradictory, much of the attention in early CRM programmes
was directed toward overcoming the effects of
autocratic captains who fail to solicit or accept Data that show the nature
critical information from junior crewmembers, and types of errors
many current problems seem to be associated occurring
with weak leadership and the abdication of
authority. While the importance of the identified Training in error
markers is not surprising, the results do provide avoidance and
important validation of the importance of CRM- management strategies for
related behaviours. crews
Training in evaluating and
reinforcing threat
Data, CRM and Safety Culture recognition and error
The analysis of data from a variety of management for
sources (training evaluations, incident reports, instructors and evaluators
surveys, and LOSA) aid organisations in the Trust is a critical element of a safety culture,
diagnosis and understanding of their culture and since it is the lubricant that enables free
its subcultures. Without an understanding of its communication. It is gained by demonstrating a
own cultures, organisations cannot mount non-punitive attitude toward error (but not
effective programmes to optimise them. Data on violations) and showing in practice that safety
how crews deal with threat and avoid and concerns are addressed. Data collection to
manage error help organisations develop and support the safety culture must be ongoing and
maintain a safety culture. LOSA data, in findings must be widely disseminated.4 CRM
particular, are of enormous value because they training must make clear the penultimate goals of
are proactive and allow organisations to take threat recognition and error management.
appropriate action before accidents and incidents Ancillary benefits such as improved teamwork
occur. Proactive interventions are a defining and morale are splendid, but not the driving
characteristic of an effective safety culture. force. Finally, instructors and check airmen need
Data also identify critical areas for ongoing special training in both evaluating and
CRM training. However, as noted earlier, CRM reinforcing the concepts and in relating them to
is not a universal panacea for safety problems in specific behaviours.
the aviation system. Accidents and incidents If all of the needed steps are followed
almost always have multiple roots and many and managements credibility is established, a
cannot be changed by training alone. true safety culture will emerge and the
Organisations nurturing a safety culture must contribution of CRM to safety will be
also deal with those issues identified by LOSA recognised.
and other data sources, interventions may include
revising procedures, changing the nature and References
scope of technical training, changing scheduling Helmreich, R.L., & Foushee, H.C. (1993). Why
and rostering practices, establishing or enhancing Crew Resource Management? Empirical
a safety department, and a variety of other and theoretical bases of human factors
actions. training in aviation. In E. Wiener, B.
There are basic steps that every Kanki, & R. Helmreich (Eds.), Cockpit
organisation needs to follow to establish a Resource Management (pp. 3-45). San
proactive safety culture that is guided by the best Diego, CA: Academic Press.
possible data on its operations. These include the Helmreich, R.L., Hines, W.E., & Wilhelm, J.A.
following: (1996). Crew performance in advanced
Trust technology aircraft: Observations in 4
airlines. The University of Texas
A non-punitive policy Aerospace Crew Research Project
toward error Technical Report 96-8.
Commitment to taking Helmreich, R.L., & Merritt, A.C. (1998). Culture
action to reduce error- at work in aviation and medicine: National,
inducing conditions
4
One airline had its LOSA report bound and placed copies in
every aircraft as well as every base for crews to peruse.
organizational, and professional influences.
Aldershot, U.K.: Ashgate.
Helmreich, R.L., Merritt, A.C., & Wilhelm, J.A.
(in press). Error and resource management
across organizational, professional, and
national cultures. In E. Salas, C.A. Bowers,
& E. Edens (Eds.), Applying resource
management in organizations: A guide for
training professionals. Princeton, NJ:
Erlbaum.
Helmreich, R.L., Merritt, A.C., & Wilhelm, J.A.
(1999). The evolution of Crew Resource
Management in commercial aviation.
International Journal of Aviation
Psychology, 9, 19-32.
Hofstede, G. (1980). Cultures Consequences:
International Differences in Work-Related
Values. Beverley Hills, CA: Sage.
Khatwa, R. & Helmreich, R. (1998). Analysis of
critical factors during approach and landing
in accidents and normal flight. Flight Safety
Digest. 17, 1-256.
Reason, J. (1997). Managing the Risks of
Organizational Accidents. Aldershot, U.K.:
Ashgate.
Sherman, P.J., Helmreich, R.L., & Merritt, A.C.
(1997). National culture and flightdeck
automation: Results of a multination
survey. International Journal of Aviation
Psychology, 7, 311-329.
University of Texas Crew Research Project
Website:
www.psy.utexas.edu/psy/helmreich/nasaut.htm
PRESENTATIONS
Monitoring
Normal Line Operations:
A Progress Report
Captain Dan Maurino
Flight Safety and Human Factors - ICAO
First IATA/ICAO LOSA/TEM Conference
Dublin, Ireland
5 - 7 November 2003
When the Book Goes Out of the Window
Baseline
performance

Baseline performance S A
Practical

System
LO
Drift

design Operational
deployment Operational
performance
Threat & Error Management Model (Doc 9803)
Threats
Threat Induced
Inconsequential Threat Management Incident or
Accident
Crew Error

Crew Error
Responses

Undesired Aircraft
State

Crew Undesired Error Induced


Aircraft State Incident or
Responses Accident
The Expanding Role of the TEM Model
TEM Model as
9Training tool (Numerous airlines & IATA)
9Safety management tool (IATA/SAC)
9Research tool (Boeing/Madrid)
9Licensing tool (ICAO)
TEM - Licensing Tool (ICAO FCLT/P)
Competencies SKAs Performance criteria

Follow SOPs
Threat Management Perform Briefing
CRM skills State plans
Assign workload
TEM Contingency planning
Error Management principles Monitor/Cross-check
Manage automation
Undesired Aircraft
State Management

2003 Milestones
The LOSA experience in FUTURA
9First LOSA in a Latin airline
9First non -TLC LOSA
The LOSA experience in Braathens
9First European LOSA
The LOSA experience in Aeromexico
9First TLC LOSA in a Latin airline
Air New Zealand LOSA
9First repeat LOSA other than Continental
2003 A Most Significant Milestone
The Expansion of LOSA to ATM
Eleventh Air Navigation Conference (ANConf/11)
9Develop guidelines to monitor normal ATS operations based
on LOSA
Normal Operations Safety Survey (NOSS)
Project Team
Kick-off workshop Friday afternoon
9ASA, CANSO, Airways NZ, Eurocontrol, IATA, ICAO,
IFATCA, LVNL, NASA/UT, NATS, NavCanada, UK CAA
ICAO NOSS world-wide conference last quarter 2004
Clarifying the Interfaces
ICAO, UT and TLC
Concept: the need to monitor normal operations on a
systematic basis (normal operations monitoring)
Tool: LOSA So far, the only existing tool
UT: research component of the concept
TLC: implementation of the tool
ICAO strongly endorses the concept, and supports LOSA
as the only existing tools
9ICAO will support any other tool to be developed based on
the TEM Model and the Ten Operating Characteristics
A Peacekeeper Perspective
LOSA, CRM and NOTECHS
LOSA: a tool to capture systemic data
NOTECHS: a CRM evaluation template
LOSA: cognition & context
NOTECHS/CRM: behaviours & stereotypes
CRM: just one piece of the infinitely broader
information picture developed from LOSA
9A situation, generated by all the wrong reasons, that
must be put behind
What the Future Holds
ICAO fully endorses and supports normal ops. monitoring
9 Enabling partner with all organizations and aviation domains
9 IATA IFALPA - US ALPA IFATCA Boeing - Bombardier
9 LOSA/TEM Conferences
9 ICAO LOSA Manual (Doc 9803)
9 June 2002 edition of the ICAO Journal
9 Amendment No 1 ICAO HF Training Manual (Doc 9683)
Amendment to Doc 9683 to include NOSS/TEM in ATS
Recommendation and subsequent standard [in Annex 6] within
the context of SMS
Collecting Safety Data
from Normal Operations:

The Perspective of IATA


Jill Sladen-Pilon

Manager, Safety

1
2
Introduction to IATA

273 Airline Members from 143


countries accounting for 98% of
total international traffic

3
Leading in Safety
Lead the global airline commitment to achieve a continuous
improvement in safety.

Safety Infrastructure
Auditing Safety

Cargo Safety Data


Safety Management

Cabin Safety
Safety Training
4
STEADES
Safety Data
Management

ASR LOSA

IOSA FDM
Data
Essentials
Analysis
Core STEADES 5
STEADES
Safety Data
Management

Safety Boards

Manufacturers ICAO

Communicator
ATSPs
Risk-based Reports CAAs
Smaller Airlines

ASR LOSA
Manager
FDM Global
IOSA Individual
Data
Essentials
Analysis
Core STEADES SMSS
6
Safety Bulletin
Regular pulses of safety
intelligence
Threats to operational
safety
Released monthly, 3rd
edition in production
Now available in English,
French and Spanish

7
1. IATAs position on LOSA

IATA endorses LOSA as a tool for


normal operations monitoring and
toward understanding and
reducing operational errors, when
it incorporates the 10 essential
operating characteristics.

8
2. Communication

IATA is part of the LOSA advisory


board
communicate lessons learned

9
3. TEM as a Training Tool

CRM/Threat and Error Management-NEW

Aviation Training and Development Institute


KNOWLEDGE. EXPERIENCE. NETWORKING. SKILLS. RESULTS

10
IATA Safety Manual

1. Safety Management Systems for Airlines


2. IATA Safety Management Support System
3. Risk Management
4. Threat & Error Management
5. Flight Data Management and Analysis
6. Emergency Response Planning
7. Safety Report

11
4. IATA Safety Report 2002
Accident Analysis
Colour printed
CD-ROM Including:
Supporting Documents
Safety Toolkit
CEO Brief
Related web links

12
IATA Data Classifications
ENVIRONMENTAL

E1 Meteorology
E2 ATC/conflicting traffic
TECHNICAL E4 Birds/Foreign Object
Damage
T2 Engine failure, malfunction
T11 System failure

ORGANISATIONAL
HUMAN Factors
O2 Inadequate SOPs

13
5. TEM as an analytical tool
Safety Report
our current classifications employ this
philosophy
Considering LOSA/TEM coding of threats
and errors
Incident Review Meetings
reporting of incidents using a threat and
error framework

14
Threat and Error
Management
Analytical Toolkit: Template

15
Threat and Error Management (TEM)
Threats

Threat Mngmt

Errors
Error
Mngmt

16
Outcome
Introduction to Event
XYZ Airways BA-100 aircraft, scheduled
flight ABC to CDE. Takeoff in fair
weather
No. 2 engine failed ~10 minutes (60
miles) after takeoff, air turnback &
single engine approach to ABC
Crash occurred ~1 minute after single
engine go-around was ordered, 2km
lateral from end of runway
Fire destroyed airplane; 36 fatal / 1
survivor
17
Threats

1. Engine failure during climb


2. Problems with compass/
pressurization
3. Night engine out approach
4. Engine out go-around

18
Threat Management
1. Engine failure during 1. Managed - Engine
climb shutdown according to
SOP
2. Problems with 2. Mismanaged - Working
compass/ pressurization problem
pressurization during engine out approach
3. Night engine out 3. Mismanaged - Low altitude
approach outbound on procedure/Off
course at minimums
4. Engine out go- 4. Mismanaged - Pitched up
around to stick-shaker/ Speed well
below Vref 19
Errors

1. Control of pressurization
2. Low on instrument approach
3. Off course on ILS
4. Pitch control on engine out go-
around

20
Error Management
1. Control of 1. Mismanaged - Let distraction
pressurization carry over to instrument
2. Low on instrument approach
approach 2. Mismanaged - FO questioned
application of SOPs/ FO
3. Off course ILS callouts
3. Mismanaged - At minimums
mile off course
4. Pitch control of
engine out go- 4. Mismanaged - Initial pull-up
around put airplane in slow
speed/low altitude/ engine out
position
21
Prevention Strategies

Training should include the


management of distractions
during engine failure situations
Engine out training at night
during low visibility

22
Benefits
Clear presentation of the facts
Identifying Threats and Errors
leads to recognising effective
countermeasures and prevention
strategies.

23
Unlock the value of your data

24
LOSA: Line Operations Safety
Audit: History and Status

Robert L. Helmreich, PhD, FRAeS

Human Factors Research Project


The University of Texas at Austin

Dublin
November 5, 2003
Data to Isolate Safety Issues
Accident investigation
Limited, non-representative sample
Incident reports (CHIRP, ASRS and ASAP)
Data slanted to events resulting from system and flight crew failures
UT HF developing data category system with AA & CO
Formal checkrides (Line and Proficiency)
Data show crew capability and procedural knowledge
Flight Data Recorders QAR (FOQA)
Data show what happened in terms of flight parameters
Non-jeopardy observation of normal flights-LOSA
Give data on why things happen and how they are managed
Provides realistic baseline of safety data
Is proactive
LOSA
Jump seat observations of flight crew performance during
regular scheduled flights

Observers unobtrusive collecting data not


participating in flight
Team of observers from different backgrounds
Line pilots / Union representatives
Check airmen
Safety and Training pilots
Credible outside observers (for reliability check)

All data are DE-IDENTIFIED and CONFIDENTIAL


Scientific Background
LOSA uses systematic observation of behavior
Systematic observation is a validated
methodology using observers trained and
calibrated to high reliability
Systematic observation has been employed in
scientific studies of crew performance in
demanding environments
For example, Aquanauts in undersea habitats during
Project SEALAB and Project Tektite
History of LOSA
Formal LOSA developed at request of Delta
Airlines to validate operational impact of
Human Factors (CRM) training

The focus of initial LOSA was systematic


assessment of CRM-related crew behaviors

In 1997, collaboration with Continental


Airlines to expand LOSA to include threats
and errors and their management
Purpose of LOSA
Provide valid empirical data on:
1. Crew performance strengths and
weaknesses
Proficiency
Decision-Making
CRM skills
Procedural compliance
Threat and error management
2. System performance strengths and
weaknesses
Culture
Airspace System airports and navigational Aids
Aircraft design / automation
Standards / Training / Safety / Maintenance
Crew support ATC, Cabin, Ground, and Dispatch
Threat and Error Management
LOSA (1997-2003)
Continental Latin EVA Air
America Uni Air (Taiwan)
Continental Express Frontier
Air New Zealand QANTAS
Air Micronesia Singapore
Continental Silk Air
Braathens
Delta
Alaska
USAirways
Air New Zealand (2nd)
Cathay Pacific
LOSA Components
Part 1. Threat prevalence & mgt
Part 2. Error prevalence & mgt
Part 3. CRM Countermeasures
Behavioural markers
Part 4. Flight crew interview
Part 5. Flight crew survey
LOSA Data: Flight Crew
For each flight segment, observers collect data on:
General Flight Crew Information Flight Crew Performance
Demographics Behavioral markers
Attitudes / perceptions / Crew errors and
safety culture (FMAQ) violations
Safety interview
Undesired aircraft states
comments
Technical data for
Flight Description
approaches
Observer narrative
Type and stability
Overt threats
Threat management
Operational complexity
Error management
Undesired state
management
Threat
Threat Categories
Threats - Originate outside the flight crews influence but require active
management to prevent them from becoming consequential to safety

Environmental Threats Airline Threats


Adverse weather Operational time pressure
ATC events / errors Cabin events / errors
Terrain MX events / errors
Traffic A/C malfunctions / MELs
Airport conditions Ground / Ramp events /
errors
Dispatch events / errors
Ground crew events / errors
Latent Threats
Aspects of the system that predispose the
commission of errors or can lead to undesired
aircraft states
ATC practices
Organizational, national, professional culture
Aircraft characteristics
Qualification standards
Regulatory practices and oversight
Flawed procedures
Scheduling and rostering practices
Threat Prevalence
Last 10 LOSAs
1835 flights with 7576 threats
4.1 threats per flight average

98% of flight segments had one or more threats


Range across airlines 94% to 100%

2/3 were environmental threats; 1/3 were airline threats


44% of environmental threats occur in descent / approach / land
72% of airline threats occur in predeparture

Most prevalent threats


Adverse weather 26% of all threats
ATC clearances/late changes - 21% of all threats
Threat Management
Last 10 LOSAs
960 mismanaged threats in 1835 flights
Average 1 mismanaged threat every 2 flights
87% of threats effectively managed
Range across airlines 79% to 92%
Environmental and airline threats managed equally well
Highest mismanagement rates
17% of ATC threats were mismanaged
15% of Aircraft malfunctions /MELs threats were mismanaged
Error
Error Avoidance

Complete error avoidance is impossible


errors are inevitable
80%+ of all flights observed had one or
more errors
Must look for sources of error to strengthen
system defenses
Broad Error Categories

Intentional Noncompliance violations


ex) Performing a checklist from memory
Procedural Followed procedures but wrong
execution
ex) Wrong altitude setting dialed into the MCP
Communication Missing information or
misinterpretation within cockpit
ex) Miscommunication by crew with ATC
Decision Discretionary decision that unnecessarily
increases risk
ex) Unnecessary navigation through adverse weather
Decision Error

Choice increasing risk in a situation with


multiple courses of action possible
time available to evaluate alternatives

no discussion of consequences of alternate


courses of action
no formal procedures to follow
Error Prevalence

Last 10 LOSAs
1835 flights with 5172 errors = 2.8 errors per
flight on average
82% of flight segments had one or more errors
Range across airlines 70% to 94%
32% of all errors are intentional noncompliance
errors
Error Response

25% Detected and action taken

30% Detected and ignored


Almost all intentional noncompliance

45% Fail to detect


Red Flag: improve monitoring and cross-checking
Error Outcomes
71% Inconsequential
21% Undesired aircraft state
8% Additional error

1484 mismanaged errors in 1835 flights = 0.8


mismanaged errors per flight on average

77% of intentional noncompliance errors were


inconsequential
Are procedures in need of review?
Undesired Aircraft State
A compromised situation placing the flight at increased
risk
Lateral deviation Unstable approach

Vertical deviation Abrupt aircraft control

Speed too high Long landing no go

Speed too low around

Incorrect aircraft configuration Firm landing


Flight controls
Systems Forced landing
Fuel Wrong taxiway, ramp,
Automation
runway, country

Runway incursion
Error/Outcome Summary

Procedural

Communication

Decision

Non-Compliance

0 20 40 60 80 100

% of errors % consequential
7% of errors involved a lack of technical proficiency
Phase of Flight Effects
Threats by Errors by
Phase of Flight
Phase Phase
Pre-Departure / Taxi 40% 26%
Takeoff / Climb 15% 20%
Cruise 8% 6%
Descent / Approach /
Land
33% 44%
Taxi / Park 4% 4%

Descent / approach / land phase contains the most variability


in crew performance and most consequential errors
Threat and Error
Is a high level of threat associated with
more errors?
There are slightly more unintentional errors
when there are many threats, but the
relationship is weak
There is no relationship between threats and
intentional non-compliance
Profile of Airline X
Average from 4 LOSA
Major Statistics Airline X Archive Comparison
Airlines
Average number of
threats per flight 4.3 4.3
% of threats that were
mismanaged 8% 15%
Average number of
errors per flight 3.0 2.5
% of errors that were
mismanaged 18% 31%
Airline X
Average from 4 LOSA
Major Statistics Airline X Archive Comparison
Airlines
Average number of
mismanaged errors per 56 75
100 flights
Average number of
undesired states per 42 53
100 flights
Average number of
mismanaged undesired 6 8
states per 100 flights
Myths from the Days of Jurassic
Jets
Myth 1: The glass cockpit will eliminate
human error on the flight deck
Reality: Automation is the second largest
source of pilot error
Myths

Myth 2: Dog and duck automation


Myths

Myth 2: Dog and Duck automation will


prevail
3 individuals in crew
Pilot, dog, duck
Pilot reaches for controls
Duck quacks

Dog wakes and bites pilot

Reality: the human is still in the loop


FMAQ Safety Scale Items
My suggestions about safety would be
acted upon if I expressed them to
management

Management will never compromise safety


concerns for profitability

I am encouraged by my supervisors and


coworkers to report any unsafe conditions
High scores on the FMAQ Safety
Scale are associated with fewer
violations of SOPs and fewer
Undesired Aircraft States
University of Texas
Threat and Error Management
Model (UT-TEMM)
The Threat and Error Management model was derived
from LOSA data and guides further refinement.

The model is being used by airlines as a framework


for analysis of incident and ASAP data

IATA is using it as the framework for analysis of


worldwide accidents and incidents
Threat and Error Management Model
Threats:
Latent and Overt

Threat M anagement

Errors

Inconsequential Error M anagement Incident / Accident

Undesired Aircraft
State

Undesired Aircraft
State M anagement
Applying LOSA Data
LOSA data have two primary uses:
1. Assessing system safety
2. Identifying issues for action
LOSA database has data on airports, aircraft, crew
experience, organizational and professional culture
Providing airlines with feedback on their own
operations
Observers provide valid record of what crews do on the
line
Show areas of strength as well as those needing
improvement
Data help airlines prioritize and evaluate safety efforts
CRM training
Using Error Data for
Organizational Interventions
Violations - suggest poor procedures, weak captain
leadership and/or a culture of non-compliance

Procedural errors - may indicate poor workload


management and/or poor procedures

Communications errors - may reflect inadequate CRM


(monitoring and challenging) or complacency

Decision errors - may indicate need for more CRM


training on expert decision making and risk assessment
LOSA and CRM

CRM has evolved through 6 generations

LOSA data provide airlines with critical


topics for 6th generation CRM with its
focus on threat and error management
Emerging Problems with CRM
Not accepted by all (boomerangs)

Purpose not understood -- to help us get


along better

Lacking a universal rationale that can be


accepted in every culture and organization

Slippage over time need for refresher


Fifth Generation CRM :
Error Management 1996-
Focus on managing human error
Training in limitations of human
performance
Universal nature of human error
use of incident & accident data to illustrate

Continuation of earlier generation training


topics under error management framework
Continental Airlines used LOSA data
to develop Threat and Error
Management CRM training
the 6th Generation
Sixth Generation CRM:
Threat and Error Management
1999 -
Fifth generation was useful, but created
resistance among pilots who did not like
the idea that their task was managing
their own errors
LOSA documents not only errors but also
the nature and prevalence of threats in
the operating environment shows
superior performance as well as problems
Training Issues in Threat
and Error Management
Human limitations as sources of error
The nature of error and error management
Culture and communications
Expert decision-making
Training in using specific behaviors and procedures as
countermeasures against threat and error
Briefings
Inquiry
Sharing mental models
Conflict resolution
Fatigue and alertness management
Analysis of positive and negative events and accidents
Significant LOSA Events

LOSA Weeks
Hong Kong
Panama
Dubai
Dublin

ICAO Journal May 2002 features LOSA


ICAO LOSA Handbook 2002
Recognition and Extension of LOSA

Flight Safety Foundation/Aviation Week and Space


Technology Laurels, 2001
DOSA - Dispatch LOSA (with Continental)
University of Texas Center of Excellence in Patient
Safety adopting LOSA methodology for
systematic observational research into operating
rooms and emergency rooms
The LOSA Collaborative
An umbrella research organization led by James Klinect
Collaborative functions
1) Implementing LOSA and maintaining the integrity of the
process and database by certifying observers and data
2) Supporting research efforts of the University of Texas
Human Factors Research Project
3) Offering user interface data to manufacturers to enhance
design process and de-identified system data to industry
and regulators
4) Coordinating a support group of participating LOSA airlines
to exchange information on data use and oversight for the
process
The University of Texas
Human Factors Research Project

www.psy.utexas.edu/HumanFactors
ICAO / IATA LOSA & TEM Conference
November 5-7, 2003 Dublin, Ireland

The Ten Operating


Characteristics of LOSA

James Klinect
The University of Texas / The LOSA Collaborative

The University of Texas Human


Factors Research Project
Line Operations Safety Audit (LOSA)
LOSA is not a:
Regulatory compliance audit
New form of line checks
Next generation CRM training course

LOSA is similar to a cholesterol check


Snapshot of system performance strengths and weaknesses

LOSA is defined by having 10 operating characteristics


Supported by ICAO, IATA, US ALPA, IFALPA, The University of Texas and
The LOSA Collaborative
LOSA Operating Characteristics

1. Jumpseat observations during normal operations

2. Anonymous and confidential data collection

3. Voluntary crew participation

4. Joint management / union sponsorship

5. Trusted and trained observers


LOSA Operating Characteristics

6. Safety-targeted data collection form

7. Trusted data collection site

8. Data cleaning roundtables

9. Data-derived targets for enhancement

10. Results feedback to line pilots


Why LOSA Operating Characteristics?

Regulator Check Airman Nobody


Angel Natural
Performance
- Pilot trust in LOSA + Performance
- LOSA value to airline +

LOSA Observer

LOSA operating characteristics enhance:


Pilot trust in LOSA
Probability of observing as close to natural performance as possible
LOSA value to the airline
Summary
LOSA is similar to a cholesterol test
Snapshot of performance strengths and weaknesses

LOSA operating characteristics define the project by


enhancing pilot trust and insuring value to the airline

For LOSA to be successful, airlines must:


Embrace LOSA as a learning project
Be prepared for positive or negative results
Have a defined safety change process to implement data-driven
safety solutions next presentation by Don Gunther
Questions?

The University of Texas


Human Factors Research Project

www.psy.utexas.edu/humanfactors

www.losacollaborative.org
The Safety Change Process
Following Line Operations
Safety Audits (LOSA)

Captain Don Gunther


Director Human Factors & Safety
Continental Airlines
The Continental Airlines
Safety Change & Training
Development Program

Process For A Greater Level Of Safety


Are We Really Safe?
Safer than in the past?
Not as Safe as we need to be?

How Do We Know?

No ones been Killed in our operations.


We havent had a resent Major Accident.
SO WHAT!
The Wake-up
Any Carrier can suffer a major accident at any time. Were
only as safe as our last Accident.

We must continue to investigate accidents so they are not


repeated. But, we must also reduce accident precursors to
eliminate future accidents.

Remember, Things are happening in everyday, normal flight


operations and -- we dont know about them!

All flights are exposed to risk. The only sure way to avoid the
risk of an accident is to not fly at all.
Because Flying IS our Business:

System Threats must be identified and


reduced/eliminated.

Crew Errors must be avoided and managed.

A Safety Measurement System must be used to


identify Targets for improving the normal
flight operations safety margin.
In The Past - Safety Changes:
Were made in response to
Accidents/incidents
FAR changes
FAA directives
NTSB accident investigations.

Were made, based on the experience & intuition of Flight


Operations Managers.

Were not based on any current operational data concerning


accident/incident precursors, because there was no data
available.

Were successful in achieving a very low accident rate.


Todays Safety Change Program
Discovers Targets using safety data and analysis concerning
accident PRECURSORS.

Discovers the What, Why, How associated with safety


events.

Todays safety data tools include:


Flight Operations Quality Assurance (FOQA)
Line Operations Safety Audit (LOSA)
Aviation Safety Action Program (ASAP)
Continental Airlines Safety Information System (CASIS)
Advanced Qualification Program (AQP)

Provides Flight Operations Managers additional insight


required to improve upon the current accident rate.
Why LOSA, ASAP FOQA, etc?

The essence of a good flight data analysis and


reporting system is that it should be confidential and
non-punitive. The concept is that it is better to know
about a potential problem - so that it can be analyzed
and the underlying reasons corrected in order to
prevent its reoccurrence before it leads to something
more serious - than to punish those that might have
made an error, etc
-Flight Safety Foundation
November 2002
Is Data All We Need?

No.

The new explosion in data collection has


brought forward the New Data Wave.
Is Data All We Need?

No. The new explosion in data collection


has brought forward the New Data Wave.

The data must be mined and analyzed to


identify the Safety Targets

The Safety Change Program then takes the


targets and translates them into Changes.
Data Driven - Safety Change &
Training Development Program
4Measure (with LOSA,
LOSA FOQA, ASAP) to obtain
targets
4Detailed analysis of targeted issues
4List of potential changes for improvement
4Risk analysis and prioritization of changes
4Selection and funding of changes
4Implementation of changes (training development &
actual training)
4Time for changes to stabilize while training is
completed.
4Re-measure
What Types Of Changes Are Made?

Changes in operational philosophy, policies and


procedures.

Changes to the System, the aircraft, hardware,


software

Changes to Threat & Error Management Strategies


and Countermeasures. (Humanware)
The Safety Change &
Training Development Program

Change itself is a safety threat.

An explanation of why the change was made


improves acceptance.

Change is more readily accepted when there are


data to support the change.

When change data & information precede the


change, training to proficiency is achieved sooner.
Training Development
Is meant to be proactive.

Needs to be based on valid data.

May require thinking outside the box.

Must be human centered and tested.

Needs to be a continuous process.


Use of LOSA, FOQA, ASAP
Data
in the Process For
A Greater Level Of Safety
The Continental Airlines Safety
Change & Training Development
Program
Safety Changes
(FOQA)-
improved arrivals into Tel Aviv, Mexico City using
FMS
Pitch Report to counter tail-strikes
Information flow to Maintenance and Engineering (flap
issues, data base issues, predictive W/S performance)
(ASAP)-
Change in altitude restriction for arrivals into EWR
Improved Pre-departure Clearance format
Joint FAA/Industry effort to work ATC issues at
targeted airports
LOSA
Line Observation Safety Audit
LOSA 1996
Error Managements
early focus was:

Managing crew error


Error Management

RESIST
HARDWARE & SOFTWARE THAT
EXISTS BEFORE THE HUMAN ENTERS
RESISTANCE
HARDWARE & SOFTWARE THAT EXISTS BEFORE
THE HUMAN ENTERS

GPWS SOPs
TCAS CHECKLISTS
TRAINING AUTOMATION
MANUALS ATC
Error Management

RESIST
HARDWARE & SOFTWARE THAT
EXISTS BEFORE THE HUMAN ENTERS
RESOLVE
WHAT THE HUMAN BRINGS TO THE SYSTEM
RESOLVE

WHAT THE HUMAN BRINGS TO THE SYSTEM

PROFICIENCY DECISION MAKING


VIGILANCE EXPERIENCE
ASSERTIVENESS LEADERSHIP
MONITORING & SIT. ASSESSMENT
CHALLENGING CHECKLIST DISCIPLINE
NASA Guidelines
Positively delegate flying and monitoring duties

Monitoring is as important as flying

Flying pilot does not become involved with


secondary tasks

When conflict arises-resolve with outside source

When in doubt-must express!


MONITORING & CHALLENGING

Monitor
Take
Challenging Action
steps

Express
your
view

Error Resolved
Monitoring &
Challenging
Monitor
Take
Action

The Heart and Sole of


Express
Threat & Error
your view
Management
Error Resolved
The Process
1st year: LOSA and data analysis followed
by course development
2nd & 3rd years: Training course for all
crewmembers, Check Airmen training and
imbedding of EM into courseware, etc.
4th year: Preparation for next LOSA and
targeting areas to be measured and new
areas to be emphasized.
Line Observation Safety Audit
LOSA 2000
LOSA data shows another area of
vulnerability...

THREATS
THREATS
Influences that can lead to crew error
Passenger events
Distractions ATC
Cabin Crew
Terrain
Weather

Maintenance Similar call sign

Ground Crew Time pressures

Flight
Heavy traffic
diversion

System malfunction
Unfamiliar airport

Automation event Missed approach


THREATS
Threats
Do not equal errors
Increase error potential

Threats = Red Flags


Strategies
Industry, corporate and personal
countermeasures crews use to manage
threats
Will reduce the number of hard errors
Will improve the error management process by
increasing the awareness of potential errors and
the proper countermeasures to be used
Threat and Error
Management
The Process
1st year: LOSA and data analysis followed
by course development
2nd & 3rd years: Training course for all
crewmembers, Check Airmen training and
imbedding of TEM into courseware, etc.
4th year: Preparation for next LOSA and
targeting areas to be emphasized.
TEM
the challenge

How to improve Threat


identification
Get it on the RADAR
Weather

Time Pressure Unfamiliar Airport

System Malfunction Terrain

Automation Event
Distraction Radio Congestion
Diversion

Missed
Cabin Approach
Events

STRATEGIES
to counter THREATS

HARDWARE / SOFTWARE TOOLS


Checklists, FARS, SOPS, Aircraft Warnings, Etc.

TOOLS YOU BRING TO THE FLIGHT

Resolve to use SOPS, Crew


Briefings, Communications
Etc.
Side 1

Cathay Pacific
LOSA
Capt. Henry Craig
Cathay Pacific

Training for Safety


Aiming for Excellence
CX LOSA Cathay Pacific Airways 2003
Side 2

Why LOSA at Cathay?


Proactive not Reactive
Wanted a health check!
Conducted a home-grown LOSA in 1998
Other data doesnt tell the full story
A culture where good enough isnt good enough
A sound track record, doesnt guarantee a safe future
Significant changes had already taken place

CX LOSA Cathay Pacific Airways 2003 LOSA.2


Side 3

Why LOSA at Cathay?


Proactive not reactive
Crew Demographic Changes
Recruitment (F/Os Cadets; S/Os)
Experience levels changing
(especially F/Os and Relief Pilots-in-Charge)
F/Os: less sectors; shallower background; younger

CX LOSA Cathay Pacific Airways 2003 LOSA.3


Side 4

Why LOSA at Cathay?


Proactive not reactive
Crew demographic changes

Aircraft Changes
Fleet size doubled (in just over 10 years)
Predominately analogue to all EFIS
Predominately 3-man cockpit to 2-man cockpit
ULR & Crew Complement & Automation
= less manual flying
CX LOSA Cathay Pacific Airways 2003 LOSA.4
Side 5

Why LOSA at Cathay?


Proactive not reactive
Crew demographic changes
Prophesy
Aircraft changes Hearsay
Opinion
Snapshot through Measurement Guesswork
Already data-rich (QAR; Pilot Training Reports;
CAR; CRP; ASR etc.)
Establish an accurate baseline in Line Operations
Ensure change is data/process driven - not PHOG
(Ian Brooks)
LOSA is data with a human face - it closes the gap

CX LOSA Cathay Pacific Airways 2003 LOSA.5


Side 6

Considerations
Find a LOSA Champion

Dont underestimate -
amount of hard work required
amount of co-ordination and co-operation required
number of roadblocks to overcome
number of stakeholders who want ownership

Establish agreement with all stakeholders:


the accountants; the pilots; the schedulers; the cabin
crew; the outports

CX LOSA Cathay Pacific Airways 2003 LOSA.6


Side 7

The Steps
Be patient!
Fact-finding mission to LOSA Collaborative
Count the Cost / Appoint a Program Manager
Liaise with LOSA Collaborative (listen well)
work out a schedule
appoint & train auditors
Produce a written agreement between Company and Pilots
Union
Communicate (NTC; Crews News; email; Union magazine;
letter to ISM; pilot forums / C&T meetings)
Be flexible!
CX LOSA Cathay Pacific Airways 2003 LOSA.7
Side 8

LOSA Flights June 2001


Aircraft type Observations
A330/340 regional 55
B777 51
A330/340 augmented 28 *
B747- 400 26 *
Total: 160
(* less than targeted)

CX LOSA Cathay Pacific Airways 2003 LOSA.8


Side 9

Cathay LOSA Report (March 2002)

Threat Results
Error Results
Undesired Aircraft State Results
Threat & Error Countermeasure Results
Crew Interview Comments

Results measured against the LOSA archive


(5 other anonymous airlines)

CX LOSA Cathay Pacific Airways 2003 LOSA.9


Side 10
An event or error outside the
Threats crews influence, that requires
crew management and attention
(LOSA Report)

Common Threats:
Adverse Weather
e.g. Thunderstorms, Windshear etc.
ATC
e.g. Difficult clearances, language, errors etc.

Aircraft Malfunctions
e.g. Systems, engine overtemps & vibration,
brake overheats, APU, cabin etc

Others
e.g. Cabin distractions, ground & ramp problems,
ground maintenance etc.

CX LOSA Cathay Pacific Airways 2003 LOSA.10


Side 11

Undesired The aircraft is placed in a


compromising situation
Aircraft States that reduces safety
margins (LOSA Report)
U.A.S.
2/3 of these were linked to a threat
1/4 were linked to weather (June in SE Asia!)
Aircraft Handling
Speed; vertical & lateral deviations; weather penetration
Descent/Approach/Landing
Unstable approach; continued landing; long/short landings
Incorrect Configuration
Systems; flight controls; automation; or engine
More examples
Late configuration; wrong heading/taxiway; no anti-ice; flap overspeed

CX LOSA Cathay Pacific Airways 2003 LOSA.11


Side 12

Moving On
LOSA produced no major surprises
(command leadership / no intentional non-compliance / cabin threats)

Verifiable data supporting Targets for Enhancement


(took time - up to 12 months - to digest the report)

Required deeper analysis to achieve implementation


(LOSA project manager to produce specific recommendations)

Posted abbreviated LOSA report on FOP Website


(Full hardcopy report available for all aircrew to read)

LOFT with Threat and Error concepts and components


(briefing / debriefing built on LOSA results & TEM principles)

CX LOSA Cathay Pacific Airways 2003 LOSA.12


Side 13

Moving On
Cathay considered LOSA as one of a number
of safety-measurement inputs.

LOSA is not a stand-alone program.

FDAP (bi-monthly analysis and reports)


Undertook ALARS (Flight Safety Foundation) Toolkit
evaluation
Flightcrew Training Reports (under constant review)

CX LOSA Cathay Pacific Airways 2003 LOSA.13


Side 14
Moving On
We used LOSA to validate data from other sources
- saw similar indicators elsewhere

Adherence to SOPs
Workload management Automation management
Monitor / cross- check Outdated briefings
ATC threats (exhaustive and by rote)
CX LOSA Cathay Pacific Airways 2003 LOSA.14
Side 15
Moving On

LOSA together with other data


has driven some changes:

New stabilised approach criteria

New SOP for Flap and Gear selection

New take-off and approach briefings


(efficient and relevant name the threat!)

New standardised approach and landing callouts

Introduction of Threat & Error Management program

CX LOSA Cathay Pacific Airways 2003 LOSA.15


Side 16
Looking Ahead
Life is lived forward but understood backward.
Kierkegaard

Everyone wants hindsight in advance!

LOSA is
ensuring safety comes first
in Cathay...

LOSA results, integrated with a TEM culture, give us a chance


to see some things before they happen.

CX LOSA Cathay Pacific Airways 2003 LOSA.16


AIR NEW ZEALANDs LOSA
PROGRAMME

Captain Chris Kriechbaum Captain Julian Alai


CRM Programme Manager LOSA Project Manager
LOSA Programme

1998
2001 - aborted
2003
Draft report stage
Final Report December 2003

Dublin LOSA Week


Air NZ - our vitals then and now!!
660 pilots, 1700 cabin crew - a small airline!
Two pilot union groups
All Boeing fleets, 7 x 747, 13 x 767, 16 x 737
Mix of glass (744 & 767) and non glass (742 & 732)
Now all glass
Long ( 14hrs max) & Short (.5 hrs min.) haul mix
Sectors audited - 97/1998, 200/2003
No QAR data - then or now
A 320 Fleet implementation imminent
Dublin LOSA Week
Business Rationale behind the 1998
LOSA
An observation of crew in the real environment
Pro-active approach to safety.
Required facts not opinions.
Wanted an international comparison of our CRM
programme.
Needed a threat and an error bench mark.
Needed a safety snapshot from a crew
perspective
Down under inferiority complex??
Dublin LOSA Week
1998 Data
The main errors included:
Failure to cross-verify
52% of flights had at MCP/FMC/CDU settings
least one error Omitted or incomplete F/A
78 separate errors briefings
recorded on a total of Incorrect settings / frequencies
91 flights selected
Between Zero and 5 Incomplete cockpit briefings
errors per flight
Checklist omitted, incomplete, or
Average of 0.86
done from memory
errors on every flight
Consequential error Failure to set altitude alerter
rate high Missed ATC calls or incorrect
read back
Dublin LOSA Week
1998 RESULTS
CRM behavioural markers generally positive

Consequential error rate, the incentive for change

Automation issues were a dominant theme

Recommendations for procedure changes

Culture issues

Dublin LOSA Week


1998 CHANGES
9Change in the Pilot/Management culture
- Just Culture Introduction
9Skill improvement in error management
- TEM Training
9Leadership training for Captains
- Enhanced Command Course
9Improve F/A briefings
- Combined EP training
9 Terrain & Altitude awareness training
- Improved Situational Awareness Modules
9Verification of FMC/MCP settings
- SOP Changes
Dublin LOSA Week
THE BENEFITS
Established a baseline
Elevated CRM profile with in the Company
Credibility in International Aviation Community
Removed the scatter gun effect of training
Combined Training
Targeted Error Management re- focus
Updated Command Course
SOP changes
Just Culture
Value for money
Justification of Capital and other Expenditure.

Dublin LOSA Week


Motivation behind the 2001 LOSA

Ansett Australia purchase & amalgamation of


both Flight Operations.
Base-line measurement before merging.
Measure the success of operational changes
since the 1998 audit.
Aborted, due to Ansetts being put into
receivership.

Dublin LOSA Week


THE 2001 EXPERIENCE

Selection of observers

- Involved union input

- Training Managers matrix

Set schedule for sectors

Dublin LOSA Week


Business Rationale behind the 2003
LOSA
Make a comparison of the operational safety
health of the operations system with that obtained
in 1998
Base-line before the A320 implementation.
Attempt to measure our risk profile
Provide further guidance towards future
improvements.
Timing - a window of opportunity before the new
A320 fleet.
Dublin LOSA Week
Convincing the Bean-counters!

THE BUSINESS CASE


Financial scrutiny this time round was intense.
Long term approach was adopted, especially in
terms of potential financial benefits.
Change in ANZ Operations management
- Politics due to external pressures, was higher
- Bigger promises made
- Final sign-off was only 10 days before start
LOSA Project Manager appointed - Julian Alai
Dublin LOSA Week
THE 2003 EXPERIENCE

Observer selection
- Involved union input
- Refined selection matrix
- Mix of internal and external observers
- Pilot and non current pilots
In depth observer training, including a
NZCAA representative.

Dublin LOSA Week


LESSON LEARNED

Use of non pilots - both good and bad in this.


Time frame was too short - created pressures
Too much publicity?? - reject rate higher than
anticipated
Methodology in 98 was significantly different
than in 2003.
Set schedule for sectors - consider a more
flexible method
Comparisons 98 to 03, were difficult to make.
Significantly more information to work with.

Dublin LOSA Week


WHERE TO FROM HERE?

Consideration of integrating LOSA outputs


with other safety data
Mini audits planned
Possible joint audits with other airlines
Pooling of observers
Keep all observers current

Dublin LOSA Week


Contact Details

Captain Julian Alai


julian.alai@airnz.co.nz

Captain Chris Kriechbaum


chris.kriechbaum@airnz.co.nz

Dublin LOSA Week


Dublin 5 November 2003
Capt. Morten Ydalus/Capt. Hvard Vestgren
Braathens Flight Crew Dept.
Brief Overview

S2
Why LOSA in Braathens?

S3
Need to know facts
Establish status of crew performance:
Proficiency
Desicionsmaking
CRM skills
Loyalty to procedures

Establish status in relation to system performance:


Culture
Airspace systems airports / navaids
Aircraft design / automation
Procedures
Flt. Standards / Training / maintnance
Crew support: ATC, Cabin; Ground, Flight Watch

S4
Turbulent years

Braathens Went through some very


difficult years with almost bankruptcy,
large personel cutbacks, and a
takeover by the traditional competitor
What do these facts mean for
Operational quality and safety?

S5
LOSA

S6
Preparation phase

Must set up proper organization


Planning is essential
Positive relations with the Union
Selection of observers
Diverse group
Info to Pilots
LOSA Newsletters, hard copy and on Intranet
Observer training
Very important and intense
Logistical issues
Tickets, Id cards
S7
174 Observations Jan - Feb
Winter Operations

All observers used Laptops


New data program from TLC
Still each observation took 2-3 hours to record
Secure transfer of data direct to TLC
Worked well
Web survey answered by appr. 1/3 of
the pilot group

S8
Data Cleaning in Austin

Lot of work
Union present
Very educational
Got a feel for all the data before The Report

S9
Report

Very Thorough
Lot of data
TLC good presentation to management

S10
After Report

S11
Threats
Passenger events
Distractions ATC
Cabin Crew
Terrain
Weather

Maintenance Similar call sign

Ground Crew Time pressures

Flight
Heavy traffic
diversion

System malfunction
Unfamiliar airport

Automation event Missed approach

S12
Results - Threats

4.7 threats pr flight, very high relative to other LOSA


Archive airlines
50% of these were due to Adverse Weather, 20%
due to Airport and Ground Conditions (Winter
operations)
Braathens has one of the best Threat management indexes ever
observed by the LOSA Collaborative. Braathens Crews are excellent
Enviromental Threat Managers.
Very few threats observed from Dispatch and Cabin
Crew
Aircraft malfunctions / MEL items
Some ATC threats

S13
Errors
HUMANS MAKES MISTAKES!

Examples of errors:
Crew forgot to select autobrake RTO
Premature level off
Not adhering to ATC clearance

S14
Results - Errors
Rate of proficiency errors was very low, suggesting
no apparent knowledge or skills deficit
Procedural drift, the inevitable adaption of
procedures to better suit the temporal contraints of
the local enviroment.
Half the errors logged were intentional
noncompliance errors, 75% had no consequential
outcome
25% of the intentional non-compliance errors led to
new errors or an Undesired Aircraft state
80% of all Aircraft handling errors were mismanaged

S15
Results - Procedures
Descent/approach/land contains most threats and errors
Rank does not appear to affect crew performance
Threat managment slightly better when Captain flying
No effect of rank on error occurence or management
Braathens crews were very experienced, crews with least
experience made fewer errors.
Delays had no effect on crewperformance, rather opposite
Web survey identified several automation traps and areas of
confusion between 737 NG and Classic
Web survey identified a wish for more training in Turbulence and
Extreme WX conditions
Intentional Non-Compliance:
OPS, FMA Callouts, MA construction, decend wind entries
Unstable Approaches
Deviations from defenition caused by, terrain GP/PAPI transitions, short rwys,
landing techniques / procedures
Aircraft malfunctions / MEL items

S16
Sensitive data

Info to pilot group


Pilot meetings
Intranett
Info on TEM courses
Difficult with printed info (sensitive
information we do not want to have
lying around)
Top Management reluctant to release
data except to working groups
What if press gets hold of it?

S17
Work groups

S18
Work Groups
Flight Operations / Procedures
New / modified flight procedures
New Flight Manuals (electronic)
Stabilised approach project
z New data module on Line Checks, recording approach
parameters
z FOQA collaboration/FOQA issues
Technical
Mel issues
CDL issues
Automation issues
TEM

S19
TEM in Braathens

S20
CRM/Leadership

TEM course from 1. Dec 2003


1 day all flight crew
CCM on PGT
Developed with help of Don Gunther
Using actual LOSA flights as cases

S21
Safety Change Process

S22
Braathens Safety Change Process

1. Statistical
7. Install 8.Re- analysis
changes Measure

6. Approve Safety 2. Change


changes authority &
Change responsibility

5. Risk
assessment / 3. Process
priorities strategies
4. In-depth
analysis

S23
10 Operating Characteristics

S24
Ten Operating Characteristics
(Doc 9803)

1. Jump seat observations during normal flights


2. Voluntary crew participation
3. Confidential, safety-minded data collection
4. Joint airline-pilot sponsorship
5. Targeted observation instrument
6. Trusted, trained, standardized observers
7. Trusted data collection site
8. Data cleaning round tables
9. Data-derived targets for enhancement
10. Feedback of results to line pilots

S25
Did we comply?

On the 9 first completely


On results to Pilots, not as well as we
would have liked, but we are still
working on it

S26
Future

S27
Thank You!

S28
LOSA FUTURA
LOSA
experience within
FUTURA
First ICAO-IATA LOSA &TEM Conference
Dublin, Ireland
November 2003
Cptn
Cptn.. Robert
Robert Aran i Escuer
Escuer
Responsible
Responsible for
for CRM
CRM && Human
Human Factors
Factors
Compaa
Compaa Hispano-Irlandesa de
Hispano-Irlandesa de Aviacin
Aviacin (FUTURA)
(FUTURA)

Gabinete de Seguridad Unidad CRM y FFHH


Due to its own nature LOSA would never be
applicable to organizations with pathologic
LOSA FUTURA
characteristics, being barely efficient in those
considered bureaucratic for their reaction
capacity is extremely slow..

1
Three lessons learned by FUTURA
Gabinete de Seguridad Unidad CRM y FFHH
Those organizations whose activity is only
focused on the survival of its project due to
LOSA FUTURA
the lack of resources or because their goal is
short-term profitability or because they are
based on labor disputes, latent or active, are
not the ideal ground for LOSA.

2
Three lessons learned by FUTURA
Gabinete de Seguridad Unidad CRM y FFHH
In spite of certain weaknesses, LOSA is the
most powerful proactive safety tool ever
LOSA FUTURA
designed to sustain safe operations at the
highest possible level of an organization

3
Three lessons learned by FUTURA
Gabinete de Seguridad Unidad CRM y FFHH
LOSA FUTURA
The required cultural change

Management compromise with Human Factors.


Survey conducted to determine the Organizational
Status.
Adaptation of CRM Training to JAR requirements.
FOQA program implementation.
Flight and Cabin crews joint CRM practices.
MRM implementation.

LOSA implementation.
Introduction of TEM workshops and practices.

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
LOSA Program Chronology
Initial approach to the LOSA concept (UT papers)
LOSA Week held in Panama
First TEM Workshop held in San Salvador

Signature of LOSA Protocol by Pilot Trade Unions


Training Course for LOSA Observers
Creation of the LOSA Committee in Futura
Creation of the LDB (LOSA Data Base)
LDB Training for Observers.
LOSA publicity on the Intranet and mailing to all
pilots.
Starting of the Observation.
Gabinete de Seguridad Unidad CRM y FFHH
LOSA FUTURA

Three key points for


the implementation
of LOSA
within Futura
Gabinete de Seguridad Unidad CRM y FFHH
LOSA FUTURA
1. Signature of LOSA Protocol
LOSA REF -UCRM-P01

PROTOCOL PAGE 1

LINE OPERATIONS SAFETY AUDIT

1. INTRODUCTION:

This Protocol, whose signature by the representatives of the


different Pilot Trade Unions will validate the implementation
of LOSA in the Company and introduces the bases of a
program to manage human error in aerial operations. That
program hereinafter called LOSA (line Operations Safety
Audit) is proposed by the Safety and Security Office as the
basis for the development of measures to counteract
operative errors.

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
2. Creation of LOSA Data Base
Some Search Criteria in Futuras LDB

Non-stabilized approaches, according to: Airport and Number of flights


per Airport.
Errors, according to Airport, phase of Flight , type of Error and code.
Errors Committed and Detected according to: Who makes the Error,
Who detects the Error, Airport and phase of Flight.
Flight
Flight Threats, according to Airport, phase of Flight, kind of Threat and
Code.
Undesired Aircraft States, according to:
Airport, phase of Flight, Undesired Aircraft State type and code.
Comparison of Errors, Flight Threats and U-A-S, according to the
Airport.
Behavioral markers, according to the phase of Flight.
General Crew Efficiency
Gabinete de Seguridad Unidad CRM y FFHH
0 2 0
Non-Stabilized Approaches
LOSA FUTURA
0 0 0 MAN 1 0 0 0
0 0 0 MXP 1 0 0 0
0 0 0 NWI 2 0 0 0
8 ORK 3 0 0 0
7 0 0 0
SNN 1 0 0 0
6 0 0 0
5 TMP 1 0 0 0
N Flights
4
Errors 500
3
N Flights Errors 500Errors 1000 Errors 1500 Errors 1000
2 7 0 0 0
6 0 0 0 Errors 1500
1 8 1 0 0
0 3 0 0 0
ACE
8 AGP5 LPA 2 PMI 0TFS FUE DUB
5 0 0 0
Airport
6 0 1 0
Supuesto ficticio. DEMO BDL-FUA

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
Error Code (Todas)
Number of Errors and Type according
Suma de Errors to the phase of Flight
35

30

25 Error Type
Intentional / no compliance
20
Procedural

15 Communication
Decision
10 Proficiency

0
Predepart / Taxi Takeoff / Climb Cruise Des / App / Land Taxi-in

Phase of flight
Supuesto ficticio. DEMO BDL

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
3. Observers Training

It is essential a high level of standardization as well as


the maximum knowledge of:

Standard Operational Procedures (SOPs)

The Aircraft Type

The philosophy and functioning of ICAO Doc 9803

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
LOSA adaptation
Doc 9803 AN/761 by ICAO contains the basic documents
for its implementation, however each organization should
adapt LOSA depending on :

The Aircraft Type

The Observation Purpose

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA

LOSA
as an Integral Tool for
Safety

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
Datos Generales de la Observacin LOSA-FUTURA
Febrero - Marzo 2003

N Vuelos

N Crews

% Crews

Vuelos Largos

Medios

Cortos

0 10 20 30 40 50 60 70 80

Gabinete de Seguridad Unidad CRM y FFHH


More than 400 LOSA data obtained during the last LOSA
observation, have proved its utility to
LOSA FUTURA

Eliminate, modify or develop SOPs and checklists items


Redesign training modules for:
-Dispatchers
-Mechanics
-Cabin crew
-Pilots
Learn how crews manage threats
Learn why specific events took place
Learn how crews manage errors
Design TEM training based on operational data
Gabinete de Seguridad Unidad CRM y FFHH
LOSA FUTURA
The future of LOSA within Futura
JOINT WORK
FOQA LOSA
Error Code (Todas)

Suma de Errors

35

30

25 Error Type

Intentional / no compliance
20
Procedural
15 Communication
Decision
10
Proficiency
5

0
Predepart / Taxi Takeoff / Climb Cruise Des / App / Land Taxi-in

Gabinete de Seguridad Unidad CRM y FFHH Phase of flight


LOSA FUTURA
After LOSA observations, Futura will
implement the Organizational
corrective actions

but, what to do with all these


data of Systemic nature?

Gabinete de Seguridad Unidad CRM y FFHH


LOSA FUTURA
Systemic Nature Threats

ATC

Slots

Aeropuerto

Callsigns

Total

0 5 10 15 20 25 30 35

Porcentaje N de Eventos
Gabinete de Seguridad Unidad CRM y FFHH
LOSA FUTURA

FUTURA
Safety Cabinet
CRM & Human Factors Unit

unidadcrm@futura-aer.com

Gabinete de Seguridad Unidad CRM y FFHH


ICAO-IATA LOSA & TEM Conference Dublin,Ireland, 5 to 7November 2003 Collecting Safety Data from In Service Occurrences and Learning
from them
l Where did we learn from?

4 Goldmines of information if put to good business practice


Presented by l What did we deliver ?
4 Briefing Notes
4 FCOM / QRH enhancements
4 Operational and Human Factors Studies
J-J Speyer
Director Operational Evaluation,
Human Factors and Communication l What did we learn ?
4 Lessons -learned in design, operations, training

AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.
Collecting Safety Data from In Service 4 Assessing Robustness of Operational & Training Assumptions
Prevention Strategies and Safety Awareness Information
Occurrences and Learning from them 4

l Where are we going ?

by J-J Speyer and Michel Trmaud, Airbus Customer Services 4 Threat and Error Management

4 Continued Flight Safety Enhancement


Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference

Collecting Safety Data from In Service Occurrences And learning from them through proper analysis

Goldmines of information if put to good business practice Need for Analytical Methods and Tools ( GAIN WG B )

COLLECT
COLLECT
l Identify reports with similar characteristics,
ANALYZE
ANALYZE
1 l Extract information from reports in a structured way,
FORMULATE AND
FORMULATE AND
IMPLEMENT
IMPLEMENT
l Derive patterns and trends in large amounts of data,
AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

MONITOR
MONITOR l Identify important , contextual and event sequence data,

4DATA and TEXT MINING METHODS,


From Role of Analytical Methods and Tools ( GAIN )
4Air Safety and Human Factors Reports, FOQA Data,
Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference

Need for Multiple Reporting Sources Need for Analysis Workflow

Understand the
Understand the Facts
Facts
ISOs What ?? Why
What Why ??

Operational Review Applicable


Applicable
Review
Support Feedback
Standards
Standards
Events
Training
Feedback Identify Operational
Identify Operational
and Human
and Human Factors
Factors

FOM Feedback ASRS,


LOMS / AIRS / LOAS
BASIS SIE, Identify
Identify
Lessons-learned
Lessons-learned
AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

Confidential
Reports
Problems Statements Develop
Develop
Contributing Factors Prevention Strategies
Prevention Strategies
Other
Recommendations Events

Develop Safety
Develop Safety
Awareness Tools
Awareness Tools

Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference

Page 1
Need to reach Informed Decision Making Need to look Beyond Reported Events : Prevention

l Events categorization : Passenger aircraft, At 37 000 ft


Avionics Smoke warning + Cargo Compartment Fire

4Event consequences Heavy smoke in cockpit


Crew sequence of actions ?

4Event descriptors : Freighter aircraft, In final approach

System of function affected Avionics Smoke warning + Cargo Compartment Fire


Heavy smoke in cockpit
Condition descriptor No crew action, landing continued
runway vacated and aircraft evacuated

l Operational and human factors markers


4Subtly intertwined and embedded into one another
AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.
l Assessing robustness of :
4Operational assumptions Reported Potential
Event Event
4Training assumptions ( Prior - Precursor )

Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference Collecting Safety Data from In Service Occurrences and Learning from them ICAO-IATA LOSA & TEM Conference

Back to Analysis Workflow Operational and Human Factors Analysis

Understand the
Understand the Facts
What ?? Why
Facts
Why ??
l Tailored to Airbus needs as aircraft manufacturer :
What

4 Identify recurring factors


Review Applicable
Review Applicable
Standards
Standards 4 Identify recurring patterns
4 Identify lessons -learned
Identify Operational
Identify Operational
and Human
and Human Factors
Factors
2 4 Develop prevention strategies
Identify
Identify
Lessons-learned
4 Support lessons-learned process
( design, procedures, training )
Lessons-learned
AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

AIRBUS 2003 S.A.S. All rights reserved. Confidential and proprietary document.

Develop
Develop
Prevention Strategies
Prevention Strategies l Markers optimised for compatibility with :