Threat and Error Management

James Klinect
The University of Texas Human Factors Research Project
IFALPA Caribbean / South America Regional Conference November 19th, 1999

Presentation Outline

Define error management

External threats: An overlooked concept in error management Threat and error management model

Line Operations Safety Audit (LOSA): Data source for threat and error
General LOSA results Behavioral markers Summary The value of threat and error management

What is Error Management?

Error Deviation from organizational expectations or crew intentions Error management - process of correcting an error before it becomes consequential to safety
Error Management

Error

Error Response

Error Outcome

An Overlooked Concept
External Threats safety threatening events that originate outside of the cockpit
Adverse Weather Terrain Aircraft malfunctions Poor airport conditions Abnormal operations Operational pressures ATC event or error Maintenance event or error Ground event or error Dispatch event or error Cabin event or error

Threats increase the operational complexity for a flight

Threat and Error Management Model

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management

Error

Error Response

Error Outcome

Data Sources for Threat and Error

Accident Reports
Problems 1. 1.5 accidents per million departures (not enough data) 2. Retrospective data only reflects failed system or crew performance

Incident Reports
Problems 1. Reports are subjective, first person accounts 2. Retrospective data

Need objective data that highlight both effective and ineffective threat and error management

Line Operations Safety Audit (LOSA)

Non-jeopardy observations of flight crew performance during normal operations
Union supported One to three months of observation Team of airline and U.T. project observers (10-15) Data do not identify pilots and are confidential

Measures:
External threats and their management Flight crew errors and their management CRM behavioral markers Pilot suggestions through interviews Latent system failures

LOSA Demographics
Data taken from three airlines
184 flight crews 314 flight segments

External Threats

LOSA External Threat Results

72% of the flights had at least one external threat
Two external threats per flight Most external threats on a flight = 11 Most Frequent Threats for an airline that primarily flew into Latin America
Adverse weather 34% of the flights

ATC events or errors 34%

Aircraft malfunctions 15%

External Threats by Phase of Flight

Phase of Flight
Pre-Departure / Taxi Takeoff / Climb Threats by Phase 22% 28% Most Frequent Threats by Phase
A/C malfunctions Ground events / error Operational pressures

Cruise
Descent / Approach / Land Taxi / Park

10%
39% 1%
Terrain Adverse weather ATC events and errors

Summary - External threats in the pre-departure phase are much different than those in the most threatening phase (descent / approach / land)

Flight Crew Error

Threat and Error Management Model

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management

Error

Error Response

Error Outcome

LOSA Error Results

64% of the flights had an error

Two errors per flight Most errors on a flight = 14

Most common flight crew errors

Automation errors 31% of all errors Wrong MCP and FMC modes and settings Failure to cross-verify Checklist errors 21% of all errors Checklist performed from memory Nonstandard usage Missed items

Error by Phase of Flight

Phase of Flight Pre-Departure / Taxi Threats by Phase 22% Error by Phase 23%

Takeoff / Climb
Cruise Descent / Approach / Land Taxi / Park

28%
10% 39% 1%

24%
12% 39% 2%

Summary - The most threatening phase (descent / approach / land) also contains the most error

Expanding the Model with Error Types

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management

Error Intentional Noncompliance Procedural Communication Proficiency Decision

Error Response

Error Outcome

Error Types
1. Intentional Noncompliance violations

ex) Performing a checklist from memory

2. Procedural Followed procedures but wrong execution

ex) Wrong altitude setting dialed into the MCP

3. Communication Missing information or misinterpretation

ex) Miscommunication with ATC

4. Proficiency Error due to a lack of knowledge

ex) Lack of knowledge with automation

5. Decision Discretionary crew decision that unnecessarily increased risk

ex) Unnecessary navigation through adverse weather

Error Frequencies
Noncompliance Procedural Communication Proficiency Decision 0
6 5 6 29 54

20

40

60

80

100

Percent Frequency

Summary Willful violations (intentional noncompliance errors) are the most frequently observed errors

Expanding the Model with Error Responses

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management

Error

Error Response Trap Exacerbate Fail to Respond

Error Outcome

Error Responses
Trap 36% of all error responses
Error is successfully detected and managed

Exacerbate 11%
Error is detected but mismanaged to produce a consequential outcome

Fail to Respond 53%

Error is not detected therefore not managed

Expanding the Model with Outcomes

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management

Error

Recovery

Error Response Undesired Aircraft State

Additional Error

Error Outcomes
Inconsequential 85% of all errors
Consequential Outcomes 15% of all errors Additional Error 3% Undesired Aircraft States 12%

Undesired Aircraft States

Undesired Aircraft States Flightcrew induced deviations from normal flight that threaten safety
Lateral deviation Vertical deviation Speed too high or low Unstable approach Near miss Fuel level below minimums Long landing Hard landing Landing off centerline Wrong taxiway or ramp Wrong runway Wrong airport or country

Error Frequencies and Consequences

Noncompliance Procedural Communication Proficiency Decision 0
6 5 6 13 69 43 2 23 29 54

20

40

60

80

100

Percent Consequential

Percent Frequency

Summary Proficiency and decision errors are the most difficult to manage

Intentional Noncompliance as a Norm

One LOSA observer noted the following during a IOE ride The check airmen ran the entire taxi checklist from memory. Bad example to set.
Airlines cannot allow violations to normalize Why? It cultivates complacency and a disregard of rules Crews that commit at least one intentional noncompliance error are two times more likely to: Commit non-intentional errors Have non-intentional errors with consequential outcomes

The Final Expansion: State Responses

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management Error

Recovery

Error Response Undesired Aircraft State Undesired Aircraft State Response Mitigate Exacerbate Fail to Respond

Additional Error

Error Induced Incident or Accident

Undesired Aircraft State Management

When an aircraft enters an undesired aircraft state, flightcrews manage the state not the error
Undesired aircraft state responses
Mitigate 79% of all error induced aircraft state responses Exacerbate 2% Fail to respond 12%

Threat and Error Management Behavioral Markers

Behavioral Markers
After conducting observations on 3800 flights across seven airlines......
Fifteen behavioral markers have been determined to be the most relevant in threat and error management The fifteen markers can be divided into four groups
1. 2. 3. 4. Team Climate Planning Execution Review / Modify

Team Climate Markers

Leadership
Captain showed leadership and coordinated flight deck activities

Communication Environment
Environment for open communication was established and maintained

Flight Attendant Briefing

If performed, did the flight attendant brief address safety concerns and crew coordination during emergencies

Planning Behavioral Markers

Briefing
Required briefing was interactive and operationally thorough

Contingency Planning
Crew members developed strategies to manage threats before they encountered them

Workload Assignment
Roles and responsibilities were defined for normal and non-normal situations

Plans Stated
Operational plans and decisions were communicated and acknowledged by other crew members

Execution Behavioral Markers

Monitor / Cross-Check
Crew members actively monitored and cross-checked systems and other crew members above and beyond procedures

Workload Management
Operational tasks were prioritized and properly managed to handle primary flight duties

Vigilance
Crew members remained alert of the environment, safety threats and the progression of the flight

Automation Management
Automation was managed to balance situational and/or workload requirements

Review / Modify Markers

Evaluation of Plans
Existing plans were regularly reviewed and discussed

Inquiry
Crew members asked questions to investigate and/or clarify current plans

Assertiveness
Crew members stated critical information and/or solutions with the appropriate persistence

Adaptability
Existing plans were altered under contingency situations in a timely manner

One Last Look At The Model

External Threats Threat Induced Incident or Accident

Inconsequential

External Threat Management Error

Recovery

Error Response

Additional Error

Undesired Aircraft State

Error Induced Incident or Accident

Undesired Aircraft State Response

The Value of Threat and Error Management

Reinforces the goals of CRM training
1. Manage external threats to safety 2. Avoid flight crew error 3. Manage flight crew error

Encourages the examination of system defenses that help flight crews manage threat and error
Aircraft design, technology, and manufacturing Regulations and procedures

Training (Proficiency and CRM)

Line support - ATC, Maintenance, Ground Handling, Dispatch and Cabin

Our Website

www.psy.utexas.edu/psy/helmreich/nasaut.htm