Está en la página 1de 41

Assessing & Measuring Operational Risk Why COSO is Inappropriate

ISDA PRMIA
London, United Kingdom January 18, 2005 Ali Samad-Khan President, OpRisk Advisory LLC ali.samad-khan@opriskadvisory.com www.opriskadvisory.com

Agenda

I II III IV V VI VII

Introduction Definition and Categorization COSO Based Risk Assessment Integrated Risk Measurement and Management Alternative Approaches to Risk Assessment Control Assessment Summary & Conclusions

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

INTRODUCTION

When considering risk and control assessment, what are the priorities?

Establish a disciplined process. Its the process that matters, the results are less important A good process lays the foundation for a good risk management culture Establish a process that will produce the most reliable results. The results are more important If it is clear to end users that the results are fictitious then the entire risk management program will be discredited; the operational risk program will be seen to be adding little value Demonstrate practical value and the program will be a success and subsequently create the right culture

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Consider introducing a disciplined process that produces accurate results which facilitate educated decisions making.

RISKS

CONTROLS

What type of risks do I face?

Which are the largest risks?

How well are these risks being managed?

Manage Controls through Cost Benefit Analysis

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

To ensure the goal is practical, one needs to express it in the context of a business problem

Consider two risks: Unauthorized Trading and Money Transfer Past Audits reveal that both risks are under-controlled To address Unauthorized Trading risk one must improve segregation of duties and audit frequency. (Solution: hire four new staff; cost = $600,000 per year) To address Money Transfer risk one must improve the system (Solutions: buy new system; cost = $5 million) You have $4 million in your budget. Where do you invest your money?

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Effectively managing operational risk requires a foundation designed to turn raw operational risk data into information that supports managerial decision making

MANAGEMENT
Awareness of real exposures

INFORMATION

Economic Profit

DATA
Loss data collection Risk indicator data collection Control selfassessment Risk assessment and analysis Automatic notification Follow up action reports

Expected Loss how much do I lose on average Unexpected Loss how much I could reasonably expect to lose in a bad year Control Scores how good are the controls I have in place

Knowledge of controls quality Cost benefit analysis Improved risk mitigation and transfer strategy

FOUNDATION
Risk strategy, tolerance
Roles and responsibilities Policies and procedures Risk definition and categorization

Management & Control Quality


Copyright 2004, OpRisk Advisory LLC. All rights reserved.

DEFINITION AND CATEGORIZATION

What does operational risk include?

Transaction
Inadequate Supervision

Execution
Information

Settlement
Key Man Theft

Technological
Lack of Resources

Reputation
Insufficient Training Compliance Poor Management

Relationship
Fraud People Fiduciary Legal/Regulatory Customer Fixed Cost Structures Business Interruption

Criminal
Rogue Trader Physical Assets Business Strategic

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

The universe of operational risks spans causes, events and consequences

CAUSES
Inadequate segregation of duties Insufficient training Lack of management supervision Inadequate auditing procedures

EVENTS

CONSEQUENCES
Legal Liability

Internal Fraud External Fraud Employment Practices & Workplace Safety Clients, Products & Business Practices Damage to Physical Assets Business Disruption & System Failures

Regulatory, Compliance & Taxation Penalties

Loss or Damage to Assets Restitution

EFFECTS Monetary Losses

Inadequate security measures

Loss of Recourse


Poor systems design Poor HR policies

Write-down

Reputation Execution, Delivery & Process Management Business Interruption

OTHER IMPACTS Forgone Income

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Placing loss data within a Business Line/Risk matrix helps reveal the risk profile of each business

INTERNAL FRAUD Corporate Finance Number Mean Standard Deviation Trading & Sales Number Mean Standard Deviation Number Mean Standard Deviation Commercial Banking Number Mean Standard Deviation Number Mean Standard Deviation Agency Services Number Mean Standard Deviation Number Mean Standard Deviation Retail Brokerage Number Mean Standard Deviation Number Mean Standard Deviation Total Number Mean Standard Deviation 362 35,459 5,694 50 53,189 8,541 45 47,870 7,687 41 43,083 6,918 37 38,774 6,226 44 46,529 7,472 40 41,876 6,725 48 50,252 8069 43 45,226 7,262 710 45,653 7,331

EXTERNAL FRAUD 123 52,056 8,975 4 78,084 13,463 4 70,276 12,116 3 63,248 10,905 3 56,923 9,814 4 68,308 11,777 3 61,477 10,599 4 73,773 12719 4 66,395 11,447 152 67,021 11,555

EMPLOYMENT PRACTICES & WORKPLACE SAFETY 25 3,456 3,845 35 5,184 5,768 32 4,666 5,191 28 4,199 4,672 26 3,779 4,205 31 4,535 5,045 28 4,081 4,541 33 4,898 5449 30 4,408 4,904 268 4,450 4,950

CLIENTS, PRODUCTS & BUSINESS PRACTICES 36 56,890 7,890 50 85,335 11,835 45 76,802 10,652 41 69,121 9,586 37 62,209 8,628 44 74,651 10,353 40 67,186 9,318 48 80,623 11182 43 72,561 10,063 384 73,245 10,158

DAMAGE TO PHYSICAL ASSETS 33 56,734 3,456 46 85,101 5,184 42 76,591 4,666 37 68,932 4,199 34 62,039 3,779 40 74,446 4,535 36 67,002 4,081 44 80,402 4898 39 72,362 4,408 351 73,044 4,450

EXECUTION, DELIVERY & PROCESS MANAGEMENT 150 1,246 245 210 1,869 368 189 1,682 331 170 1,514 298 153 1,363 268 184 1,635 321 165 1,472 289 198 1,766 347 179 1,589 312 1,598 1,604 315

BUSINESS DISRUPTION AND SYSTEM FAILURES 2 89,678 23,543 3 134,517 35,315 3 121,065 31,783 2 108,959 28,605 2 98,063 25,744 2 117,675 30,893 2 105,908 27,804 3 127,090 33365 2 114,381 30,028 21 115,459 30,311

TOTAL 731 44,215 6,976 398 66,322 10,464 360 59,690 9,417 322 53,721 8,476 292 48,349 7,628 349 58,018 9,154 314 52,217 8,238 378 62,660 9886 340 56,394 8,897 3,484 56,926 8,981

Retail Banking

Payment & Settlements

Asset Management

Insurance

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

COSO BASED RISK ASSESSMENT

Risk can also be assessed using a likelihood-impact approach. This approach has been well documented by the Committee of Sponsoring Organizations of the Treadway Commission (COSO).

Source: COSO
Copyright 2004, OpRisk Advisory LLC. All rights reserved.

The COSO view of risk assessment is based on the likelihood and impact of a specific type of event; the output is probability weighted impact. The high risk area is in the top right corner of the matrix.

COSO
High (3)

3 2 1
Low (1)

6 4

9 6 3
High (3)

LIKELIHOOD

Med (2)

Low (1)

COSO 2
Med (2)

IMPACT

Likelihood x Impact = Risk


Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Under the risk management industry approach, the high risk area is the bottom right cell in the matrix.

Risk Management Industry


High (3)

n/a

n/a n/a

LIKELIHOOD

Med (2)

Low (1) Low (1)

COSO
Med (2) High (3)

IMPACT

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

When compared, there are significant differences .

Risk Management Industry

COSO
Phantom Risks

High (3)

n/a

n/a Likelihood n/a

High (3)

3 2 1
Low (1)

6 4

9 6 3
High (3)

Likelihood

Med (2)

Med (2)

Low (1)

COSO
Low (1) Med (2) High (3)

Low (1)

COSO 2
Med (2)

Real Risks

Impact

Impact

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Under the COSO approach one calculates risk through likelihood-impact analysis

Likelihood x Impact = Risk


Risk 1 : 10% x $10,000 = $1,000

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Likelihood-impact analysis can yield more than one result

Likelihood x Impact = Risk


Risk 1 : Risk 2 : 10% x $10,000 = $1,000 1% x $50,000 = $ 500

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Using likelihood-impact analysis one can calculate multiple outcomes

Likelihood x Impact = Risk


Risk 1 : Risk 2 : 10% x $10,000 = $1,000 1% x $50,000 = $ 500 . . . . Risk 999 : 5% x $25,000 = $1,250 Risk 1000 : 20% x $ 6,000 = $1,200

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

The many probability and impact combinations represent a continuum

20% x $ 6,000

10% x $10,000 5% x $25,000 1% x $50,000

0-10

1020

2030

3040

4050

Impact

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

INTEGRATED RISK MEASUREMENT AND MANAGEMENT

Risk is measured using internal and external loss data. The two measures of exposure are the aggregate mean and aggregate Value at Risk (VaR).
INDIVIDUAL LOSS EVENTS RISK MATRIX FOR LOSS DATA LOSS DISTRIBUTIONS VAR CALCULATION TOTAL LOSS DISTRIBUTION

74,712,345 74,603,709 74,457,745 74,345,957 74,344,576

Frequency of events

INTERNAL FRAUD Corporate Finance Number Mean Standard Deviation Trading & Sales Number Mean Standard Deviation Number Mean Standard Deviation Commercial Banking Number Mean Standard Deviation Number Mean Standard Deviation Agency Services Number Mean Standard Deviation Number Mean Standard Deviation Retail Brokerage Number Mean Standard Deviation Number Mean Standard Deviation Total Number Mean Standard Deviation 36 35,459 5,694 50 53,189 8,541 45 47,870 7,687 41 43,083 6,918 37 38,774 6,226 44 46,529 7,472 40 41,876 6,725 48 50,252 8069 43 45,226 7,262 435 45,653 7,331

EXTERNAL FRAUD 3 52,056 8,975 4 78,084 13,463 4 70,276 12,116 3 63,248 10,905 3 56,923 9,814 4 68,308 11,777 3 61,477 10,599 4 73,773 12719 4 66,395 11,447 36 67,021 11,555

EMPLOYMENT PRACTICES & WORKPLACE SAFETY 25 3,456 3,845 35 5,184 5,768 32 4,666 5,191 28 4,199 4,672 26 3,779 4,205 31 4,535 5,045 28 4,081 4,541 33 4,898 5449 30 4,408 4,904 302 4,450 4,950

CLIENTS, PRODUCTS & BUSINESS PRACTICES 36 56,890 7,890 50 85,335 11,835 45 76,802 10,652 41 69,121 9,586 37 62,209 8,628 44 74,651 10,353 40 67,186 9,318 48 80,623 11182 43 72,561 10,063 435 73,245 10,158

DAMAGE TO PHYSICAL ASSETS 33 56,734 3,456 46 85,101 5,184 42 76,591 4,666 37 68,932 4,199 34 62,039 3,779 40 74,446 4,535 36 67,002 4,081 44 80,402 4898 39 72,362 4,408 399 73,044 4,450

EXECUTION, DELIVERY & PROCESS MANAGEMENT 150 1,246 245 210 1,869 368 189 1,682 331 170 1,514 298 153 1,363 268 184 1,635 321 165 1,472 289 198 1,766 347 179 1,589 312 1,812 1,604 315

BUSINESS DISRUPTION AND SYSTEM FAILURES 2 89,678 23,543 3 134,517 35,315 3 121,065 31,783 2 108,959 28,605 2 98,063 25,744 2 117,675 30,893 2 105,908 27,804 3 127,090 33365 2 114,381 30,028 24 115,459 30,311

TOTAL 315 44,215 6,976 441 66,322 10,464 397 59,690 9,417 357 53,721 8,476 321 48,349 7,628 386 58,018 9,154 347 52,217 8,238 417 62,660 9886 375 56,394 8,897 3,806 56,926 8,981

Retail Banking

Payment & Settlements

Asset Management

167,245 142,456 123,345 113,342 94,458

Insurance

Severity of loss

VaR Calculator e.g., Monte Carlo Simulation Engine


Mean

Risk

99th Percentile

Annual Aggregate Loss ($)

0-10

1020

2030

3040

4050

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Risk is measured using internal and external loss data. The two measures of exposure are the aggregate mean and aggregate Value at Risk (VaR).
INDIVIDUAL LOSS EVENTS RISK MATRIX FOR LOSS DATA LOSS DISTRIBUTIONS VAR CALCULATION TOTAL LOSS DISTRIBUTION

74,712,345 74,603,709 74,457,745 74,345,957 74,344,576

Frequency of events

INTERNAL FRAUD Corporate Finance Number Mean Standard Deviation Trading & Sales Number Mean Standard Deviation Number Mean Standard Deviation Commercial Banking Number Mean Standard Deviation Number Mean Standard Deviation Agency Services Number Mean Standard Deviation Number Mean Standard Deviation Retail Brokerage Number Mean Standard Deviation Number Mean Standard Deviation Total Number Mean Standard Deviation 36 35,459 5,694 50 53,189 8,541 45 47,870 7,687 41 43,083 6,918 37 38,774 6,226 44 46,529 7,472 40 41,876 6,725 48 50,252 8069 43 45,226 7,262 435 45,653 7,331

EXTERNAL FRAUD 3 52,056 8,975 4 78,084 13,463 4 70,276 12,116 3 63,248 10,905 3 56,923 9,814 4 68,308 11,777 3 61,477 10,599 4 73,773 12719 4 66,395 11,447 36 67,021 11,555

EMPLOYMENT PRACTICES & WORKPLACE SAFETY 25 3,456 3,845 35 5,184 5,768 32 4,666 5,191 28 4,199 4,672 26 3,779 4,205 31 4,535 5,045 28 4,081 4,541 33 4,898 5449 30 4,408 4,904 302 4,450 4,950

CLIENTS, PRODUCTS & BUSINESS PRACTICES 36 56,890 7,890 50 85,335 11,835 45 76,802 10,652 41 69,121 9,586 37 62,209 8,628 44 74,651 10,353 40 67,186 9,318 48 80,623 11182 43 72,561 10,063 435 73,245 10,158

DAMAGE TO PHYSICAL ASSETS 33 56,734 3,456 46 85,101 5,184 42 76,591 4,666 37 68,932 4,199 34 62,039 3,779 40 74,446 4,535 36 67,002 4,081 44 80,402 4898 39 72,362 4,408 399 73,044 4,450

EXECUTION, DELIVERY & PROCESS MANAGEMENT 150 1,246 245 210 1,869 368 189 1,682 331 170 1,514 298 153 1,363 268 184 1,635 321 165 1,472 289 198 1,766 347 179 1,589 312 1,812 1,604 315

BUSINESS DISRUPTION AND SYSTEM FAILURES 2 89,678 23,543 3 134,517 35,315 3 121,065 31,783 2 108,959 28,605 2 98,063 25,744 2 117,675 30,893 2 105,908 27,804 3 127,090 33365 2 114,381 30,028 24 115,459 30,311

TOTAL 315 44,215 6,976 441 66,322 10,464 397 59,690 9,417 357 53,721 8,476 321 48,349 7,628 386 58,018 9,154 347 52,217 8,238 417 62,660 9886 375 56,394 8,897 3,806 56,926 8,981

Retail Banking

Payment & Settlements

Asset Management

167,245 142,456 123,345 113,342 94,458

Insurance

Severity of loss

VaR Calculator e.g., Monte Carlo Simulation Engine


Mean 99th Percentile

Annual Aggregate Loss ($)

0-10

1020

2030

3040

4050

What is the impact of the tail on the mean?

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

By comparing changes in the control environment one can predict changes in each business risk profile
CONTROL ASSESSMENT/INDICATOR SCORE

VAR

CAPITAL

Adjustment for Quality of Current Control Environment

210
Previous score

100

190
Current score 50

Linking capital to changes in the quality of internal controls provides an incentive for desired behavioral change
Copyright 2004, OpRisk Advisory LLC. All rights reserved.

The risk management industry approach can be used to integrate measures of risk and control, which can be used for allocating economic capital

RISK MATRIX FOR CAPITAL

INTERNAL FRAUD Corporate Finance Previous VaR Prev/Current Score Final Capital 21,000,000 50 55 19,000,000

EXTERNAL FRAUD 36,000,000 60 58 35,000,000

EMPLOYMENT PRACTICES & WORKPLACE SAFETY 62,000,000 75 71 65,000,000

CLIENTS, PRODUCTS & BUSINESS PRACTICES 75,000,000 61 61 75,000,000

DAMAGE TO PHYSICAL ASSETS 124,000,000 45 55 104,000,000

EXECUTION, DELIVERY & PROCESS MANAGEMENT 86,000,000 50 52 83,000,000

BUSINESS DISRUPTION AND SYSTEM FAILURES 36,000,000 50 55 32,000,000

TOTAL 362,000,000 50 55 326,000,000

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

ALTERNATE APPROACHES TO RISK ASSESSMENT

What other approaches can be considered for risk assessment?

Directly estimate frequency and severity parameters through expert judgment Estimate frequency and severity distributions using institutional memory Estimate risk (VaR) directly using internal and external loss data and disciplined scenario analysis

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

ALTERNATE APPROACHES TO RISK ASSESSMENT

Even with significant amounts of historical loss data it is virtually impossible to reliably estimate severity parameters.

Number of Events

Size of Loss

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

It is also very difficult to reliably estimate severity probabilities at different quantiles. Multiple estimates often create internal inconsistency

1 in 1 years

= $1,000

1 in 10 years = $10,000 1 in 20 years = $25,000 1 in 100 years = $50,000

0-10

1020

2030

3040

4050

Impact

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

Disciplined scenario analysis has been found to be moderately reliable and has produced valuable business benefits.

The analysis is based on factual, historical (external) loss data

Risk magnitude is clearly defined as potential loss at a specified confidence level, such as 99%
A 99% level event is defined to mean the second highest loss in one hundred years

This is further clarified put into practical terms based on loss experiences of ten peer banks; (similar size, similar controls), the second highest loss in the last ten years for the peer group
The whole purpose of this analysis is to allow the bank to compare the magnitude of loss at the same probability level: 50 foot tidal wave vs. 100 tidal wave $10 million money transfer loss vs. $100 million sales practices loss

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

CONTROL ASSESSMENT

We start with the risks, and using loss data identify control weaknesses and their underlying causes

RISKS

LOSSES
167,245 142,456 123,345 113,342 94,458

CAUSES
Segregation of duties

CONTROL ISSUE
Vacation policy

Internal Fraud

Data manipulation

Data Integrity

RISKS

LOSSES

CAUSES

INDICATOR
Staff Training Budget

EDPM

74,712,345 32,603,709 457,745 5,345,957 44,576

Insufficient training Number of Reconciliation Errors Number of Customer Complaints

Lack of management supervision

Risks are manifested in losses


Copyright 2004, OpRisk Advisory LLC. All rights reserved.

The goal is to identify which specific controls ought to be assessed

RISKS

LOSSES

CAUSES

CONTROL ISSUE
Access to information Timeliness of information

Internal Fraud Credit Card Counterfeiting

2,000 events Losses =$40M

Leakage of Confidential Information

Special clearance for overseas travelers

Special procedure in terms of Business Intelligence Algorithms

Risks are manifested in losses


Copyright 2004, OpRisk Advisory LLC. All rights reserved.

The next step is to examine the underlying business processes to better understand how well the risks are currently being managed and controlled
Process Chart
Applications Received. Data Entry . Completed entry applications forwarded to officers.

Credit Information Verification Application processing officers. Officers decision: - Approved / Declined / Further information Allocation of Application

End
Copyright 2004, OpRisk Advisory LLC. All rights reserved.

A control assessment scoring process must be relevant, consistent and objective

Relevance
Answer Choices Weighting Scale

The control issues must be relevant to a business line and risk


The answer choices should be consistent The control issues must be weighted according to relevance All scores must be converted to a consistent scale, e.g., 0 to 100 The process for normalizing scores must be theoretically valid The process must be transparent to allow for buy in and to identify opportunities for improvement Responses must be validated to avoid gaming the system

Normalization Transparency

Validation

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

SUMMARY & CONCLUSIONS

Integrated risk measurement and management can produces value, where the results are meaningful

Risk assessment is feasible and practical, but must be implemented only after one fully understand the meaning of the word risk and the technical challenges. Control assessment is feasible and practical, but must only be implemented after one understands the how to make a subjective process more objective. Integrated risk and control assessment or measurement promotes educated decision making, which in turn facilitates prudent risk management an can contribute to the creation of a good risk culture.

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

COSO was conceived in the early 1990s the first attempt at standardizing what was a subjective and inconsistently applied method for identifying, assessing, controlling and managing operational risks

However, in the context of modern operational risk management, we have learned: The definition of risk magnitude under COSO is inconsistent with that used in the risk management industry, including the BIS The approach is highly subjective, resource intensive and generates a huge catalog of unmanageable risks COSO approach to risk assessment (likelihood - impact analysis) focuses attention on what are likely to be phantom risks, not real risks. It produces both false positives and false negatives. Any prioritization of controls based on this spurious and misleading information may lead to enhancing controls in areas that are already over-controlled, while ignoring areas of control weakness

Copyright 2004, OpRisk Advisory LLC. All rights reserved.

When the answers are unclear

is it because we are asking the wrong questions?

Biographical Information

Ali Samad-Khan is President of OpRisk Advisory LLC. He has eight years experience in operational risk measurement and management and approximately twenty years of professional experience. His areas of expertise include: establishing an integrated operational risk measurement and management framework, developing policies and procedures, internal loss event database design and implementation; data quality assessment, data sufficiency, risk indicator identification, risk and control self assessment, disciplined scenario analysis, causal/predictive modeling, advanced VaR measurement techniques and economic capital allocation. . Mr. Samad-Khan has advised many of the worlds leading banks on operational risk measurement and management issues. His significant practical experience in this field comes from managing the implementation of ten major operational risk consulting engagements at leading institutions in North America, Europe and Australia. Key elements of the ORA framework and methodology have been adopted by dozens of leading financial institutions worldwide and have also been incorporated into the BIS guidelines.

Mr. Samad-Khan has frequently advised the major bank regulatory authorities, including: the Risk Management Group of Basel Committee on Banking Supervision, the Board of Governors of the Federal Reserve System, the Federal Reserve Bank of New York, the Financial Services Authority (UK) and the Australian Prudential Regulatory Authority. He also holds seminars and workshops for the Bank of International Settlements (BIS) and the Institution of International Finance (IIF).
Prior to founding OpRisk Advisory, Mr. Samad-Khan was CEO of OpRisk Analytics LLC, which was acquired by SAS in 2003. (From June 2003 to September 2004 Mr. Samad-Khan provided transitional support for the acquisition of OpRisk Analytics, serving as SAS Head of Global Operational Risk Strategy.) He has also worked at PricewaterhouseCoopers (PwC) in New York, where for three years he headed the Operational Risk Group within the Financial Risk Management Practice, in the Operational Risk Management Department at Bankers Trust as well as the Federal Reserve Bank of New York and the World Bank. Mr. Samad-Khan holds a B.A. in Quantitative Economics from Stanford University and an M.B.A. in Finance from Yale University. Articles include: Is the Size of an Operational Loss Related to Firm Size, with Jimmy Shih and Pat Medapa, Operational Risk, January 2000; Measuring and Managing Operational Risk, with David Gittleson, Global Trading, Fourth Quarter, 1998. Working papers include: How to Categorize Operational Losses Applying Principals as Opposed to Rules March 2002 and Categorization Analysis January 2003.
Copyright 2004, OpRisk Advisory LLC. All rights reserved.

También podría gustarte