ISO 9001 Impact Survey-eBook - Ver2

Based on Project TE/RAS/09/003
A survey covering quality management system

development, certication, accreditation and
economic benets
ISO 9001Its relevance and impact in
Asian Developing Economies
UNITED NATIONS INDUSTRIAL DEVELOPMENT ORGANIZATION
Vienna, 2012
ISO 9001 Its relevance and impact
in Asian Developing Economies
Based on Project TE/RAS/09/003
A survey covering quality management system
development, certication, accreditation and
economic benets
In conjunction with:
International Organization for Standardization
International Accreditation Forum
Norwegian Agency for Development Cooperation
This document has been produced without formal United Nations editing. The designations employed
and the presentation of the material in this document do not imply the expression of any opinion
whatsoever on the part of the Secretariat of the United Nations Industrial Development Organization
(UNIDO) concerning the legal status of any country, territory, city or area or of its authorities, or
concerning the delimitation of its frontiers or boundaries, or its economic system or degree of
development. Designations such as developed, industrialized and developing are intended for
statistical convenience and do not necessarily express a judgment about the stage reached by a
purtcuur country or ureu n the deveopment process. Menton o rm numes or commercu products
does not constitute an endorsement by UNIDO.
CONTENTS
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Executive summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2. Context of the project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3. Project methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4. Results of the survey of purchasers perceptions of ISO 9001-
certifed suppliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5. Survey of certifed organizations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6. Assessment of certifed organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
References and bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Annex: Acknowledgements for each of the participating countries . . . . . . . . . . . . . . . . . . 82
V
PREFACE
UNIDO
Since their initial publication in 1987, the ISO 9000 standards have undergone three revision cycles
(in 1994, in 2000 and again in 2008/2009) and have had a great impact on the implementation of
international trade and quality management systems by organizations throughout the world. In
particular, the ISO 9001 standard has been applied in a variety of economic sectors, including industry,
as well as regulatory frameworks established by governments in many areas. This standard focuses on
the management of the processes needed for an organization to be able to demonstrate its ability to
satisfy customer needs and expectations on a constant basis. Successful implementation of ISO 9001
by un orgunzuton requres the ormu uudtng o ts quuty munugement system by u certcuton
body, vhch tse needs to be duy uccredted. \hst there ure numerous certcuton bodes uround
the world, there is usually only one government-recognized accreditation body in any given country.
1hese uccredtuton bodes provde recognton o euch other's uccredted certcutes under the
lnternutonu Accredtuton lorum's Mut-uteru Recognton Arrungement |MLA). 1hs oers certed
organizations a unique opportunity to improve their competitive advantage by gaining access to
nternutonu murkets und ormu tenders vhere certcuton to lSC 900l s oten u pre-requste.
In recent years, however, some concerns have been expressed in the Asian developing economies
und esevhere ubout vhether uccredted certcuton to lSC 900l hus been uchevng the desred
outcomes, und vhether excessve uttenton to obtunng certcuton hus shted the ocus uvuy rom
the effectiveness of the quality management systems of the organizations involved. These concerns
have also been shared by the ISO, the IAF and others. This publication represents an attempt to
address these concerns in the context of business-to-business transactions in the manufacturing and
construction sectors. It is the outcome of a project initiated by UNIDO, funded by Norad and
supported with technical inputs by ISO and the IAF.
It is pleasing to see that the results have demonstrated (with some exceptions) that the implementation
o lSC 900l und the ussocuted certcuton hus been u good nvestment o resources, rom both the
perspectve o the certed orgunzutons und thut o ther customers |the muor purchusng
organizations in the region). The project has, however, highlighted some areas of weakness and we
expect that all the parties involved will take the necessary actions to ensure continued improvements
n the lSC 900l mpementuton und certcuton process.
Kandeh K. Yumkella
Director-General
VI
Norad
Over the last ten years there has been a rapid expansion of developing countries interest in ISO
900l, but oten the quest or certcuton hus been mted by u uck o nrustructure und poor
uvureness umong government und prvute sector orgunzutons o vhut certcuton und uccredtuton
uctuuy meun. Norud hus mude sgncunt nvestments n recent yeurs to hep seected Asun
developing countries to develop such infrastructure and awareness by funding projects aimed at
settng up uccredtuton bodes, certcuton bodes und other conormty ussessment uctvtes.
1here hus uso been u debute on the eectveness o uccredted certcuton - vhether the ocus hus
shifted from one in which organizations strived to develop an effective quality management system
thut coud subsequenty be certed, to one n vhch the uchevement o certcuton s the ony gou,
with a tendency to cut corners as necessary to achieve that goal. This debate centres on:
- vhether orgunzutons ure dervng tungbe benets through lSC 900l certcuton,
- vhether certcuton bodes ure curryng out the certcuton process eectvey,
- whether the varied expectations from different stakeholders are being met.
Norad was therefore pleased to be able to provide funding for this project, which was aimed at
obtaining objective evidence with which to be able to determine the true effectiveness of accredited
certcuton und ts vuue or the Asun deveopng economes. \e cun see rom the resuts thut the
mpementuton o un uccredted certcuton to lSC 900l vthn the munuucturng und constructon
sectors do generuy brng postve resuts both to the certed orgunzutons und ther customers, und
ultimately to the economies themselves. Of course the system is not perfect, and this report has
highlighted several areas of concern where improvements can be made to ensure the continued
eectveness und credbty o the uccredted certcuton process.
Dag Larsson
Senior Adviser
VII
ISO
The International Organization for Standardization (ISO) develops standards in response to clearly
dented demunds or murket und gobuy reevunt soutons. lSC does not, hovever, provde
certcuton servces to ussess compunce to the stundurds t deveops. lnsteud t vorks vth ndustry
und vth uccredtuton und certcuton bodes to encouruge mpementuton o the stundurds n
ways that meet the intent of the experts who wrote them.
Ceury, the use o lSC stundurds by those vho v benet rom them s un obectve o lSC und ts
members, one in each of 160 different countries. But ISO is neutral about how those who use the
standards demonstrate compliance to them.
The ISO 9000 family of quality management standards are amongst the most widely-used and well
known in ISOs catalogue of 18,600 standards. This study provides useful information about the take-
up by und benets or those n the countres covered by the survey vho huve decded to seek un
uccredted certcuton o ther quuty munugement system bused on lSC 900l.
The increase in the numbers of those using the ISO 9000 family of standards is a global phenomenon,
but the fastest growth in the uptake of these voluntary standards is in rapidly emerging countries. This
is not an accident: there is a clearly demonstrated link between uptake of standards and economic
growth, just as there is a link between standards and innovation.
The results of the survey also show that there is no reason for long discussions on policy nor for
theoretical debate on the next steps to be taken to foster economic development. This survey shows,
ugun, thut the lSC 9000 umy brngs reu benets. A key next step must be or sustuned nternutonu
funding for quality improvement and economic development in the region. This can be achieved if
such support is sustained, properly managed, and targeted through organizations and agencies, such
as industry associations and standards and conformity organizations, that have a direct link to those
vho v benet.
Rob Steele
Secretary-General
VIII
IAF
The Chair of IAF at the time this project was initiated was Dr. Thomas Facklam. I was then the Chair
of the IAF Technical Committee and in this position I was supportive of the project because of the
opportunity it gave to obtain some independent, objective, and reasonably representative information
ubout uccredted certcuton to lSC 900l.
1hs proect hus the potentu to n one muor mssng eement n the quest to mprove the ntegrty,
credbty und vuue o uccredted certcuton to lSC 900l.1hs crtcu eement s u methodoogy
or evuuuton/meusurement o the overu eectveness o the certcuton process.
As ve seek to muke mprovements n uccredted certcuton, t s mportunt to evuuute/meusure ts
status at a given point in time, implement improvements, and then re-evaluate/re-measure the status
at an appropriate time in the future to determine if the interventions resulted in true improvements.
Sound familiar? Sound like Plan-Do-Check-Act?
The results of this study are generally positive. They show that organizations do get value from
uccredted certcuton to lSC 900l, thut users cun rey on uccredted certcuton to lSC 900l us u
reusonube buss or huvng condence thut the products or servces provded by u certed
orgunzuton v u ther expectutons, und thut, despte commercu und compettve pressures
thut cun undermne the mpurtuty und eectveness o uudts und certcuton, the uudts und
certcutons ure, n most nstunces, eectve und vuuube.
The challenge now is to use this study to further improve accredited conformity assessment processes:
to implement the Act part of the PDCA continual improvement cycle.
Randy Dougherty
Chair and President
IX
ACKNOWLEDGEMENTS
The project was carried out with extensive collaboration from the following people, at the
international level:
Mr Rob Steele ISO Secretary-General
Mr Sean MacCurtain ISO Head of Conformity Assessment
Mr Olivier Peyrat ISO/CASCO Chair
Mr Randy Dougherty IAF President
Dr Thomas Facklam IAF Immediate Past President
Mr B. Venkataraman IAF/ILAC Development Support Committee Co-chair
Mr Roger Bennett IAF Executive Committee member
Mr Graham Talbot IAF Executive Committee member
Mr John Owen IAF Secretary
ln euch o the purtcputng countres, the proect beneted rom the hurd vork und dedcuton o u
teum o ocu consutunts upponted speccuy or ths proect, vth support rom ther respectve
nutonu stundurds bodes, ocu uccredtuton bodes, LNlDC oces und others. A compete st o
participants is given in the annex.
The following individuals contributed their time, energy and professional experience by commenting
on and validating the various questionnaires used in the project:
- Mr Luiz Fernando Corra Ferreira (Brazil)
- Mr Luiz Carlos do Nascimento (Brazil)
- Mr Gordon Wong (Hong Kong, China)
- Mr Carmine Reda (Italy)
- Ms Ana Paula Ramos (Portugal)
- Mr Colin MacNee (United Kingdom)
- Mr Paul Snowden (United Kingdom)
- Mr Craig Williams (United States)
Statistical analysis of the survey results was carried out with the invaluable assistance of the Institute
of Applied Statistics and Quality Management at the University of Lucknow, India, under the guidance
of Dr Anil K. Srivastava.
The support of Mr Amit Master and Mr Shardul Patel of 4C Consulting Pvt. Ltd. (Ahmedabad, India)
n deveopng und muntunng the proect-specc vebste s uso gruteuy ucknovedged.
X
Mr John Owen (IAF), Mr Xiao Jianhua (CNAS) and Mr Zhou Wenquan (CNAB) were instrumental
in organizing the international workshop in Shanghai, China. Mr Stefan Marinkovic (ISO/CASCO),
Ms Joyce Bleeker (ISO/CASCO), Ms Pauline Joris (AFNOR) and Ms Pascale Mienville (AFNOR)
provided valuable assistance in organizing the international workshop and webinar in Paris, France.
This project was coordinated by Dr Nigel H. Croft, under the guidance of Mr Ouseph Padickakudi,
Project Manager, and Dr Lalith Goonatilake, Director of the Trade Capacity Building Branch of
UNIDO, with administrative support from Mr Mathias Schaetz, Mr Pradeep Paulose and Ms Ivana
Nudunovc. 1he report vus edted by Mr Muuchy Scuon. 1he nuncu support rom Norud und
the active participation in the project of Mr Dag Larsson are gratefully acknowledged.
XI
ABBREVIATIONS AND ACRONYMS
AB Accreditation Body
AFNOR Association Franaise de Normalisation (France)
APG Auditing Practices Group (Joint initiative of ISO and the IAF)
BAB Bangladesh Accreditation Board
BIS Bureau of Indian Standards
BoA Bureau of Accreditation (Viet Nam)
BPS Bureau of Product Standards (Philippines)
BSN Badan Standardisasi Nasional (Indonesian National Standards Body)
BSTI Bangladesh Standards and Testing Institution
BQSP Bangladesh Quality Support Programme
CASCO Toolbox Series of conformity assessment standards issued by ISO/CASCO
CAB Conformity Assessment Body
C Certcuton ody
CR Certcuton/Regstruton ody |superseded termnoogy, but st reerred to n
some old IAF documents)
EMS Environmental Management System
FCL Foreign Critical Location
IAF International Accreditation Forum
IAG ISO 9000 Advisory Group (Multi-stakeholder group including ISO/TC 176,
ISO/CASCO, ISO/COPOLCO and the IAF)
IEC International Electrotechnical Commission
ILAC International Laboratory Accreditation Cooperation
Inmetro Instituto Nacional de Metrologia, Normalizao e Qualidade (Brazil)
ISMS Information Security Management System
ISO International Organization for Standardization
ISO/CASCO ISO Policy Committee on Conformity Assessment
ISO/COPOLCO ISO Committee on Consumer Policy
ISO/TC 176 ISOTechnical Committee 176 for Quality Management and Quality Assurance
KAN Komite Akreditasi Nasional (Accreditation Body of Indonesia)
XII
MLA Multi-lateral Recognition Arrangement
MSMU/MED Maldives Standards and Metrology Unit, Ministry of Economic Development
NAC Nutonu Accredtuton ourd or Certcuton odes |lndu)
NBSM Nepal Bureau of Standards and Metrology
NAC National Accreditation Council of Thailand (now NSC)
NSC National Standardization Council of Thailand (formerly NAC)
Norad Norwegian Agency for Development Cooperation
lAC lhppne Accredtuton Cce
PDCA The Plan-Do-Check-Act cycle (also referred to as the Deming Cycle)
PNAC Pakistan National Accreditation Council
PSQCA Pakistan Standards and Quality Control Authority
QCI Quality Council of India
QMS Quality Management System
OHSMS Occupational Health and Safety Management System
SAARC South Asian Association for Regional Cooperation
SDOC Self-Declaration of Conformity
SLAB Sri Lanka Accreditation Board for Conformity Assessment
SLSI Sri Lanka Standards Institution
SME Small or medium enterprise
SQCA Standards and Quality Control Authority (Bhutan)
STAMEQ Viet Nam Ministry of Science and Technology Directorate for Standards,
Metrology and Quality
TISI Thai Industrial Standards Institute
UNIDO United Nations Industrial Development Organization
WTO World Trade Organization
WTO/TBT World Trade Organization Agreement on Technical Barriers to Trade
1
EXECUTIVE SUMMARY
Background
What is ISO 9000?
The International Organization for Standardization (ISO) currently has a portfolio of over 18,500
standards, of which the ISO 9000 series relating to quality management is undoubtedly the best
known. Within this series, ISO 9001:2008 [1] (Quality management systems Requirements), is
vdey used by orgunzutons uround the vord to demonstrute thut they huve u ceury dened und
well managed set of processes that enable them to consistently provide products (including services)
that meet customer and applicable statutory and regulatory requirements.
What does ISO 9001 Certifcation mean?
A common way for organizations to show that they meet all the requirements of ISO 9001 is by using
un ndependent thrd-purty |u certcuton body |C)) to curry out un uudt o the orgunzuton, vhch,
successu, v resut n the orgunzuton beng ssued vth u certcute o conormty und the
ntuton o u progrumme o on-gong surveunce by the certcuton body to ensure thut the system
s muntuned n uccordunce vth the stundurd. 1he vuue o certcuton s the degree o pubc
condence und trust thut s estubshed by un mpurtu und competent ussessment by u thrd-purty.
1he orgunzuton's current und potentu customers cun be condent thut ther supper |the certed
orgunzuton) hus ndeed dened ts processes und s munugng them n u vuy thut v consstenty
provide them with conforming products.
Where does accreditation ft in?
Addtonu condence s provded vhen the certcuton body s uccredted by u recognzed
accreditation body (AB). Typically there is only one accreditation body per country. In the case of
lSC 900l certcuton, the uccredtuton process veres the certcuton body's conormty to the
ISO/IEC 17021 standard [2] (at the time this project was carried out, this meant ISO/IEC 17021:2006
Conormty ussessment Requrements or bodes provdng uudt und certcuton o munugement
systems, which has since been revised to ISO 17021:2011).
How does accredited certifcation facilitate trade?
Accreditation bodies may choose to participate in multilateral recognition arrangements under the
coordination of the International Accreditation Forum (IAF), to ensure that the criteria being used for
accreditation are comparable around the world. This is intended to facilitate international trade by
gvng the correspondng uccredted certcuton nternutonu vudty und recognton, ndependent
o the geogruphcu ocuton o the certed orgunzuton.
2 ISO 9001 ITS RELEVANCE AND IMPACT IN ASIAN DEVELOPING ECONOMIES
Number of ISO 9001 accredited certifcates
The countries involved in this project were Bangladesh, Bhutan, India, Indonesia, Malaysia, Maldives,
Nepu, lukstun, lhppnes, Sr Lunku, 1huund und Vet Num. 1he utest gures uvuube rom the
lSC 3 |bused on December 2009 dutu) shov thut there vere 68,563 uccredted certcutes to lSC
900l n these countres, representng 6.44% o the vordvde totu o l,064,785 certcutes.
Objectives of this study
1he orgnu proect proposu 4 outned three obectves o ths study, t shoud provde:
- Improved empirical evidence on the economic impact of management system
certcuton n ndustres n SAARC und South Lust Asun countres,
- lmproved knovedge buse or nutonu stundurds, certcuton und uccredtuton bodes,
governments, reguutors, lSC und lAl on munugement system certcuton und supportng
schemes,
- Best practices and policy guidelines for improvement of functioning of management
system certcuton und uccredtuton bodes.
Summary of the project methodology
The project methodology was agreed jointly by UNIDO, ISO and the IAF and is described in detail
in the body of this report, but basically comprised the following phases:
Survey of institutional purchasers
A survey was carried out of major institutional purchasing organizations (the customers and
potentu customers o lSC 900l-certed orgunzutons). 1he um vus to ussess the purchusers'
eve o uvureness und use o uccredted certcuton to lSC 900l us u supper seecton und
evaluation tool, and the satisfaction of these major purchasers with the performance of their
lSC 900l-certed suppers.
Interviews with selected purchasers
Face-to-face meetings were held with a small sample of purchasing organizations who had
demonstruted ther ceur understundng o uccredted certcuton |durng the ntu survey
questionnaire) in order to obtain more detailed information about the performance of their suppliers.
Survey of certifed organizations
A survey o lSC 900l-certed orgunzutons vus conducted to ussess |umong other thngs) ther
reusons or mpementng u quuty munugement system und or seekng certcuton, ther use o
consutunts, the seecton crteru thut they upped n choosng u certcuton body, ther eves o
sutsucton vth the certcuton body, und ther perceptons ubout the certcuton process.
EXECUTIVE SUMMARY 3
Visits to certifed organizations
A seres o one-duy vsts vus mude to 56l certed orgunzutons. 1hese vere not ormu uudts but
were characterized as market surveillance visits, carried out by experienced quality professionals
und umed ut ussessng the eve o condence n the eectveness o the orgunzuton's quuty
munugement systems und n the certcuton process.
Sampling criteria and limitations/bias in the methodology
The project was intentionally limited primarily to manufacturing and construction-based industries
involved in business-to-business transactions. Service sectors and business-to-consumer
transactions were not targeted.
It is important to emphasize that participation in the project was necessarily voluntary and that
organizations could not be required to take part.
ley ndngs
Economic benefts
1here ure ceur emprcu economc benets to the eectve mpementuton und uccredted
certcuton o quuty munugement systems n the munuucturng sectors o the Asun deveopng
countries in which the project was conducted.
Credibility of ISO 9001
Cveru, the perceptons o both the lSC 900l stundurd und uccredted certcuton to lSC 900l n
the region are good, though the role of accreditation is not well understood either by purchasers or
by certed orgunzutons.
Purchasers perceptions of their ISO 9001-certifed suppliers
1he purchusers surveyed vere muny sutsed vth the perormunce o ther lSC 900l-certed
suppers |vth some exceptons), und, n generu, lSC 900l-certed suppers perormed better or
much better thun non-certed suppers, bused on u number o purumeters. Cne ureu o concern
thut vus dented, hovever, vus the poor responsveness o certed orgunzutons to customer
complaints.
Lack of transparency in some certifcation bodies
A number o certcuton bodes n the regon |muny runchsees o overseus certcuton bodes)
reused to provde uny normuton ubout the orgunzutons they hud certed. 1hs rused doubts
ubout the eve o trunspurency und openness o the certcuton bodes |one o the core prncpes
mentoned n lSC/lLC l702l 2 n order to promote condence n uccredted certcuton).
Market surveillance of certifed organizations
A murket surveunce methodoogy nvovng short |one-duy) vsts to certed orgunzutons vus
shown to be effective in distinguishing between superior and unsatisfactory performances of suppliers
(as perceived by their customers), based on the subjective evaluation of a series of parameters related
to the implementation of the quality management system.
Performance of certifed organizations
Cveru, the perormunce o the 56l certed orgunzutons thut vere vsted vus good, und
demonstruted the eectveness o the uccredted certcuton process vthn thut sumpe. Hovever,
a small percentage of the organizations visited demonstrated unsatisfactory results, which
emphasizes the need to avoid complacency and to drive further improvements in the accredited
certcuton process.
Use of the quality management system to drive improvement
There is very little awareness in the region about the ISO 9004 guidance standard (ISO 9004:2009
Managing for the sustained success of an organization A quality management approach) [5].
1hs stundurd s not used or certcuton purposes but provdes recommendutons on hov to use the
quality management system to drive continual improvement and achieve long-term (sustained)
success.
Handling of customer complaints
1here s u very ov uvureness umong u users |purchusers, certed orgunzutons, certcuton
bodies, consultants and others) about the ISO 10002 standard (ISO 10002:2004 Quality management
Customer satisfaction Guidelines for complaints handling in organizations) [6]. This is
particularly important in light of the relative dissatisfaction of purchasers with the way in which their
certed suppers deu vth compunts.
Performance of small and medium enterprises
1he resuts conrm thut the eve o condence n the perormunce o smu und medum szed
enterprises (SMEs organizations with less than 250 employees) is lower than for larger organizations.
Use of consultants
1he vust muorty |over 80%) o the certed orgunzutons surveyed hud used the servces o u
consultant to help with the implementation of their quality management system. This was more
pronounced for micro, small and medium organizations.
Audit durations
Interestingly, in spite of previous anecdotal evidence to the contrary, the results indicate that
certcuton bodes, n generu, respect the uudt durutons prescrbed by the lAl Mundutory
Document MD5 [8]. In particular, audit durations for micro, small and medium enterprises tend to
exceed the recommendations of IAF MD5.
EXECUTIVE SUMMARY 5
Weak areas of implementation of ISO 9001
1he veukest ureus o mpementuton thut vere dented durng the vsts to certed orgunzutons
were:
- A generu uck o ocus on preventve uctons,
- loor use o the lun-Do-Check-Act upprouch |lSC 900l Cuuse 4.l) to munuge the
quuty munugement system |MS) processes,
- loor cuture o contnuu mprovement,
- Luck o udequute cuuse unuyss und eectve correctve ucton or process, product und
system nonconormtes,
- Ltte use o the process upprouch throughout the orgunzuton.
Diferences in performance of certifcation bodies and accreditation bodies
Notube derences vere observed n the perormunce und eve o condence n orgunzutons
certed by derent certcuton bodes und under uccredtuton rom derent uccredtuton bodes.
Detued normuton s beng mude uvuube on un ndvduu buss to those nvoved.
Performance of local franchisees of foreign certifcation bodies
1he survey ceury hghghted the reutvey poor perormunce o orgunzutons thut hud been certed
by ocu runchsees o oregn certcuton bodes, vhen compured vth ocu certcuton bodes or
brunch oces o nternutonu certcuton bodes.
Time taken to implement and certify the quality management system
A ceur trend vus observed thut shoved thut more recenty certed orgunzutons ure uchevng
certcuton n shorter tmes thun those thut huve been certed or onger perods. 1he vsts to
certed orgunzutons conrmed, hovever, thut the eve o condence n the quuty munugement
system o those orgunzutons thut hud spent ess thun sx months mpementng ther system vus
substuntuy over thun n thut o those vho hud tuken onger.
6
1. INTRODUCTION
The ISO 9000 standards
The International Organization for Standardization (ISO) currently has a portfolio of over 18,500
standards, of which the ISO 9000 series is undoubtedly the best known. The current ISO 9000 series
comprises more than 20 guideline and requirements standards related to quality management, and
is developed and maintained by ISOs Technical Committee TC 176.
1he stundurd thut s used us u buss or certcuton s lSC 900l:2008 uuty Munugement
Systems Requirements [1], for which the stated objective (Clause 1.1) is:
to specify requirements for a quality management system where an organization
(a) needs to demonstrate its ability to consistently provide product that meets customer and
applicable statutory and regulatory requirements, and
(b) aims to enhance customer satisfaction through the effective application of the system,
including processes for continual improvement of the system and the assurance of conformity to
customer and applicable statutory and regulatory requirements.
ISO 9001 addresses the need for: the commitment of top management to the quality management
system, u customer ocus throughout the orgunzuton, u ceur quuty pocy und pocy obectves
dened by top munugement, denton o the responsbty und uuthorty o the vurous personne
nvoved n the quuty munugement system und communcuton betveen them, ensurng the
uvuubty o resources |ncudng competent personne), upproprute eves o documentuton, und
control of the various operational processes, from sales through product design and development and
manufacture (or service provision) to process monitoring, product inspection and after-sales support.
In order to ensure on-going conformity to ISO 9001 and continual improvement of the system, the
standard further requires the organization to carry out its own internal audits and reviews of its
system at regular intervals and to undertake corrective and preventive actions as needed.
ISO 9001 is based on two key concepts: a clear understanding of the organizational processes and
ther nteructons to ensure product conormty, und the munugement o those processes und the
system as a whole, using a Plan-Do-Check-Act methodology. It is applicable to any kind of
organization large or small, private or public, and in the manufacturing, service or other
economic sectors.
As a result, ISO 9001 has been widely accepted around the world as a basis for organizations to
provde condence n ther ubty to understund customer und stututory/reguutory requrements,
and to systematically provide products (including services) that meet those requirements. Its aim is
an effective quality management system (one that achieves the expected outcome of consistent,
1. INTRODUCTION 7
conormng products), but not necessury un ecent one |vhch does so vth the mnmum vuste
o resources). Athough ecency muy not be crtcu rom the perspectve o the certed orgunzuton's
customers, however, it is clearly important for the organization itself, and is a key topic of the
guidance standard ISO 9004.
ISO 9004:2009 Managing for the sustained success of an organization A quality management
approach [5] goes beyond the basic requirements of ISO 9001, and guides organizations towards
sustained success by identifying and meeting the needs and expectations not only of their customers
but also of other interested parties over the long term and in a balanced way. This standard is
speccuy not ntended or the purpose o certcuton, but muy be used us u buss or se-ussessment
(by the organization) of the maturity of its quality management system. ISO 9004 addresses topics
such us strutegc punnng, pocy depoyment, empoyee motvuton, munugement o nuncu und
nuturu resources, knovedge munugement und nnovuton thut ure not speccuy covered n
ISO 9001.
lSC 900l und lSC 9004 vere rst pubshed n l987 und huve snce undergone three revsons |n
1994, in 2000 and again in 2008/2009) to incorporate the latest trends and philosophies in quality
management. They are intended to be used separately or together, but it is important to emphasize
the specc ocus o euch. Muny o the perceved benets thut cun be derved rom un lSC 9000-bused
quality management system will not be achieved by implementing ISO 9001 alone. ISO 9001
focuses only on the management system requirements needed to ensure the provision of consistent,
conforming products to customers, thereby enhancing customer satisfaction. Other organizational
benets ncudng mprovements n ecency muy be u by-product o lSC 900l mpementuton
but can only be ensured by the effective deployment of the long-term, multi-stakeholder approach
promoted by ISO 9004, which is outside the scope of this project.
1he deveopment o the rst edton o the lSC 9000 stundurds, pubshed on l5 Murch l987, vus
bused on trudtonu quuty ussurunce stundurds used n the munuucturng sector |speccuy n the
engineering, aeronautical, nuclear and military-related industries) that had been around since the
1950s. The quality system requirements standard, ISO 9001, (quality management was to come
later) was developed to address contractual situations where the supplier had to demonstrate the
ability to design/develop, produce, install and service its products. A minor revision to ISO 9001
vus mude n l994 but vthout uny sgncunt chunge n ocus.
In order to make the ISO 9000 standards more business-focused and easier to understand for those
not involved in the manufacturing industries, however, ISO placed great emphasis during the
deveopment o the lSC 9000:2000 umy o stundurds on denng und depoyng throughout the
standards a set of eight quality management principles. These principles had been developed during
the mid-1990s and drew on the philosophies of recognized quality gurus such as Deming, Juran,
Crosby, Ishikawa and Feigenbaum, among others. They are explained in detail in the ISO 9000:2005
standard (Quality management systems Fundamentals and vocabulary[9]). They now provide
the conceptual basis for both the ISO 9001 requirements standard and the ISO 9004 guideline
standard. The principles are:
- Customer focus
- Leadership
- Involvement of people
- Process approach
- System approach
- Fact-based decision-making
- Continual improvement
- Mutuuy-benecu supper reutonshps
Conlormly lo ISO 9001 can bc dcmonslralcd by rsl, sccond or lhrd-arly asscssmcnls.
Currcnlly ovcr 1,000,000 organzalons worldwdc hold accrcdlcd lhrd-arly
ccrlcalcs lo ISO 9001.
In addition to a shift from Quality Assurance to Quality Management, ISO 9001:2000 also saw a
change in focus from many hard engineering-based requirements to softer, more abstract people-
focused requirements. Requirements related to management commitment and employee awareness
were introduced, and, whilst these are much more appropriate for a modern organization, they are
uso more dcut to uudt thun some o the od buck or vhte |yes/no) prescrptve requrements o
the 1994 versions. The recently-published ISO 9001:2008 standard incorporated only a few small
changes to existing (ISO 9001:2000) requirements and retained the eight quality management
principles at its core.
lt s not surprsng, then, thut most peope tend to ussocute lSC vth lSC 9000, und u certcute
of conformity to the ISO9001 requirements standard has become commonly (but incorrectly) known
us lSC Certcuton. lSC s not, hovever, drecty nvoved n the lSC 900l certcuton process,
nor does t speccuy promote or requre certcuton us u meuns o demonstrutng conormty to ts
standards. Its activities are essentially limited to the development and publication of international
standards, including ISO 9001 and other requirements standards such as ISO/IEC 17021 [2] and ISO/
IEC 17011 [10] (developed by ISOs Conformity Assessment Committee ISO/CASCO) that relate to
the certcuton und uccredtuton processes respectvey. lSC does, hovever, muntun cose contucts
vth the munugement system certcuton communty through ts usons vth the lnternutonu
Accreditation Forum (IAF).
Accredited certifcation to ISO 9001
A common way to demonstrate conformity to ISO 9001 is by using an independent third-party (a
certcuton body) to curry out un uudt o the orgunzuton, vhch, successu, v resut n the
orgunzuton beng ssued u certcute o conormty. 1hs s ntended to provde condence to the
organizations customers (and potential customers) that they can expect the organization to
consistently provide them with conforming products. Although this is the most widely accepted
method o conormty ussessment or lSC 900l, t s not the ony one, und ndeed certcuton s
not a required way of demonstrating conformity to any ISO standard. Other ways of demonstrating
conformity include second-party evaluations of suppliers, carried out by the purchasers, and self-
declarations of conformity (for example, by a corporate entity to ensure that its various divisions
and plants around the world have a quality management system that meets not only ISO 9001 but
also its corporate policies and objectives). ISO itself is neither directly involved in nor requires
thrd-purty certcuton.
1. INTRODUCTION 9
ln order to demonstrute ts ovn competence to curry out the thrd purty certcuton process, the
certcuton body muy choose to be uccredted by un uccredtuton body, ths uccredtuton v be
bused prmury on the requrements dened n lSC/lLC l702l 2. lSC/lLC l702l s bused on sx core
prncpes: mpurtuty, competence, responsbty, openness, condentuty und responsveness to
compunts, und ony those certcuton bodes thut meet u the uccredtuton requrements ure
uuthorzed to ssue uccredted certcutes. Accredtuton bodes muy uso choose to become sgnutores
of the IAFs Multi-lateral Recognition Arrangement (MLA) for quality management systems, subject to a
peer evaluation process based on the requirements of ISO/IEC 17011 [10]. In both cases (accreditation
o certcuton bodes und peer revevs o uccredtuton bodes), the requrements o lSC/lLC l702l
and ISO/IEC 17011 are supplemented by IAF mandatory and guidance documents. One of the criteria
for acceptance as an IAF MLA signatory is that the accreditation body must formally recognize
accreditation granted by other MLA signatories as being equivalent to its own accreditation. In the
context of trade capacity-building in the Asian developing countries, this multilateral recognition of
certcutes ssued by uccredted certcuton bodes s o vtu mportunce n provdng certed
organizations with access to world markets without the need for multiple accreditations.
Mcmbcrsh ol IAF s ocn lo accrcdlalon bodcs and olhcr slakcholdcrs such as
assocalons ol ccrlcalon bodcs and nduslry grous. 7hcrc arc currcnlly S6 counlrcs
wlh accrcdlalon body mcmbcrs ol lhc IAF (Sourcc: IAF wcbslc www.al.nu)
1he vurous nteructons n the conormty ussessment suppy chun ure shovn n gure l. 1he um
s to promote ree trude by provdng condence throughout ths suppy chun so thut customers |und,
ultimately, consumers) can have a reasonable expectation of receiving conforming products from
suppers |the certed orgunzutons) vherever n the vord these ure ocuted. 1he utmute obectve
s to ucheve the gou o Cnce certed, uccepted everyvhere.
Figure 1. Interactions within the conformity assessment supply chain aimed at providing
condcncc in lSO 9001 ccrtication
Chain of confdence-promoting activities
reducing technical barriers to trade (WTO/TBT)
(ISO/CASCO)
ISO/IEC 17011 +
IAF mandatory and
guidance documents
(ISO/CASCO)
ISO/IEC 17021 +
IAF mandatory and
guidance documents
(ISO/TC176))
ISO 9001:2008
National standard
(Translation if needed)
ISO standards Accredited certifcation process
IAF MLA
Accreditation body
Certifcation body
Organization
Customers
{
{
{
{
{
{
{
Confdence
Confdence
Confdence
Confdence
(
IS
Nat
Confdence
A certcute o conormty to lSC 900l s not u e-tme uvurd but must be reneved ut reguur
ntervus by the certcuton body, typcuy every three yeurs, vth u perodc surveunce mechunsm
in between to monitor the on-going conformity and effectiveness of the system.
lt s uso mportunt to recognze thut lSC 900l s not u busness exceence mode, ts ums ure qute
busc to gve condence n the orgunzuton's ubty to consstenty provde conormng products
to its customers. The way in which the organization manages its business in order to achieve this
objective depends very much on its nature (size, type and economic sector, among others), and ISO
900l provdes some exbty n hov to meet ts requrements. lt s, hovever, mportunt never to ose
sght o ths prmury obectve, rrespectve o hov muny documented procedures und vork
instructions the organization has developed, how many hours of training it conducts, how many
peces o meusurng equpment t cubrutes, the resut does not provde condence n the
consistency of the product the organization provides, then the system is not effective. It is, of course,
unrealistic to expect that any system will always be 100% effective, and for this reason ISO 9001
includes a requirement for organizations to continually improve the effectiveness of their quality
management system.
In an effort to shift the (incorrect) focus of some organizations, consultants, auditors and others from
concentrating only on the documentation and administrative requirements of ISO 9001, recent
strategic initiatives by ISO and the IAF have placed a much greater emphasis on ensuring the
effectiveness of the quality management system. A recent ISO/IAF joint communiqu, Expected
outcomes rom uccredted certcuton to lSC 900l ll, vus umed prmury ut emphuszng ths
pont to uccredted certcuton bodes und ther uudtors, und the muntru Cutput mutters! s nov
part of the everyday lexicon of those involved in the conformity assessment of management systems.
The latest ISO statistics from December 2009 [3] show that there are now over a million organizations
vordvde thut ure certed to lSC 900l. 1he muorty o these huve dedcuted sgncunt eorts und
resources (often aided by the sensible use of competent consultants) to implementing robust systems
capable of producing the desired outputs consistent, conforming products. These systems, in
turn, huve been uudted und certed by ethcu, mpurtu certcuton bodes usng competent teums
of professional auditors. Croft and Dougherty [12] call all those involved in this process the good
guys. They have done a good job and brought credit to the name of ISO 9000.
Unfortunately, in the late 1990s, disturbing trends began to emerge of less scrupulous players (the
bad guys[12]) appearing on the scene (probably driven by the commercial factors surrounding the
vhoe busness o certcuton) und threutenng to undermne the credbty o the lSC 900l
certcuton process us u vhoe. 1here s much unecdotu evdence thut these bud guys ure operutng
in Asian developing countries, but the extent of their activities has thus far not been clear, nor has
ther mpuct on the overu credbty und eectveness o the certcuton process.
11
2. CONTEXT OF THE PROJECT
There are considerable differences in the level of maturity of ISO 9001 implementation and the
associated conformity assessment infrastructure between the various countries in the Asian region
that were included in this project. This can be seen to some extent in table 1, where it can be noted
thut the countres ncuded n the study runge rom lndu |vth umost 40,000 uccredted certcutes
and a long history of standardization and conformity assessment) to countries, such as Bhutan, the
Mudves und Nepu, vhere there ure currenty ony u hundu o uccredted certcutes und no
formally constituted accreditation body.
Table 1. Details of the standards and accreditation bodies in each of the participating countries,
and thc numbcr of accrcditcd ccrticatcs in cach (according to thc ofciaI 2009 lSO
Survcy[3]): countrics invoIvcd in thc projcct
Note: Numbers reute to the 1C1AL number o certcutes n December 2009, ncudng those ssued under overseus
accreditations. (Data from ISO Survey)
TabIc 2. fvoIution of thc numbcr of accrcditcd lSO 9001 ccrticatcs for thc participating
countrics, according to thc ofciaI 2009 lSO Survcy[3]
Country Standards body Accreditation body Certs
Bangladesh BSTI BAB 8S2
Bhutan SQCA 7
India BIS NABCB 37 493
Indonesia BSN KAN S 476
Malaysia Standards Malaysia Standards Malaysia 6 463
Maldives MSMU/MED 5
Nepal NBSM 55
Pakistan PSQCA PNAC 2 112
Philippines BPS PAO 1 992
Sri Lanka SLSI SLAB 678
Thailand TISI NAC 6 097
Viet Nam STAMEQ BOA 7 333
Country Dec
2002
Dec
2003
Dec
2004
Dec
200S
Dec
2006
Dec
2007
Dec
2008
Dec
2009
Bangladesh 6 49 182 570 570 284 845 852
Bhutan 3 6 10 4 7
India 2247 8367 12558 24660 40967 46091 37958 37493
Indonesia 308 1318 3134 4068 4783 4532 5713 5476
continued
Country Dec
2002
Dec
2003
Dec
2004
Dec
200S
Dec
2006
Dec
2007
Dec
2008
Dec
2009
Malaysia 1119 3076 4337 5695 6786 7838 6267 6463
Maldives - 1 1 1 1 2 4 5
Nepal 4 6 25 83 115 77 65 55
Pakistan 186 464 695 2013 2291 2580 2268 2112
Philippines 270 456 1108 1414 2007 2199 2412 1992
Sri lanka 34 90 148 244 318 496 549 678
Thailand 938 1675 2620 3231 3913 5915 5275 6097
Viet Nam 354 1237 1598 2461 3167 4282 3071 7333
Total 5466 16739 26406 44443 64924 74306 64431 68S63
Note: A more detued unuyss o the utest trends n munugement systems certcuton und the utest versons o the lSC
Survey o Certcutons cun be dovnouded ree o churge rom vvv.so.org
lgure 2 shovs the overu grovth n uccredted certcuton n the regon rom 2002 to 2009. lt shoud
be noted thut the decreuse n the number o certcutons n some countres rom December 2007 to
December 2008 has been attributed by ISO to improvements in the data reporting and counting
methodology used in their 2008 survey and onwards. This corrected previous over-estimates due to the
doube-countng o some certcutes n eurer surveys |vhen the sume certed orgunzuton vus
reported by tvo or more sepurute enttes, or exumpe by both the heud oce und brunch oce o the
certcuton body, or by tvo uccredtuton bodes vhere the certcuton body hud mutpe uccredtutons).
figurc 2. Data from thc lSO Survcy showing thc ovcraII incrcasc in accrcditcd ccrtication
in thc countrics invoIvcd in thc projcct from 2002 to 2009
80000
70000
60000
50000
40000
30000
20000
10000
0
2002 2003 2004 2005 2006 2007 2008 2009
Probable double-counting
errors in 2006/7 surveys
Total number of accredited certifcates in the region (20022009)
2. CONTEXT OF THE PROJECT 13
The above statistics do illustrate, however, the more than tenfold increase in the number of accredited
certcutes betveen 2002 und 2009 n the regon, vhch s evdence thut SAARC und South Lust
Asian countries look at the ISO 9001 standard as an important instrument for enhancing their trade
and business.
lt shoud uso be noted thut there ure st uppurent dscrepunces n the number o certcutes
reported in the ISO Survey. In Nepal, for example, feedback during the local workshops at the end
o the proect suggests thut the totu number o uccredted certcutes n thut country muy be
considerably more than the 55 reported in the ISO Survey for 2009. It has been suggested that,
becuuse muny uccredted certcutes n Nepu ure ssued by lndun certcuton bodes |there ure
currenty no uctve certcuton or uccredtuton bodes n Nepu), these muy huve been vrongy
reported us lndun certcutes.
14
3. PROJECT METHODOLOGY
Overall project planning (MarchOctober, 2009)
Meetings were held at the international level with interested parties from UNIDO, ISO and the IAF
to discuss and agree the overall project methodology, and, at the local level, a series of meetings and
workshops was held in nine of the twelve countries involved in the project. The latter included
discussions with senior management of the national standards bodies and accreditation bodies,
interviews of potential local consultants for the project, and a half-day workshop with interested
parties to explain the background and objectives of the project. These parties included:
- Standards body and accreditation body personnel
- Local UNIDO representatives
- Certcuton bodes
- Purchasing organizations
- Crgunzutons certed to lSC 900l
- Consultants
- Industry and consumer organizations
- Chambers of trade and/or commerce
- Government (ministries of industry, science and technology, commerce, etc.)
Typical economic sectors that were covered in the project included:
Automotive Machinery & equipment manufacture
Chemicals (including paint) Oil and gas
Civil engineering Packaging
Concrete and cement Pharmaceuticals
Construction Plastics
Defence Public utilities (including power)
Electrical/Electronics Railways
Food and beverages Rubber
Garment and textile manufacture Shipbuilding/ship-breaking
Health care Steel and non-ferrous metals
Local/regional/national governments Telecommunications
ln terms o the nutonu nrustructures or stundurdzuton, certcuton und uccredtuton, the
involvement of national standards bodies fell into two main categories:
3. PROJECT METHODOLOGY 15
- lnvovement n the estubshment o un ussocuted munugement systems certcuton
body |vth the upproprute revu sepuruton betveen the stundurdzuton und the
certcuton uctvtes). 1hs s the trudtonu mode udopted by muny o the Asun
deveopng economes und s usuuy bused on pre-exstng product certcuton uctvtes.
- Involvement with (or close structural ties to) the local accreditation body.
Based on (subjective) observations during the planning and execution of this project, it became
uppurent thut n severu countres vhere there vus u munugement system certcuton body ussocuted
with the national standards body, this had generated some friction between the national standards
body und the ocu uccredtuton body. ln order to enhunce pubc condence und the eectveness
o uccredted munugement system certcuton, t s vtu thut those nvoved vork together prouctvey
in order to overcome such differences. ISO and the IAF should encourage close collaboration, mutual
support and feedback mechanisms between national standards bodies and accreditation bodies in
the same way that ISO and the IAF collaborate internationally through the IAF/ILAC/ISO Joint Working
Group and the ISO 9000 Advisory Group.
Survey of institutional purchasers
(October 2009March 2010)
The objective of this part of the project was to assess the level of awareness and use of accredited
certcuton to lSC 900l us u supper seecton und evuuuton too, und the sutsucton o muor
purchusers vth the perormunce o ther lSC 900l-certed suppers.
The questionnaires used in this survey were developed by a consensus-building process that included
experts from ISO and the IAF. Useful inputs were provided by Inmetro (Brazil), who had carried out
u survey o ruzun purchusng orgunzutons' perceptons o ther lSC 900l-certed suppers n
2006 [13].
Questionnaires were initially validated by asking a number of large institutional purchasers in other
economies to complete a questionnaire and provide feedback, which was then incorporated prior to
meetng vth seected representutves o purchusers n the Asun deveopng countres or nu
validation.
The questionnaires were sent out to selected institutional purchasing organizations in each of the
ten participating countries (Bhutan and the Maldives were not included in this part of the project)
after initial contacts had been made between the UNIDO local consultants and the purchasing
organizations. The consultants were requested to consult only major purchasers (in terms of
purchasing power) in their country, regardless of those purchasers knowledge of ISO 9001.
Interviews with selected purchasers (JanuaryMarch 2010)
1he obectve o ths purt o the proect vus to obtun u contro sumpe o certed orgunzutons
whose performance (as perceived by their customers, the purchasing organizations) was known.
used on the resuts o responses to the questonnures, purchusers vere dented vho hud
demonstruted ther understundng o uccredted certcuton und hud mpemented sound purchusng
and supplier evaluation/feedback processes. A small sample of these knowledgeable purchasers
was invited to participate in face-to-face interviews with the project consultants. These interviews,
conducted at the purchasing organizations premises, included detailed discussions and a review of
their supplier performance data. More detailed information about the performance of ISO
900l-certed suppers vus obtuned, uong vth normuton ubout the best und vorst
performing suppliers (as perceived by the purchasers).
lunnng o vsts to certed orgunzutons
(JanuaryApril 2010)
1he ntenton n the ntu proect pun vus to muke u rundom sumpe o lSC 900l-certed orgunzutons
in the manufacturing and construction sectors in each participating country, using a sample size equal
to the squure root o the number o certed orgunzutons n thut country. ln generu, ths number vus
achieved, as shown in table 3, which is based on the ISO Survey data for December 2008 [14] (the
latest information that was available at the time the visits were being planned).
TabIc 3. PIanncd and actuaI sampIc sizcs for onc-day markct survciIIancc" visits to ccrticd
organizations
The selection criteria for the consultants to be hired by UNIDO for this part of the project were
dened. A key chuenge vus to denty peope vth sucent n-depth knovedge und experence
n uudtng lSC 900l-bused quuty munugement systems but ut the sume tme huvng no concts o
nterest by perormng vork or or on behu o uny certcuton body.
Country No. of ccrticatcs
(Dcccmbcr 2008)
Planned sample
(bascd on \ of numbcr
of ccrticatcs)
ActuaI sampIc sizc
Bangladesh 845 29 30
Bhutan 4 2 4
India 37958 195 204
Indonesia 5713 76 77
Malaysia 6267 79 77
Maldives 4 2 2
Nepal 65 8 10
Pakistan 2268 48 25
Philippines 2412 49 19
Sri lanka 549 23 25
Thailand 5275 73 38
Viet Nam 3971 63 50
Grand Total 647 S61
A total of 28 consultants were contracted, and all participated in one of three intensive 5-day training
courses thut ncuded u one-duy cubruton vst to u certed orgunzuton |to ensure consstency
between the various consultants) and an examination.
Requests were sent out to all IAF accreditation body members asking for details of their accredited
certcuton bodes thut ssued certcutes n the purtcputng countres. |Note: as will be seen later,
not all accreditation bodies provided this information.)
1he uccredted certcuton bodes vere then contucted to request detus o the nume, scope und
geogruphcu ocuton o euch o ther lSC 900l-certed cents |n uccordunce vth lSC/lLC l702l:2006
Cuuse 8.3), vth the obectve o compng u u st o certed orgunzutons n euch country und then
selecting a random sample of those organizations in the manufacturing/construction sectors. (Note:
ugun, us v be seen uter, u sgncunt number o certcuton bodes dd not provde ths normuton.)
Survey o certed orgunzutons |lebruury}une 20l0)
An extensve veb-bused survey turgetng lSC 900l-certed orgunzutons vus conducted n u
12 participating countries. This was based on a questionnaire that had been developed by a consensus
process with ISO/TC 176, ISO/CASCO and the IAF, and validated by individual experts from those
organizations.
Cne-duy vsts to certed orgunzutons |Apr}uy 20l0)
1he methodoogy or the one-duy vsts to certed orgunzutons vus deveoped by u process o
consultation and consensus, mainly within the IAF Technical Committee Task Force on Accreditation
Market Surveillance, during meetings in Vancouver, Canada (Oct 2009) and Rio de Janeiro, Brazil
(March 2010). There has been a close interaction with this group throughout the project, and the
results will be important to the Task Force in developing future guidance for the possible implementation
of similar methodologies within the IAF.
The following were key concepts in the one-day visits:
- The visits were not intended to be repeat audits and were not focused on conforming/
nonconforming outcomes.
- 1he resuts o the vsts vere strcty condentu und vere reported nether to the
orgunzuton's certcuton body nor to the respectve uccredtuton body. 1hs vus
consdered essentu n order to obtun ugreement rom the certed orgunzutons to
receive the visits.
- 1he vsts vere umed ut determnng condence eves reuted to vurous uspects o the
orgunzuton's quuty munugement system und the overu eve o condence n the
certcuton process. 1he u check-st covered the oovng 26 topcs:
l. lntu certcuton body uudt duruton
2. Surveillance audit duration and frequency
3. Scope stutement on the orgunzuton's certcute
4. }ustcuton or uny excusons |lSC 900l:2008 Cuuse l.2)
5. Top management commitment
6. Internal communication
7. Understanding and implementation of the process approach
8. Use of PDCA to manage processes (ISO 9001:2008 Clause 4.1)
9. Quality policy
10. Meaningful objectives
ll. uuty munuu us u true reecton o the quuty munugement system
12. Appropriate use of quality management system documentation
13. Resource management (including human resources)
14. Work environment
15. Management of product realization processes
16. Process monitoring and measurement
17. Treatment of product nonconformities
18. Cause analysis for nonconformities and effective corrective action
19. Internal audits
20. Management review
21. Prevention of nonconformities
22. Customer feedback and complaints handling
23. Cveru condence n the ubty to produce consstent, conormng products
24. Continual improvement of quality management system effectiveness
25. Cveru condence n the certcuton body's processes
26. Cveru condence n the orgunzuton's mpementuton o lSC 900l
- It was found that all the consultants were able to discriminate between scores of 15
vthn u scue o condence eves dened or euch o the purumeters us oovs:
Grade 1 Ltte or no condence No evidence at all to support the
implementation of this topic.
Gradc 2 Some evidence presented,
but not at all convincing
Some evidence was presented, but in the
professional judgment of the consultant
(based on experience), there would probably
be evidence to support a nonconformity in
an audit situation.
Gradc 3 OK No reason to doubt
that this is being addressed
correctly
This was the default grade, where there was
no evidence to suggest reasons for concern,
based on the consultants experience and
professional judgment.
Gradc 4 Clear evidence that this is
being done and meets the
intent of ISO 9001
Sucent obectve evdence vus uvuube to
provde u good eve o condence thut the
organization is meeting the intent of ISO 9001.
Grade 5 We can be proud to use
this organization as a
benchmark for this topic
It was emphasized during the training that
Grade 5 was to be reserved for truly excellent
performance, and not to be awarded lightly.
Analysis of results (AugustSeptember 2010)
The results were analysed using standard, simple, statistical tools. For the majority of the results, the
use of graphical techniques such as pie charts, bar charts and histograms was found to be a satisfactory
and user-friendly way of presenting the data.
In cases where more detailed analysis was found to be necessary, this was carried out by the Institute
of Applied Statistics and Quality Management of Lucknow University, India.
No attempt was made to rank the performance of each country. This was considered to be outside
the scope of the current project and, in view of the many variables involved, not a viable proposition.
lnternutonu und ocu |country-specc) eedbuck
workshops (October 2010March 2011)
Workshops to present the project results and promote feedback from interested parties were carried
out as follows:
International workshops
These were held in Shanghai, China and Paris, France (with a simultaneous webcast) and were
uttended by u totu o upproxmutey l50 purtcpunts rom stundurds bodes, certcuton bodes,
accreditation bodies and other interested parties.
Local (country-specifc) workshops
These were held during January to March 2011 in each of the participating countries (except Bhutan
and the Maldives), with a total of over 900 participants, including all major interested parties.
20
4. RESULTS OF THE SURVEY OF
PURCHASERS PERCEPTIONS OF
ISO 9001-CERTIFIED SUPPLIERS
The questionnaire was replied to by 429 purchasing organizations in 10 countries (all the SAARC and
SE Asian countries involved, with the exception of Bhutan and the Maldives, who participated only
in the second part of the project). This represented a response rate of approximately 17% of the 2,513
purchasing organizations contacted.
Demographics of respondents
Demogruphc detus o the respondents ure gven n gures 3 to 6.
figurc 3. GcographicaI distribution of thc 429 purchasing organizations participating in
the survey
109
89
56
43
30
29
25
14
10
0
20
40
60
80
100
120

I
n
d
i
a

V
i
e
t

N
a
m
I
n
d
o
n
e
s
i
a
B
a
n
g
l
a
d
e
s
h
P
h
i
l
i
p
p
i
n
e
s
N
u
m
b
e
r
Country
Number of respondents per country
P
a
k
i
s
t
a
n
M
a
l
a
y
s
i
a
T
h
a
i
l
a
n
d
S
r
i

L
a
n
k
a
N
e
p
a
l
24
4. RESULTS OF THE SURVEY OF PURCHASERS PERCEPTIONS OF ISO 9001-CERTIFIED SUPPLIERS 21
figurc 4. Sizc distribution of thc 429 purchasing organizations who participatcd
(by numbcr of cmpIoyccs)
figurc S. Sizc distribution of thc 429 purchasing organizations who participatcd
(by annuaI purchasing budgct in US doIIars)
19
0%
1049
3%
50199
11%
200299
6%
300499
14%
500999
22%
10004999
31%
5000
13%
Number of employees
$1 million
8%
>$15
million
13%
>$510
million
18%
>$10100
million
35%
>$100 million
26%
Annual purchase budget
figurc 6. Numbcr of suppIicrs activcIy cvaIuatcd by cach of thc 429 purchasing organizations
surveyed
Quality management systems of purchasing organizations
Figure 7 shows that 95% of the 429 respondents to the survey claimed that their organization had a
quality management system that included the purchasing process and met the requirements of ISO
900l. Some 8l% cumed to huve un uccredted certcute. 1hs s nterestng n ght o the reutvey
ov eves o uvureness ubout certcuton und uccredtuton demonstruted by these purchusers |see
later), but it does highlight the fact that many major organizations in the Asian developing countries
(such as these purchasers) had already implemented ISO 9001.
figurc 7. Rcsponscs from thc 429 purchasing organizations about thcir own IcvcI of conformity
to lSO 9001
10
15%

>1000
3%
Number of suppliers actively evaluated
1011000
27%
11100
55%
No/Dont know
5%
Yes, based on internal
audits/SDOC
11%
Yes, based
on second
party audits
2%
Accredited
certifcation
81%
Does their own QMS meet ISO 9001?
Non-accredited
certifcation
1%
Purchasers familiarity with ISO 9001
The vast majority of purchasers who responded to the survey (82%) said they either rely heavily on
lSC 900l or use t requenty |see gure 8). Some 9% hud heurd o t but ddn't use t, vhst l%
(representing only 4 major purchasers in the region) claimed never to have heard of ISO 9001.
figurc 8. Rcsponscs from thc 429 purchasing organizations to thc qucstion Arc you famiIiar
with thc lSO 9001 quaIity managcmcnt systcm standard!"
Use of ISO 9001 as a supplier evaluation tool
Of the purchasers who took part in the survey 79% claimed they used ISO 9001 as a component of
their supplier evaluation process.
Purchasers awareness of roles of ISO, accreditation
bodies and the IAF
Despite the large percentage of purchasing organizations that claimed to use ISO 9001 as a basis
for supplier evaluations, their awareness of accreditation and the roles of ISO and the IAF was
very poor.
Cny 38% o the respondents vere uvure o the uct thut lSC's roe n lSC 900l certcuton s
mted to thut o stundurd-vrter. Some 9% beeved thut lSC uctuuy certes orgunzutons vho
meet the requirements of the standard, whilst the majority (53%) believed that ISO is responsible for
uccredtng certcuton bodes to curry out certcuton.
Never heard of it
1%
Use it frequently
36%
Heard of it but
never use it
9%
Rely heavily on it
46%
Familiarity with ISO 9001
Use it occasionally
8%
Overall, awareness among the purchasing organizations about the role and importance of accreditation
was very disappointing. Only 32% overall could name their local accreditation body (after excluding
from the analysis those countries where there is no accreditation body). This compares with 37% who
were able to name any foreign accreditation body.
Some regional variations were apparent, however (See table 4), with recognition of the local
accreditation bodies being greatest in the following countries:
- Indonesia (KAN 70%)
- Thailand (NAC (now NSC) 64%)
- Pakistan (PNAC 53%)
With the exception of Indonesia, the Philippines and Thailand, a higher percentage of purchasers in
all the other countries were able to name a foreign accreditation body than could name the local
accreditation body.
TabIc 4. lcvcI of namc rccognition of IocaI and forcign accrcditation bodics by purchasing
organizations in thc participating countrics
a
Excludes Bhutan, Maldives and Nepal, where there is no local accreditation body.
Perhaps of even greater concern is the low level of awareness about the IAF among major purchasers.
Cny 34% o the respondents cumed to knov ubout the lAl und vhut s does, 3l% hud heurd o the
lAl but ddn't knov vhut t does, und 35% hud never heurd o t.
This is particularly relevant if the IAF is to promote the use of its MLA mark in order to provide
condence n uccredted certcuton uround the vord. Athough no deveoped economes vere
included in this project, informal feedback obtained during the international workshops would
indicate that the levels of awareness are not much greater in those countries than in the Asian
developing economies.
Overall, this lack of awareness of major institutional purchasers about accreditation is of great
concern in the Asian developing countries for two reasons:
Country
a
locaI AB Percentage of purchasers
who could correctly name
the local AB
Percentage of purchasers
who could correctly name
any foreign AB
Bangladesh BAB 23 37
India NABCB 26 40
Indonesia KAN 70 64
Malaysia Standards Malaysia 29 45
Pakistan PNAC 53 62
Philippines PAO 40 20
Sri lanka SLAB 10 21
Thailand NAC (now NSC) 64 24
Viet Nam BoA 6 16
- lor exporters, t s vtu thut overseus customers ure ube to derve condence rom lSC
900l certcutes provded by suppers n the Asun deveopng countres. lt s unkey
(and unrealistic to expect) that these customers will know and recognize each of the
deveopng countres' uccredtuton body nume or ogo vhen t uppeurs on u certcute,
so the use of a single, recognizable IAF MLA mark is key for trade capacity-building.
- For importers in the developing countries, it is important to be able to recognize what
s und vhut s not u vud uccredted certcute vhen presented by u potentu supper.
Whilst an importer might recognize the accreditation mark of its local accreditation
body, it is unrealistic to expect them to be able to verify the authenticity of accreditation
marks from all the other countries in which their overseas suppliers may have achieved
uccredted certcuton. 1hs euves spuce or ess scrupuous puyers |oten vu ocu
ugents) to present used certcutes vhen bddng or purchuse contructs. 1hs pructce
was evidenced during the initial visits to the Asian developing countries and is very
dcut or the ntegent uymun purchuser to denty.
Purchasers awareness of conformity assessment
methodologies
Conormty to lSC stundurds such us lSC 900l muy be demonstruted by rst, second or thrd-purty
uttestutons, l5, l6 und not u certcuton bodes |vho muke thrd-purty uttestutons o conormty)
choose to demonstrute ther competence by seekng uccredtuton. lnterestngy, vhst u sgncunt
majority of the respondents knew about and used the concepts of self-declarations of conformity,
second-purty uudtng, und uccredted certcuton, ony 23% knev ubout und hud experence vth
unuccredted certcuton. 1hs urther supports the need or u sgncunt uvureness-budng
cumpugn by uccredtuton bodes und the lAl n order to promote the udded eves o condence
that accreditation can provide.
lurchusers' vercuton o suppers' lSC 900l
certcutes
Given the high dependence that the purchasers place on ISO 9001 as a basis for supplier evaluation,
it is surprising how little they check the evidence of this. Figure 9 shows that:
- Cny 47% usk or u copy o ther suppers' certcutes vhen supper seecton hus been
based on ISO 9001.
- Cny ubout 37% check the vudty dute on certcutes presented by ther suppers.
- Only 30% look for the accreditation body logo (and, based on the low awareness about
accreditation in general, are probably not sure what they are looking for anyway).
- Cny 42% check thut the scope o certcuton s upproprute or the products they ure
purchasing.
- Less thun l0% huve ever contucted ther supper's certcuton body to check the vudty
o the certcute presented.
figurc 9. Pcrccntagc of thc 429 purchasing organizations who vcrify thc vaIidity and rcIcvancc
of thcir suppIicrs' ccrticatcs
Supplier feedback
lt cun be seen rom gures l0 und ll thut the percentuge o purchusers vho systemutcuy provde
feedback on supplier performance varies from 49% (for high value and/or high risk purchases) to
22% |or ov vuue und/or ov rsk purchuses). 1hs s encourugng und shoud enube certcuton
bodes to obtun vuuube nputs nto the uudt process by requestng ther certed cents to provde
this information from customers, as required by ISO 9001:2008 Clause 8.2.1 (Customer satisfaction).
figurc 10. DctaiIs of thc 429 purchascrs' IcvcIs of fccdback to suppIicrs of high vaIuc/
high risk products
Which of the following do you check?
(percentage of respondents)
0
5
10
15
20
25
30
35
40
45
50
Copy of
certifcate
Validity
date
AB logo Scope Consult CB
High value/high risk
We provide
systematic
feedback
49%
Only if
supplier
asks
17%
Only if we have
a complaint
32%
No
2%
figurc 11. DctaiIs of thc 429 purchascrs' IcvcIs of fccdback to suppIicrs of Iow vaIuc/Iow risk"
products
lurchusers' condence n und opnons o lSC 900l
certcuton bodes
Figure 12 shows that the overall perception by the 429 purchasing organizations of the image and
credbty o uccredted certcuton bodes vus good |7%: very good, 53%: good und 37%:
fair). Only 3% considered the credibility to be poor and only one of the purchasers surveyed
(signifying 0.23% of the total) considered it to be very poor.
figurc 12. Pcrccptions of 429 purchascrs of thc imagc and crcdibiIity of accrcditcd ccrtication
bodies in the region
Low value/low risk
We provide
systematic
feedback
22%
Only if supplier
asks
16%
No
8%
Only if we have
a complaint
54%
Overall perception about image and credibility
Good
53%
Very good
7%
Very poor
0%
Poor
3%
Fair
37%
1he purchusers vere usked or ther opnons o severu uspects o ther lSC 900l-certed suppers'
perormunce, und the resuts ure shovn n gure l3, presented n order o decreusng eve o sutsucton.
1hey vere uso usked to compure the perormunce o ther lSC 900l-certed suppers vth thut o
uncerted suppers |or the sume supper pror to becomng certed) or the sume set o crteru.
1hese resuts ure shovn n gure l4.
ln generu, the eves o sutsucton n terms o u runge o crteru ure good, und certed suppers
perorm sgncunty better thun non-certed suppers. Hghghts thut ure o purtcuur nterest ure:
- Satisfaction with the intrinsic quality of the products being supplied by the ISO
900l-certed orgunzuton vus hghest o u the crteru, vth u 96% sutsucton
eve |85% o purchusers vere sutsed und ll% very sutsed). \hen mukng the
compurson vth uncerted suppers, t cun uso be seen thut certed suppers perorm
sgncunty better n compurson |l3% o the suppers surveyed consdered certed
suppers to perorm much better und 63% better thun uncerted suppers or ths
criterion). This is consistent with the results reported in a similar study carried out in Brazil
in 2006 by Inmetro [13], where 70% of the purchasers of industrial products surveyed
consdered ther certed suppers' perormunce to be better thun uncerted suppers.
- There was a high level of satisfaction with the ease of communication during the
purchusng process |94% o purchusers vere ether sutsed or very sutsed) und vth
the accuracy of promotional and marketing information (93% of purchasers were either
sutsed or very sutsed).
- In terms of the suppliers ability to consistently provide conforming products (the
prncpu obectve o lSC 900l certcuton!), l3% o the purchusers vere very
sutsed, und 79% vere sutsed, but t s mportunt to note thut us muny us 8% vere
dssutsed. \hen mukng the compurson vth uncerted suppers, hovever, t cun
be seen thut, despte ths 8% eve o dssutsucton, the overu percepton s thut certed
suppers perorm sgncunty better n compurson |l2% o the suppers surveyed
consdered certed suppers to perorm much better und 64% better thun uncerted
suppliers for this criterion).
- 1vo topcs thut shoud be o concern to certed orgunzutons und to certcuton
bodies are that the highest levels of dissatisfaction among purchasers relate to their
suppers' ubty to meet devery commtments |l0% dssutsed und l% very
dssutsed) und ther responsveness to customer compunts |l3% dssutsed
und l% very dssutsed). ln both cuses, hovever, certed suppers vere ound to
perorm sgncunty better thun non-certed suppers. ln the context o customer
compunts hundng, t voud seem upproprute or certed orgunzutons to muke more
use o the gudunce stundurd thut speccuy uddresses ths topc |lSC l0002:2004
Quality management Customer satisfaction Guidelines for complaints handling
in organizations[6]). From the feedback workshops conducted at the conclusion of the
project, however, it is clear that this standard is not well known, and special efforts should
be made by both ISO and national standards bodies to promote greater awareness of it. It
s uso recommended thut lSC und the lAl consder deveopng gudunce or certcuton
body uudtors on hov to eectvey uudt certed orgunzutons' customer compunts
hundng processes, ths coud be done vu the Audtng lructces Croup.*
7hc ISO 9001 Audlng Praclccs Grou (APG) s an nlormal grou ol qually managcmcnl
syslcm (QMS) cxcrls, audlors and raclloncrs drawn lrom lhc Inlcrnalonal
Accrcdlalon Forum (IAF) and lhc ISO 7cchncal Commllcc 176 Qually Managcmcnl and
Qually Assurancc (ISO/7C 176)[17].
figurc 13. lcvcIs of satisfaction of thc 429 purchascrs with ccrticd suppIicrs for various critcria
figurc 14. Pcrformancc of ccrticd suppIicrs comparcd to thc pcrformancc of non-ccrticd
suppIicrs (or thc samc suppIicr prior to ccrtication), in ordcr of grcatcst to smaIIcst
improvement
Criteria ranked in order of decreasing satisfaction
Intrinsic quality of purchased product
Ease of communication during purchase process
Accuracy of promotional and marketing information
Ability to consistently provide conforming products
Customer service quality (courtesy; speed; efectiveness)
Ability to meet delivery commitments
Responsiveness to complaints
0 20 40 60 80 100
Percentage of purchasers responding
Very dissatisfed
Dissatisfed
Satisfed
Very satisfed
(From left to right):
Comparisons with non-certifed suppliers
(ranked in order of greatest improvement)
Accuracy of promotional and marketing information
Ability to consistently provide conforming products
Intrinsic quality of purchased product
Responsiveness to complaints
Customer service quality (courtesy; speed; efectiveness)
Ability to meet delivery commitments
Ease of communication during purchase process
0 20 40 60 80 100
Percentage of purchasers responding
Much worse
Worse
About the same
Better
Much better
ISO 9001-certifed
organizations are
(from left to right):
than non-certifed
suppliers
Purchasers opinions of the ISO 9001 standard
und certcuton
Figure 15 shows that the general level of satisfaction with the ISO 9001 standard is good. Some 18% of
those 429 purchasers who participated in the survey considered it to be excellent, 57% said it was
good, and 22% thought it reasonably good. Only 3% considered the standard to be inadequate
and no respondents considered it to be very poor. It is worth noting that ISO is continually seeking
feedback from users on how it can improve its standards, and the comments received during this
project will be taken into consideration when developing future versions of ISO 9001.
figurc 1S. Opinions of thc 429 purchascrs about thc lSO 9001 standard (without considcring
ccrtication issucs)
lgure l6 shovs the responses obtuned rom the 429 purchusers ubout ther condence n the ubty
o uccredted certcuton to demonstrute ther suppers' ubty to consstenty provde products
that meet customer requirements, as well as applicable statutory and regulatory requirements.
Condence eves vere generuy good, but vth some exceptons noted. Some 94% o the respondents
vere generuy sutsed |6% consderng ther eve o condence to be exceent, 50% good,
and 38% generally OK, but with occasional exceptions).
These are very encouraging results, but conversely, however, this means that 6% (or a total of 25 of
the 429 purchasing organizations surveyed) said they had inadequate (5%) or very little (1%)
condence n uccredted certcuton. 1here s st umpe scope or mprovement!
ln terms o the perormunce o uccredted certcuton bodes, gure l7 shovs thut over hu o the
purchusng orgunzutons consdered the certcuton bodes operutng n ther country to be ether
equally good or good, but with some small variations. Some 33% had no data on which to form
an opinion. Conversely, however, this means that 12% considered there were major variations
rom one certcuton body to unother, und l% |4 purchusers) consdered the certcuton bodes to
be equally bad in terms of performance.
Opinion of ISO 9001 standard
Good
57%
Excellent
18%
Reasonably good
22%
Very poor
0%
Inadequate
3%
figurc 16. lcvcI of condcncc in lSO 9001 ccrtication (429 purchascrs who rcspondcd to
thc survcy)
figurc 17. Opinions of thc 429 purchascrs rcgarding any systcmatic diffcrcnccs obscrvcd in
thc pcrformancc of accrcditcd ccrtication bodics opcrating in thcir country
Figure 18 shows that the majority of purchasers surveyed (57%) do not carry out second-party
evuuutons o ther lSC 900l-certed suppers.
C those vho do curry out uudts o ther lSC 900l-certed suppers:
- 53% (23% of the total number of purchasers surveyed) found generally satisfactory results.
- 35% (15% of the total number of purchasers surveyed) had carried out assessments with
Level of confdence in ISO 9001 certifcation
Good provides
confdence
50%
Generally OK, but
with occasional
exceptions
38%
Excellent we
can rely on it
6%
No very little
confdence
1%
Not good
inadequate
confdence
5%
Comparison between CBs
Some small
variations
41%
No data
33%
Major
variations
12%
All equally
good
13%
All equally bad
1%
some ureus o concern, ndcutng thut they hud dented nonconormtes n the quuty
management system.
- 5% (12% of the total number of purchasers surveyed) had found major areas for
concern, ndcutng thut they hud doubts ubout the vudty o the suppers' certcuton.
figurc 18. Purchascrs' usc of sccond-party suppIicr cvaIuation audits and thc rcsuIts whcn thcsc
arc carricd out on lSO 9001-ccrticd suppIicrs
ln terms o compunng to the suppers' certcuton body, gure l9 shovs thut ony 7% o the
purchasing organizations consulted had ever done so (3% with satisfactory outcomes and 4% who
vere dssutsed vth the outcome) und u urther 9% voud huve ked to, but ddn't knov hov to do
so. Some 25% of the respondents didnt know they could submit a complaint to the suppliers
certcuton body, vhst 59% never et the need.
Smur resuts vere ound n reuton to compunts to uccredtuton bodes |gure 20), ether us
a result of the escalation of complaints about suppliers or regarding the performance of a
certcuton body.
Durng the eedbuck vorkshops, there vus u generu consensus umong certcuton bodes thut they
voud vecome such eedbuck rom customers o ther certed cents, snce poory-perormng lSC
900l-certed orgunzutons uso reect budy on the repututon o the certcuton body thut ssued
the certcute. lt s mportunt, though, to emphusze thut sporudc, mnor compunts ubout
nonconormng products shoud be deut vth betveen the purchuser und ther supper, ony cuses
of repeated nonconforming products or indications that there are problems with the suppliers quality
munugement system shoud be brought to the uttenton o the certcuton body.
In view of the fact that a quarter of the purchasers surveyed were unaware that they could complain
to ther suppers' certcuton body, greuter promnence needs to be gven to the gudunce or
purchasers available on the ISO website (ISO 9001 What does it mean in the Supply Chain?
Have you carried out second-party
audits on certifed suppliers?
No
57%
Yes, generally
satisfactory
23%
Yes, but with
major areas
of concern
5%
Yes, but
with some
areas of concern
15%
[15,16]). This explains the concepts of quality management systems, how conformity to ISO 9001
muy be demonstruted und the roe o uccredted certcuton, und gves gudunce on hov purchusers
can provide feedback in cases of dissatisfaction with the performance of their suppliers.
figurc 19. Pcrccntagc of thc 429 purchascrs compIaining to a ccrtication body
figurc 20. Pcrccntagc of purchascrs compIaining to an accrcditation body about thcir suppIicrs
or about thcir suppIicrs' ccrtication body
Have you ever complained to a CB?
Never had
the need
59%
Didnt know
I could do that
25%
Didnt know how
9%
Yes, and satisfed
3%
Yes, but dissatisfed
4%
Have you ever complained to an AB?
Never had
the need
60%
Didnt know
I could do that
25%
Didnt
know how
11%
Yes, and satisfed
2%
Yes, but dissatisfed
2%
34
5. SURVEY OF CERTIFIED
ORGANIZATIONS
1hs survey used u veb-bused questonnure or u lSC 900l-certed orgunzutons to out detus
of their motives for implementing a quality management system, their experiences during
mpementuton, und ther overu sutsucton vth the resuts |ncudng the certcuton process).
In fact the spontaneous level of response to this on-line survey was very low, and the vast majority of
the questonnures vere ed out by the respondents durng vsts by our teums o LNlDC-upponted
consutunts. Note thut some questonnures vere ed out on-ne by nterested orgunzutons, und
these organizations were not necessarily included in the visit programme.
A total of 604 organizations completed the survey, distributed among the various countries as shown
n gure 2l.
figurc 21. GcographicaI brcakdown of thc numbcr of rcspondcnts to thc qucstionnairc for
ccrticd organizations

I
n
d
i
a

V
i
e
t

N
a
m
I
n
d
o
n
e
s
i
a
B
a
n
g
l
a
d
e
s
h
P
h
i
l
i
p
p
i
n
e
s
Country
Number of respondents per country
P
a
k
i
s
t
a
n
M
a
l
a
y
s
i
a
T
h
a
i
l
a
n
d
S
r
i

L
a
n
k
a
N
e
p
a
l
0
50
100
150
200
250
M
a
l
d
i
v
e
s
B
h
u
t
a
n
N
u
m
b
e
r
223
19
24
30
23
27
4
44
51
77
79
3
5. SURVEY OF CERTIFIED ORGANIZATIONS 35
Respondents included a good sample of micro-, small, medium and large organizations, as shown in
gure 22. Athough dentons o mcro-, smu und medum enterprses vury n derent economes,
it is generally considered that organizations with less than 250 employees fall into this category. See,
for example, the European Commission Recommendation 2003/361/EC [18]. This means that 67% of
the organizations who responded to the survey fall into the category of SME, based on their number
of employees.
figurc 22. Sizc distribution of rcspondcnts (numbcr of cmpIoyccs)
1here vus uso u good spreud n the perods o tme thut orgunzutons hud been certed to lSC 900l:
- 26% o the respondents hud been certed or l0 yeurs or onger, meunng thut they hud
rst mpemented ther quuty munugement system bused on the l987 or l994 versons
of ISO 9001 (or, in some cases, the old versions of ISO 9002 or even ISO 9003), before
the introduction of the process approach model adopted in ISO 9001:2000. They had
therefore had to make the transition to this new approach during the period 20012003.
- 48% o the respondents hud been certed or u perod o 4l0 yeurs, vhch meuns they
hud undergone ut eust one u certcuton cyce |ntu uudt, ooved by perodc
surveunce uudts und u recertcuton uudt).
- 26% o the respondents hud been certed or 3 yeurs or ess, meunng thut they hud not
yet been through one u certcuton cyce.
Reusons or seekng certcuton
The top three Number 1reasons for implementing a quality management system and seeking
certcuton ure shovn n gure 23. lt cun be seen thut over 54% hud nternuy-motvuted reusons
Size distribution of respondents
(number of employees)
51250
39%
Over1000
7%
1150
23%
Less than 10
5%
5011000
8%
251500
18%
(including internal improvement and corporate or top management objective), whilst 39% had
externally motivated reasons (including customer pressure and access to international markets).
Figure 23. Main reasons cited for choosing to implement a quality management system
It is interesting to see how the Number 1 reasons for implementing a quality management system
have changed over time. Figure 24 shows the various reasons cited as a function of the length of time
the respondent hud been certed. 1he oovng trends cun be observed:
- An increase over time in the percentage of respondents claiming that their Number 1
reason for implementing a quality management system was to respond to customer
pressure.
- A decrease over time in respondents claiming that their Number 1 reason for
implementing a quality management system was to gain a competitive advantage, or to
meet corporate or top management objectives.
It can therefore be concluded that the implementation of a quality management system is no longer
consdered to oer u sgncunt compettve udvuntuge, but us certcuton becomes more o un entry
criterion in order to become a supplier to many purchasers, to compete in international markets and
to purtcpute n tenders, t s the uck o certcuton thut becomes u compettve dsudvuntuge. 1hs
in turn has led to an increase over time in the percentage of organizations citing customer pressure
as their number one reason for implementing their quality management system, and an increasing
ocus on gettng the certcute ruther thun on mpementng u sound, eectve quuty munugement
system. This in turn has resulted in a quest for shorter implementation times and the subsequent
deleterious effects on system effectiveness, as will be shown later in this report.
Main reasons cited for implementing a QMS
Internal improvement
Customer pressure
Corporate or top management objective
Access to international markets
Marketing/PR
Competitive advantage
Other
Percentage of organizations citing Number 1 reason
0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0
36.1
24.9
18.5
13.9
3.1
1.5
2.0
figurc 24. Main rcasons citcd for choosing to impIcmcnt a quaIity managcmcnt systcm, as a
function of thc Icngth of timc ccrticd
Implementation time
lt cun be seen rom gure 25 thut most orgunzutons |60% o the 604 respondents) took betveen
7 months and two years from the time they decided to start implementing the quality management
system unt they obtuned certcuton. ln some cuses, hovever, the tme tuken vus surprsngy
short, vth umost 29% cumng to huve become certed vthn our to sx months uter ther
decision to begin implementing the quality management system, and a further 9% claiming to have
done so in less than three months. Although there might be special circumstances that facilitate such
short implementation periods (for example, if the organization is part of a larger entity that already
hus u corporute quuty munugement system), t s unkey thut the potentu benets to be derved
during implementation have been fully achieved in such cases.
As shovn n gure 26, there hus been un ncreusng trend n recent yeurs n the percentuge o
organizations taking 6 months or less to implement and certify their system. For those organizations that
hud been certed or over l0 yeurs, ony 26% hud tuken 6 months or ess to mpement the system und
ucheve certcuton und ony 3% hud done t n 3 months or ess. 1hs compures to 36% o orgunzutons
certed n the ust 3 to l0 yeurs vho mpemented und ucheved certcuton n 6 months or ess |vth
9% dong so n 3 months or ess), und 5l% o orgunzutons certed n the ust 3 yeurs mpementng
the system und uchevng certcuton n 6 months or ess |vth l5% dong so n 3 months or ess).
It is highly improbable that a fully operational and effective quality management system can be
mpemented n such short perods o tme, und cun huve ucheved sucent muturty or u competent
audit to be carried out. Realistically, before being audited, the quality management system should
huve pussed through ut eust one compete lun-Do-Check-Act cyce, ncudng the process dentons
and preparation of the appropriate documentation, effective implementation and deployment,
Reasons for implementing a QMS as a function
of length of time certifed
Over 10 years
410 years
0 3 years
0 20 40 60 80 100
Percentage of organizations citing as Number 1 reason
Internal improvement
Customer pressure
Corporate or top
management objective
Access to international
markets
Marketing/PR
Competitive advantage
Other
Key
Number 1 reason for
implementing QMS

L
e
n
g
t
h

o
f

t
i
m
e

c
e
r
t
i
f
e
d
followed by a cycle of internal audits, corrective actions as needed and at least one management
review. The negative impact of an unrealistically short implementation time on overall performance
of the quality management system will be reported and discussed in more detail later in this report.
figurc 2S. lcngth of timc takcn to achicvc ccrtication aftcr making thc dccision to bcgin
impIcmcntation of thc quaIity managcmcnt systcm (604 rcspondcnts)
figurc 26. lcngth of timc takcn to achicvc ccrtication as a function of timc sincc ccrtication
was achicvcd (bascd on rcsponscs from 604 ccrticd organizations)
How long to implement?
46 months
29%
03 months
9%
Over 2 years
2%
13 months
to 2 years
12%
712 months
48%
Time taken to implement and certify as
a function of length of time certifed
0
20
40
60
80
100
03 years 410 years over 10 years
Length of time certifed
P
e
r
c
e
n
t
a
g
e

o
f

o
r
g
a
n
i
z
a
t
i
o
n
s
No response
Over 2 years
13 months to 2 years
712 months
46 months
03 months
Key
Length of time taken
to implement system
and achieve certifcation
0
1
11
37
36
15
2
1
11
50
27
9
2
3
16
53
23
3
Use of consultants
Although there is no requirement for organizations to use a quality management consultant to help
vth the mpementuton o ther quuty munugement system, 8l% o the certed orgunzutons
surveyed chose to do so.
Interestingly, no trend was observed to show that the use of a consultant was associated with a
shorter mpementuton tme. As shovn n gure 27, there vus no stutstcuy sgncunt vuruton
in the percentage of organizations using a consultant as a function of the time taken to achieve
certcuton.
figurc 27. Pcrccntagc of rcspondcnts who uscd a consuItant during impIcmcntation of
thcir quaIity managcmcnt systcm, as a function of thc timc takcn to achicvc
ccrtication
\hut coud be observed, hovever, |see gure 28) vus un ncreusng trend or more recenty-certed
orgunzutons to use u consutunt thun dd orgunzutons certed or more thun l0 yeurs.
Figure 29 also shows that micro, small and medium enterprises had a greater tendency to use
consultants than had larger organizations.
Efect of using a consultant on time
taken to achieve certifcation
Time taken to achieve certifcation
Yes
No
P
e
r
c
e
n
t
a
g
e

o
f

o
r
g
a
n
i
z
a
t
i
o
n
s

r
e
s
p
o
n
d
i
n
g

Y
E
S

o
r

N
O
0
20
40
60
80
100
03
months
46
months
712
months
13 months
to 2 years
Over
2 years
figurc 28. Thc incrcasing tcndcncy among rcspondcnts from morc rcccntIy ccrticd
organizations to havc uscd a consuItant
figurc 29. Thc dccrcasing pcrccntagc of rcspondcnts using a consuItant as a function of thc
organization sizc (numbcr of cmpIoyccs)
Percentage of organizations using a consultant
as a function of length of time certifed
0.0
20.0
40.0
60.0
80.0
100.0
03 years 410 years over 10 years
Length of time certifed
P
e
r
c
e
n
t
a
g
e

o
f

o
r
g
a
n
i
z
a
t
i
o
n
s
No
Yes
Key
Did you use
a consultant?
15.1
84.9
14.3
85.7
34.2
65.8
Percentage of organizations using a consultant
as a function of length of organization size
Number of employees
P
e
r
c
e
n
t
a
g
e

u
s
i
n
g

a

c
o
n
s
u
l
t
a
n
t
50.0
60.0
70.0
80.0
90.0
100.0
1
5
1
7
6
2
7
5
6
6
8
5
1
6
2
5
8
6
1
2
5
1
2
6
1
7
5
4
6
6
5
1
1
1
5
2
6
4
5
2
7
6
4
2
5
6
1
0
O
t
h
e
r
1
1
7
6
1
5
5
0
8
7
6
1
1
7
5
6
2
6
8
7
5
4
2
6
6
2
5
This can be explained by the fact that larger organizations are more likely to have access to their own
resources, and may already have an existing management system (not necessarily meeting all the
requirements of ISO 9001) before beginning implementation.
Figure 30 provides details of how and for which activities consultants were used. It can be seen that, for
the initial phases of implementation, consultants were used primarily to assist the organizations to
develop their quality policy, quality manual, system level procedures and operating procedures/work
instructions. In a small percentage of cases, however, the consultant actually prepared these documents
for the organization, and as many as 22% of the organizations who used a consultant said that the
consultant had written their quality manual for them. This is not actually wrong, but does not help to
induce a sense of ownership of the quality management system by the organizations own personnel.
As might be expected, consultants were quite extensively used for employee awareness raising and
internal auditor training, and 15% of the organizations responding used the consultants to carry out
internal audits on their behalf.
Although the majority (55%) of the organizations carried out their management review without any
involvement of their consultant and a further 39% used the consultant to provide inputs, it is worrying
to see that 6% (a total of 30 organizations using consultants) responded that the consultant had done
the management review for them. Clause 5.6 1 of ISO 9001 is very clear Top management shall
review the organizations quality management system, at planned intervals, to ensure its continuing
suitability, adequacy and effectiveness and this is not something that can be delegated to a
consultant. Also of concern are the approximately 10% of organizations who responded that the
consutunt hud conducted u negotutons vth the certcuton body, negotuted the contruct vth
the certcuton body, und coordnuted uctvtes durng the certcuton body's ntu uudt.
figurc 30. Thc ways in which thc rcspondcnts had uscd thc scrviccs of thcir consuItant
Involvement of consultant (where used)
We did not use the consultant
for this activity
P
e
r
c
e
n
t
a
g
e

o
f

o
r
g
a
n
i
z
a
t
i
o
n
s

r
e
s
p
o
n
d
i
n
g

11
22
13 10
43
60
15
6
10 9
9
68
70
66
59
36
27
49
39
30
19
26
21
8
22
31
21
13
36
55
60
72
64
Q
u
a
lit
y
p
o
lic
y
M
a
n
a
g
e
m
e
n
t
r
e
v
ie
w
O
p
e
r
a
t
in
g
p
r
o
c
e
d
u
r
e
s
/
w
o
r
k
in
s
t
r
u
c
t
io
n
s
T
r
a
in
in
g
o
f
in
t
e
r
n
a
l a
u
d
it
o
r
s
C
a
r
r
y
in
g
o
u
t
in
t
e
r
n
a
l a
u
d
it
s
E
m
p
lo
y
e
e
a
w
a
r
e
n
e
s
s
t
r
a
in
in
g
S
e
le
c
t
io
n
o
f
c
e
r
t
if
c
a
t
io
n
b
o
d
y
F
lo
w
c
h
a
r
t
s
/
s
y
s
t
e
m
-
le
v
e
l p
r
o
c
e
d
u
r
e
s
Q
u
a
lit
y
m
a
n
u
a
l
C
o
o
r
d
in
a
t
io
n
d
u
r
in
g

c
e
r
t
if
c
a
t
io
n
b
o
d
y
a
u
d
it
C
o
n
t
r
a
c
t
n
e
g
o
t
ia
t
io
n
s
w
it
h
c
e
r
t
if
c
a
t
io
n
b
o
d
y
Consultant provided inputs,
but we did this ourselves
Consultant did this for us
Although there is no requirement for organizations to use a consultant, it is well known that the
competence of quality management system consultants and the ways in which they are used by an
organization can be key to the successful implementation of ISO 9001, particularly in SMEs. Although
there were no questions about its use in the current survey, it is recognized that the ISO 10019
standard (Guidelines for the selection of quality management system consultants and use of their
services[7]) is not widely known or utilized. This was evident from the reaction of participants in the
various international and local feedback workshops that were conducted at the end of the project.
ISO 10019 explains the various criteria that organizations may adopt when choosing a consultant,
und hov to use the consutunt's servce vsey und to muxmum benet vthout creutng un ongong
dependency (and cost). If this standard is to achieve the desired impact, though, it is vital that
organizations considering implementing a quality management system are made aware of its
existence before they enter into any contract with a consultant, and national standards bodies should
be encouruged to promote lSC l00l9 umong potentu users. Consutunts themseves mght uso nd
it useful to make potential clients aware of the content of this guidance, in order to help them to make
an informed choice about this important element, should they choose to use a consultant to help
implement their system.
Organzalons nccd lo bc madc awarc ol lhc gudancc n ISO 10019 bclorc lhcy
cnlcr nlo a conlracl wlh a QMS consullanl.
Cveru, gure 3l shovs thut the eve o sutsucton vth the consutunts vus very hgh |97% ruted
Very good or Good), but it must be remembered that this survey was carried out among
orgunzutons thut hud ucheved certcuton und so t s very key thut they voud be sutsed vth
their consultants performance.
figurc 31. Thc IcvcI of satisfaction with QMS consuItants uscd by thc 604 ccrticd organizations
that participated in the survey
Satisfaction with consultant (where used)
Satisfed
76%
Very satisfed
21%
Dissatisfed
2%
Very dissatisfed
1%
Economic impact and return on investment
It proved impossible to extract meaningful data from the question on the survey about the actual
costs o mpementuton und certcuton, n vev o the vdey vuryng sze und compexty o the
organizations involved, the various currencies, the length of time elapsed since many had achieved
certcuton, und the ubsence o reube records to tuke nto uccount nternu costs such us empoyee
time spent during implementation and on-the-job training. There was, however, a very important and
encourugng resut to u more generc queston. C the 604 orgunzutons surveyed, gure 32 shovs
that 98% responded that, independent of the amount they had invested, they considered the return
on their investment to have been Good (73%) or Very good (25%). Only 2% considered the
implementation of ISO 9001 not to have been a good investment, and none of the respondents
considered it to have been a complete waste of money.
figurc 32. Thc rcsponscs of 604 ccrticd organizations to thc qucstion RcgardIcss of whcthcr
you can quantify thc totaI costs, do you considcr that thc invcstmcnt (in your QMS)
was worthwhiIc!"
1he dcutes nvoved n soutng the eect o lSC 900l certcuton on the nuncu perormunce
of organizations and economies are well documented in the literature (see, for example, the work of
Herus, Dck und Cusedesus l9). Athough Corbett et u 20 huve shovn thut certed orgunzutons
do ndeed tend to out-perorm non-certed orgunzutons n smur ndustry sectors, t s not ceur
vhether certcuton s the cuusu uctor or the superor perormunce. ln other vords, does lSC 900l
mpementuton und certcuton enube orgunzutons to enhunce ther nuncu perormunce, or s
t thut the better-perormng orgunzutons choose to seek lSC 900l certcuton:
Regardless of these deliberations, the responses to the present survey on the overall return on
nvestment, couped vth the opnons expressed by certed orgunzutons uter n ths report |see
tube 8), provde sgncunt emprcu evdence o the postve economc mpuct o lSC 900l
mpementuton und certcuton.
Good investment?
Yes
73%
Very good
investment
25%
No
2%
Not at all
0%
Compatibility of ISO 9001 with other management
system standards
lgure 33 shovs thut 26% o the lSC 900l-certed orgunzutons thut responded to the survey uso
hud un envronmentu munugement system certed to lSC l400l. Less thun l0% hud occuputonu
health and safety management systems meeting the requirements of OHSAS 18001 [21], and less
thun 2% hud normuton securty munugement systems certed to lSC 2700l 22. Cther
certcutons hed by the orgunzutons surveyed ncuded vurous product certcutons und, n
some cuses, sector-specc certcutons to stundurds such us lSC/1Sl6949 |or the uutomotve
industry) [23].
figurc 33. Pcrccntagc of rcspondcnts hoIding ccrtications to othcr standards
lt vus nterestng to note thut o the l59 orgunzutons certed to both lSC 900l und lSC l400l,
9l% vere sutsed vth the current eve o computbty betveen the tvo stundurds.
As seen n gure 34, umost hu o these l59 orgunzutons hud mpemented the tvo stundurds us u
single, integrated system. Some 36% said their systems were only partially integrated and only
17% had implemented the two standards as completely separate systems.
One very disappointing result of the survey was the low level of awareness and utilization of the ISO
9004 gudunce stundurd 5 umong lSC 900l-certed orgunzutons. Cny l7% o the 604
respondents said they know and use ISO 9004. Some 42% had heard of the standard, but didnt
0
5
10
15
20
25
30
ISO 14001 ISMS (ISO 27001) OHSAS 18001
Other certifcations held
Others
ISO 9001-certifed organizations
holding other certifcations
P
e
r
c
e
n
t
a
g
e

o
f

r
e
s
p
o
n
d
e
n
t
s
use it and a surprising 42% were not even aware of its existence. As mentioned in chapter 1, ISO
9004 goes beyond the basic requirements of ISO 9001, and guides organizations towards sustained
success by identifying and meeting the needs and expectations not only of their customers but also
those of other interested parties over the long term and in a balanced way. If organizations in the
Asun deveopng economes ure to derve the muxmum benet rom ther quuty munugement
systems, t s mperutve thut they be encouruged to ook beyond certcuton. Athough t s
recognzed thut muny orgunzutons n these countres need lSC 900l certcuton or trude cupucty-
budng purposes, such certcuton shoud be u consequence o u ve-mpemented quuty
management system, and not an end in itself.
Nalonal slandards bodcs, consullanls, ccrlcalon bodcs and accrcdlalon bodcs all
havc a rcsonsblly lo makc ISO 9001-ccrlcd organzalons awarc ol lhc cconomc
bcncls ol lookng bcyond ccrlcalon lowards "suslancd succcss", usng lhc
ccrlcd qually managcmcnl syslcm as a slarlng onl.
Seecton o certcuton bodes by lSC 900l-certed
organizations
As shovn n gure 35, umost 80% o the orgunzutons surveyed hud chosen certcuton ether by
u brunch oce or u ocu runchsee o u oregn certcuton body. Less thun 20% vere certed by
ocu certcuton bodes. lnterestngy, umost 20% o the orgunzutons surveyed hud chunged
certcuton body ut some stuge. Some o the uctors thut nuence the choce o certcuton body
are examined later in this chapter.
Partially
integrated
36%
Fully integrated
47%
(percentage of 159 respondents who had both certifcations)
Separate
systems
17%
Integration of QMS and EMS
figurc 34. lcvcI of intcgration of QMS and fMS among thc 1S9 organizations hoIding
ccrtication to both lSO 9001 and lSO 14001
figurc 3S. Catcgory of ccrtication body uscd by thc 604 ccrticd organizations who
participated in the survey
As was the case with the purchasers (chapter 4), knowledge and awareness of the role of
uccredtuton vus poor umong certed orgunzutons. Athough most o the orgunzutons |72%)
vere ube to correcty nume the uccredtuton body vho hud uccredted ther ovn certcuton
body |und vhose nume uppeurs on ther certcute!), ony 40% vere ube to nume the ocu
accreditation body in their own economy and only a similar number (39%) were able to name any
accreditation body.
Awarcncss ol lhc IAF was also vcry low, wlh only 13% ol lhc ccrlcd
organzalons consullcd sayng lhal lhcy had hcard ol lhc IAF and kncw whal l dd,
and S3% sayng lhcy had ncvcr hcard ol l.
Crteru used to seect the certcuton body
The results presented in table 5 relate to the criteria and methodology used by the respondents
to seect ther certcuton body. ln euch cuse, the certed orgunzutons vere usked to comment
on their level of agreement or disagreement with a series of statements, and the distribution of
the results is shown. No attempt has been made to perform a statistical analysis of these results, in
view of the different nature and format of the distributions observed, but the tendencies can be
ceury dented smpy by observng the responses n the orm o hstogrums. \here comments
have been made, these are as a result of the relative and cumulative frequency of the various
responses.
Local CB
19%
Local branch of
international CB
69%
Dont know
4%
Franchisee of
foreign CB
8%
Category of certifcation body
TabIc S. Opinions of 604 ccrticd organizations about thcir ccrtication body scIcction proccss
Statcmcnt about thc
criteria for selecting
thc ccrtication body
lcvcI of agrccmcnt with thc statcmcnt
(604 rcsponscs)
Comments
(1) We had no input
our consultant
negotiated with the
cet|||ca||on body on
our behalf.
There was no great tendency
for consultants to choose the
certcuton body on behu o
the organization. This correlated
well with the results shown in
gure 30, vhere ony u smu
percentage of the organizations
surveyed said that contract
negotiations with the
certcuton body vere hunded
exclusively by their consultant.
(2) We made the
decision (about which
cet|||ca||on body |o
use) pute|y on out
own.
This corroborates the result in
(1). Organizations in general
were making their own decisions
ubout vhch certcuton body
to select.
(3) We took into
consideration
recommendations from
major customers.
A range of responses, showing
that approximately 50% of the
organizations did consider using
the sume certcuton body us
their major customers.
(4) We checked to
see uh|ch cet|||ca||on
body had |he bes|
reputation in our own
coun|ty/|ndus|ty sec|ot.
Strong tendency for organizations
to place importance on the
certcuton body's repututon n
the local country and in the
industry sector.
(5) It was important
|ot |he cet|||ca||on body
to have an international
reputation.
Very strong tendency for
organizations to place
mportunce on the certcuton
bodys international reputation.
(6) We chose the
cet|||ca||on body
because ue heatd |hey
were the easiest.
Organizations tended to deny
choosng the certcuton body
on the basis of reputation for
being easy.
(7) We chose the
cet|||ca||on body
because |hey uete
the cheapest.
Organizations in general
maintained that they did not
choose the certcuton body
based only on price.
(8) We didnt care
uhe|het |he cet|||ca||on
body uas accted||ed
or not.
Accreditation was generally
considered to be important
(despite the low level of awareness
of what accreditation actually
means).
Strongly disagree . . . . . . . . . . . . . . . . . . . . . . Strongly agree
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.4
0.2
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.4
0.2
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.1
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.4
0.2
0
Statcmcnt about thc
criteria for selecting
thc ccrtication body
(604 rcsponscs)
Comments
|o be accted||ed by out
national accreditation
body.
Local accreditation of the
certcuton body vus ony
considered to be important by
50% o the certed orgunzutons
surveyed.
|o be accted||ed by a
foreign accreditation
body.
More organizations gave
importance to foreign
accreditation than to local
accreditation.
|ot |he cet|||ca||on
body |o be accted||ed
by a membet o| |he
IAF (International
Accreditation Forum).
There were mixed responses
about the importance of the
certcuton body huvng
accreditation from an IAF MLA
signatory (which is hardly
surprising, given the low level
of awareness about the IAF).
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
Contucts vth the certcuton body pror to
the initial audit
ln gure 36, t cun be seen thut n the contucts vth the certcuton body pror to the ntu
certcuton uudt, t vus common or orgunzutons to huve teephone dscussons |65% o the
respondents) or meetngs ut the orgunzuton's premses |37%) to dscuss the certcuton process.
C the 604 orgunzutons surveyed, 34% hud hred the certcuton body to curry out u pre-uudt
|or gup unuyss) beore ntutng the certcuton uudt process. lt s mportunt to remember thut
ISO 17021:2006 [2] had not been published at the time that many of the organizations surveyed
vent through the certcuton process, und u tvo-stuge uudt vus not mundutory. lt cun be seen n
gure 37 thut the percentuge o orgunzutons optng or u pre-uudt by ther certcuton body n
recent yeurs hus decreused compured to the percentuge o orgunzutons certed or onger perods
of time who did so.
Although the numbers in absolute terms were low (6% of those surveyed), it was disturbing to see that
u number o certcuton bodes vere perormng n-house trunng |vhch, uccordng to lSC/lLC
17021, might be acceptable provided it is limited to the provision of generic information that is freely
uvuube n the pubc domun und does not provde compuny-specc soutons) or vere uctuuy
carrying out the organizations internal audits on their behalf (which is not acceptable, according to
ISO/IEC 17021 Clause 5.2.6). As noted in previous sections of this report, a small percentage of the
orgunzutons surveyed hud no contuct ut u vth ther certcuton body pror to the ntu uudt,
with all contacts being made via the consultants.
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
figurc 36. AnaIysis of thc kind of contacts bctwccn thc ccrticd organization and thc
ccrtication body prior to thc initiaI ccrtication audit (bascd on thc 604 rcsponscs
from ccrticd organizations)
figurc 37. Thc dccrcasing tcndcncy for morc rcccntIy-ccrticd organizations to havc
undcrtakcn a prc-audit by thcir ccrtication body (bascd on thc 604 rcsponscs
from ccrticd organizations)
Contacts with CB prior to initial audit?
Phone/e-mail to discuss
certifcation process
Meeting at our premises to
discuss certifcation process
Carried out pre-audit
Carried out our internal audits for us
Provided in-house training
All contacts were with consultants
0 20 40 60 80 100
% Yes
% No
Less than 3 years
410 years
Over 10 years
0 20 40 60 80 100
Percentage of organizations surveyed

L
e
n
g
t
h

o
f

t
i
m
e

c
e
r
t
i
f
e
d
Yes
No
Percentage of organizations having a pre-audit
Audit durations
Interestingly, in spite of previous anecdotal evidence to the contrary, the results showed that (based
on the responses o the 604 certed orgunzutons vho responded to the survey) certcuton
bodies, in general, respect the audit durations prescribed by the IAF Mandatory Document
MD5 [8].
lgure 38 shovs the uctuu number o uudtor-duys reported by the certed orgunzutons or the
utest u |ntu or recertcuton) uudt potted ugunst the theoretcu number o uudtor-duys
(according to IAF Mandatory Document MD5 [8]) as a function of organization size. It is reasonable
to expect that the number of auditor-days reported by the organizations is accurate since they are
unkey to be umur vth the detued content |or even the exstence!) o lAl MD5 und there vus
no evdence to suggest they hud been couched by ther certcuton body beore provdng ther
responses.
For each organizations size category, the calculated theoretical audit duration took into consideration
the ength o tme the orgunzutons n thut cutegory hud been certed or orgunzutons thut
hud been certed or more thun three yeurs, the recommended duruton or u recertcuton uudt
was considered.
Theoretical audit days = _MD5 uudtor-duys |orgs certed 3yrs) +2/3_MD5 uudtor-duys |orgs certed >3yrs)
Number of organizations in size category
Actual audit days = _ uctuu uudtor-duys
Number of organizations in size category
It is important to note, however, that ISO 17021:2006 does not prescribe the number of auditor-days
bused on the sze o the orgunzuton, t requres ony thut the certcuton body denes the tme
needed to plan and accomplish a complete and effective audit of the clients management system.
This can vary depending on a number of factors, including:
- 1he requrements o the reevunt munugement system stundurd,
- 1he sze und compexty o the orgunzuton,
- 1he technoogcu und reguutory context,
- Any outsourcng o uny uctvtes ncuded n the scope o the munugement system,
- 1he resuts o uny pror uudts,
- The number of sites and multi-site considerations.
Aso ncuded n gure 38 s the theoretcu uudt tme mnus 30%, vhch s the muxmum reducton
permitted by IAF MD5 [8] to take into consideration organizations with excluded ISO 9001
requirements, simple and/or repetitive processes, and other factors that might justify a reduction in
audit duration.
It can be seen that for micro- and small organizations (below 25 employees), the IAF guidelines are
actually exceeded. For larger organizations, the actual audit durations are below those prescribed in
IAF MD5 but generally within the limits of the 30% reductions in duration that may be permitted.
A smur puttern emerged or surveunce uudts |gure 39), but n ths cuse the uctuu uudt durutons
were higher than those recommended by IAF MD5 for organizations below approximately
125 employees in size and above 625. In no cases did the number of auditor-days fall below the
maximum 30% reduction allowed by IAF MD5 for surveillance audits.
figurc 39. Graph showing thc variation of actuaI auditor-days comparcd to thc thcorcticaIIy
caIcuIatcd numbcr of auditor-days (according to lAf MDS) for survciIIancc audits,
bascd on thc rcsponscs from 604 ccrticd organizations
figurc 38. Thc variation of actuaI auditor-days comparcd to thc thcorcticaIIy caIcuIatcd numbcr
of auditor-days (according to lAf MDS [8]) for fuII (initiaI or rcccrtication) audits,
bascd on thc rcsponscs from 604 ccrticd organizations
Actual auditor-days and theoretical auditor-days for initial and recertifcation audits
as a function of organization size (number of employees)
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
A
u
d
i
t
o
r
-
d
a
y
s
15 610 1115 1625 2645 4665 6685 86125 126175 176275 276425 426625 626875 8761175 11761550
Actual
Theoretical
Theoretical minus 30%
Number of employees
0.00
1.00
2.00
3.00
4.00
5.00
6.00
Actual auditor-days and theoretical auditor-days for surveillance audits
as a function of organization size (number of employees)
A
u
d
i
t
o
r
-
d
a
y
s
Actual
Theoretical
Theoretical minus 30%
15 610 1115 1625 2645 4665 6685 86125 126175 176275 276425 426625 626875 8761175 11761550
Number of employees
lntu certcuton process
Regurdng the ntu uudt process, the certed orgunzutons vere usked to comment on ther eve
of agreement with a series of statements, as reported in table 6.
TabIc 6. Opinions of 604 ccrticd organizations about thc initiaI ccrtication audit
Statcmcnt about thc
initiaI ccrtication
audit process
(604 rcsponscs)
Comments
(1) The audit process
was challenging.
1he muorty o the certed
organizations surveyed considered
that the audit process was
challenging, with only 10%
disagreeing.
(2) The audit process
was fair and impartial.
There was strong consensus that
the initial audit process was fair
and impartial.
(3) The audit was
|oo easy.
There was strong consensus that the
audit process was not too easy,
with only 10% of the respondents
disagreeing. This is consistent with
the results of Item 1.
(4) The auditors
were professional and
competent.
In general, there was good
agreement that the auditors were
professional and competent in
their approach.
(5) The auditors had
a good understanding
of our business and
processes.
1he certed orgunzutons
generally agreed that the auditors
had a good understanding of their
business processes. This is
interesting, and contradicts some
of the anecdotal evidence that has
often been cited.
(6) The auditors spent
most of their time looking
at procedures and
records.
There were mixed opinions about
whether or not the auditors spent
most of their time looking at
documentation, with half of the
respondents agreeing, and half
disagreeing. It is clear, however,
that some auditors need to pay
attention to this concern, and to
focus less on documentation.
(7) The auditors were
keen to observe our
processes in operation,
and to talk to the
emp|oyees |nvo|ved.
In some ways, this contradicts the
previous topic, and organizations
conrmed thut uudtors vere
observing processes in operation,
and talking to the appropriate
employees.
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
Statcmcnt about thc
initiaI ccrtication
audit process
(604 rcsponscs)
Comments
(8) The auditors told
us uha| |hey |hough| ue
should be doing.
to this topic, but it is clear that
u sgncunt percentuge o
auditors were perceived to be
giving advice which, if it is
speccuy drected tovurds
providing instructions or
solutions, is considered by
ISO/IEC 17021 Clause 3.3 to
be consultancy and is not
permitted.
(9) The auditors spent
enough time interviewing
our top management.
There was a clear tendency
to agree that auditors were
spending enough time talking
to top management, although
as many as 8% of the
organizations surveyed disagreed
quite strongly.
(10) The auditors
tate|y |e|| |he
management
teptesen|a||ves o||ce.
There was a strong tendency to
conrm thut uudtors vere not
spending too much time only
talking to the management
representative, although this was
the case in approximately 8% of
the respondents.
(11) The auditors were
ab|e |o |den|||y ptob|em
ateas |n out sys|em and
raise the appropriate
non-conformities.
There was good agreement that
auditors were able to identify
problems in the system and were
raising the appropriate
nonconformities.
(12) Our consultant
answered most of the
auditors questions.
Certed orgunzutons strongy
disagreed that their consultant
answered most of the auditors
questions.
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.5
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
Surveillance audits
Figure 40 shows that the most commonly reported surveillance frequency was yearly (69%), but with
25% maintaining a six-monthly frequency and 7% reporting a 9-monthly cycle (to include an initial or
recertcuton uudt pus 3 surveunce uudts durng u 3-yeur certcuton cyce). lSC/lLC l702l 2
requres certcuton bodes to curry out surveunce ut eust once u yeur. Athough not requent, u
total of 8 organizations claimed they were audited less than once a year, though it is important not
to automatically draw negative conclusions from this fact, which could have been a result of multi-
ste sumpng crteru vhereby not every ste vthn u mut-ste scope o certcuton hus to be
uudted every yeur. 1hs vus not vered durng the proect.
figurc 40. Graph showing thc frcqucncy of survciIIancc audits rcportcd by thc 604 ccrticd
organizations survcycd
ln terms o the surveunce uudt process, the certed orgunzutons vere ugun usked to comment
on their level of agreement with a series of statements. The results are shown in table 7.
TabIc 7. Opinions of 604 ccrticd organizations about thcir survciIIancc audit proccss
Every
6 months
23%
Once a year
69%
Every 9 months
7%
Less frequently
than once a year
1%
Surveillance frequency
Statcmcnt about
the surveillance
audit process
(604 rcsponscs)
Comments
(1) The surveillance
audits ensure that our
sys|em con||nues |o mee|
the requirements of ISO
9001.
Cood eves o condence vere
expressed about the effectiveness
of the surveillance audit process.
audits encourage us to
|mptove out sys|em.
There was strong agreement that
the surveillance audit process
encouraged continual
improvement.
aud||s ate vety use|u|.
The results show that surveillance
audits were considered to be
useful.
aud||s ate |ocused on|y
on documentation.
1he muorty o certed
organizations did not agree that
the surveillance audits only
focused on documentation,
although 15% agreed with this
statement.
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
Statcmcnt about
the surveillance
audit process
(604 rcsponscs)
Comments
(S) 1he cet|||ca||on
body a|uays uses |he
same auditor(s) for
surveillance audits.
There were mixed responses to this
statement, as might be expected.
Whilst using the same audit team
can often make the surveillance
uudts more ecent, some rotuton
of the auditors was to be
encouraged in order to minimize
excessive familiarity, which can
be a potential threat to impartiality
(ISO/IEC 17021 Clause 4.2.4).
(6) The auditor never
ta|ses any non
conformities during the
There was a strong tendency to
show that, in the opinion of
certed orgunzutons, the
certcuton body uudtors vere
raising nonconformities during
(7) \e |nd |he
surveillance audit report
|o be vety use|u|.
There was strong agreement
umong certed orgunzutons
that the surveillance audit reports
were useful.
(8) We continue to get
va|ue |ot money u||h out
cet|||ca||on.
Certed orgunzutons vere
sutsed vth ther return on
investment in the surveillance
process.
Cveru sutsucton vth uccredted certcuton
1he resuts obtuned rom the 604 certed orgunzutons surveyed ubout ther overu sutsucton
vth uccredted certcuton ure shovn n tube 8.
TabIc 8. Opinions of 604 ccrticd organizations about thcir ovcraII satisfaction with accrcditcd
ccrtication
Statcmcnt about
accrcditcd ccrtication
(604 rcsponscs)
Comments
(1) Our top
management has been
|o|a||y comm|||ed |o |he
implementation of
ISO 9001.
The respondents considered that
top management of their
organization had been committed
to the implementation of ISO 9001.
Note: during the local feedback
seminars, many informal comments
vere receved rom the certed
organizations present to suggest
that the responses to this question
might be suspect in view of the
involvement of top management in
providing responses.
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.4
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
Statcmcnt about
(604 rcsponscs)
Comments
(2) We have derived
s|gn||can| bene|| |tom
implementing ISO 9001
in our organization.
Amost u o the certed
organizations that responded to
the survey considered the
implementation of their quality
management system to have been
benecu.
(3) ISO 9001
certcuton has helped
us to achieve our
objectives.
There was a good level of
satisfaction that ISO 9001
certcuton hud heped the
organization to achieve the
objectives that had stimulated it to
implement a QMS and to seek
certcuton.
(4) ISO 9001
cet|||ca||on has he|ped
us |mptove |he uay
we work.
There was good agreement that
lSC 900l certcuton hud heped
organizations to improve their
work processes.
(5) ISO 9001
cet|||ca||on has ptov|ded
us u||h mote con|dence
|n out ab||||y |o ptoduce
conforming products.
There was good agreement
among the respondents that
certcuton provded more
condence n the certed
organizations abilities to provide
conforming products.
(6) Our customers
give us credit for our ISO
0661 cet|||ca||on uhen
|hey choose |he|t
suppliers.
ln generu, the certed
organizations surveyed considered
that their customers valued the
fact that they were ISO
900l-certed.
(7) ISO 9001
cet|||ca||on has teduced
the number of audits we
receive from our
customers.
about whether or not ISO 9001
certcuton hud reduced the
number of second-party audits
thut certed orgunzutons
received from their customers.
(8) ISO 9001 has
helped us to focus on
con||nua||y teduc|ng
internal re-work and
waste.
In general, there was agreement
that the implementation of the
QMS had helped reduce rework
and waste, thereby contributing to
overu orgunzutonu ecency us
well as effectiveness.
(9) We have obtained
more business because of
our ISO 9001
cet|||ca||on.
There was a tendency to agree that
lSC 900l certcuton hud resuted
in more business, although 10% of
the respondents disagreed.
(10) We can get a
higher price for our
products as a result of
cet|||ca||on.
Most of the respondents did not
ugree thut lSC 900l certcuton
allowed them to charge higher
prices for their products.
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
Statcmcnt about
(604 rcsponscs)
Comments
(11) We have a much
better understanding
of our processes as a
result of ISO 9001
implementation and
cet|||ca||on.
There was general agreement that
the implementation of ISO 9001
und certcuton resuted n u
better understanding of the
organizations processes.
(12) ISO 9001 has
obliged us to produce
too much un-necessary
documentation.
about whether or not ISO 9001
had necessitated too much
unnecessary documentation.
Of the 604 respondents, 30%
considered that they had been
obliged to produce too much
unnecessary documentation,
whilst 70% disagreed.
(13) We have never
set|ous|y cons|deted
chang|ng out cet|||ca||on
body.
Most of the 604 respondents
(75%) had never seriously
considered changing their
certcuton body.
(14) We have never
set|ous|y cons|deted
cancelling our
cet|||ca||on (on out
own initiative).
Athough most o the certed
organizations had never seriously
considered cancelling their
certcuton |on ther ovn
initiative), approximately
15% had considered this.
(15) Overall, the
implementation of our
qua|||y managemen|
sys|em and cet|||ca||on
to ISO 9001 has been a
good investment.
There was strong agreement
among the respondents that
ISO 9001 implementation and
certcuton hud been u good
investment. This is consistent with
other responses throughout this
survey.
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y0.2
0.1
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.2
0
'
R
e
l
a
t
i
v
e
f
r
e
q
u
e
n
c
y
0.3
0.25
0.2
0.15
0.1
0.05
0
58
6. ASSESSMENT OF CERTIFIED
ORGANIZATIONS
Overview
A totu o 56l certed orgunzutons vere vsted n the tveve countres, usng the sumpng crteru
shown in table 3. The experiences and observations during both the planning and execution of these
vsts provded vuuube nsghts nto the eectveness o the uccredtuton und certcuton processes,
us ve us the eve o condence n the certed orgunzuton's ubty to meet the ntent o lSC 900l.
A summury o the sumpng process or punnng the vsts to certed orgunzutons und the uctuu
outcomes cun be seen n gure 4l. 1hs v be dscussed n more detu uter n ths chupter.
ln u contucts vth certcuton und uccredtuton bodes, t vus emphuszed thut ths proect vus
being carried out under the overall coordination of UNIDO, with collaboration from ISO and the IAF,
und thut u the consutunts nvoved hud sgned strct condentuty ugreements us purt o ther
contracts. This was supported by formal letters of introduction from the President of the IAF and the
Sampling plan for visits to certifed organizations
{
{
- Wlde varlatlon ln lnformatlon avallable and collaboratlon from A8s
- Splrlt of |AP cross-frontler accredltatlon pollcy not yet fully
embraced or lmplemented
- Peedback already glven to |SO and |AP durlng lnternatlonal
workshops, and concerns expressed
- Some were pro-actlve and transparent
- Others agreed reluctantly
8UT
- General lack of cooperatlon from some forelgn-accredlted C8s
- Pefuslng to provlde lnformatlon
- "Delaylng" tactlcs
- Peedback already glven to |SO and |AP durlng lnternatlonal
workshops, and actlons belng taken

- Peasonable level of collaboratlon ln recelvlng vlslts

Planned Actual
Consult all A8 members of |AP whlch
accredlted C8s are lssulng certs ln the
countrles lnvolved ln the pro[ect!
Consult C8s for "Dlrectory of certled cllents"
(|SO/|LC l702l Clause 8.3)
Pandom sample of certled organlzatlons
(manufacturlng/constructlon only)
Control sample of good" and "bad" suppllers (as
determlned by the lntervlews wlth purchasers)
+
{
figurc 41. Summary of sampIing pIan for visits to ccrticd organizations
6. ASSESSMENT OF CERTIFIED ORGANIZATIONS 59
Secretury-Ceneru o lSC. Lven so, ve encountered u generu reuctunce umong muny certcuton
bodies and accreditation bodies to collaborate in the project by providing the information that was
beng requested durng the punnng stuges o the vsts to certed orgunzutons.
There was a reasonable level of collaboration from the organizations that were eventually selected
for visits, when one takes into account the relatively short time frame in which the visits were carried
out. Overall, 46% of the organizations contacted agreed to the visit. Although most of the reasons
given for not receiving the visit were related to pressure of work, lack of resources or other
commitments, it is likely that some of the organizations did not want to be visited due to concerns
about the robustness of their quality management system. The reasons for denying the visits could not
be determined in all cases, however, and it should not necessarily be implied that those organizations
had something to hide. Also worthy of note is that, on many occasions, organizations had agreed in
prncpe to purtcpute und uter bucked out |uter dscusson vth ther certcuton body und/or
consultant). As a consequence, the results of the project once again have to be analysed with caution,
vth u recognton thut the sumpng vus mted to the vng certcuton bodes und ther cents
and with no mechanism for making participation mandatory.
lerormunce o the certed orgunzutons
Overall results
1he resuts o the 56l one-duy vsts to lSC 900l-certed orgunzutons ure presented n tube 9 or
euch o the topcs covered by the proect-specc check-st. used on these vsts, the overu eve
o condence n the quuty munugement system o the orgunzutons vsted vus generuy good,
with 92% of the results falling into the following UNIDO consultant-assessment categories:
- We can be proud to use this organization as a benchmark (6%)
- Clear evidence that the intent of ISO 9001 is being met (43%)
- OK No reason to doubt that this is being addressed correctly (43%)
It should also be noted, however, that 8% of the organizations visited fell into the categories:
- Some evidence presented, but not at all convincing (7%)
- Ltte or no condence |l%)
\hst these percentuges reectng dssutsucton ure smu, they do represent u sgncunt number
of organizations in absolute terms (over 5,000 in the region as a whole). This number could be higher,
vhen ve consder the uvourube sumpng crteru |ony the vng certcuton bodes und
certed orgunzutons ugreed to be vsted), und t voud be unvse to use these resuts us u reuson
for complacency.
C equu concern s the ndng thut n l4% o the vsts the LNlDC consutunts ound tte
|l3%) or no |l%) condence thut the certcuton process hud been conducted eectvey by the
certcuton body.
TabIc 9. RcsuIts of visits to ccrticd organizations
Chcck-Iist qucstion OvcraII gradcs (S61 organizations)
|l) 1he ntu |or most recent u recertcuton) uudt duruton
was appropriate for the size and complexity of the
organization (See IAF MD5 [8]).
(2) The duration and frequency of surveillance audits are
appropriate for the size and complexity of the organization
(see Clause 5 of IAF MD5 [8]).
|3) 1he scope mentoned on the orgunzuton's certcute
accurately describes its activities and is not misleading.
|4) A excusons ure udequutey usted.
(5) There is evidence of top managements involvement with
and commitment to the implementation of ISO 9001.
(6) Internal communication is good, and employees are aware
of their roles in the QMS.
kcy: 1 littIc or no condcncc", 2 Somc cvidcncc prcscntcd, but not at aII convincing", 3 Ok - No
rcason to doubt that this is bcing addrcsscd corrcctIy", 4 CIcar cvidcncc that thc intcnt of lSO 9001 is
bcing mct", S Wc can bc proud to usc this organization as a bcnchmark".
Level 4
38%
Level 3
34%
Level 1
2%
Level 5
8%
Level 2
18%
Initial audit duration
Level 4
50%
Level 3
26%
Level 1
2%
Level 5
13%
Level 2
9%
Surveillance frequency and duration
Level 4
63%
Level 3
22%
Level 1
1%
Level 5
13%
Level 2
1%
Certifcation scope
Level 4
59%
Level 3
26%
Level 1
2%
Level 5
11%
Level 2
2%
Exclusions
Level 4
50%
Level 3
31%
Level 1
0%
Level 5
11%
Level 2
8%
Top management commitment
Level 4
45%
Level 3
42%
Level 1
0%
Level 5
4%
Level 2
9%
Internal communication
(7) The process approach is clearly understood and
implemented throughout the organization.
(8) The organization is managing its QMS processes using a
Plan-Do-Check-Acttype approach (ISO 9001 Clause 4.1).
(9) The quality policy is appropriate for the organizations
situation and culture.
(10) The organization has established and deployed meaningful
objectives at relevant functions and levels.
(11) The quality manual is a good representation of the way
the organization actually works.
(12) QMS documentation is being used and is properly
controlled.
Level 4
49%
Level 3
39%
Level 1
1%
Level 5
5%
Level 2
6%
Objectives
Level 4
39%
Level 3
43%
Level 1
1%
Level 5
3%
Level 2
14%
Process approach
Level 4
33%
Level 3
42%
Level 1
1%
Level 5
5%
Level 2
19%
Use of PDCA
Level 4
59%
Level 3
31%
Level 1
1%
Level 5
6%
Level 2
3%
Quality policy
Level 4
44%
Level 3
36%
Level 1
1%
Level 5
11%
Level 2
8%
Quality objectives
Level 4
54%
Level 3
37%
Level 1
1%
Level 5
3%
Level 2
5%
Quality manual
(13) The organization has adequate resources (competent
personnel, equipment, etc.) to support its system.
(14) The work environment is appropriate.
|l5) ley product reuzuton processes ure dented und
managed.
(16) Processes are being adequately monitored and measured.
|l7) lroduct nonconormtes ure dented und deut vth
according to documented procedures.
(18) There is a focus on identifying the cause of process, product
and system nonconformities, and on implementing effective
corrective action.
Level 4
56%
Level 3
34%
Level 1
1%
Level 5
4%
Level 2
5%
Resources
Level 4
42%
Level 3
42%
Level 1
1%
Level 5
6%
Level 2
9%
Work environment
Level 4
54%
Level 3
37%
Level 1
1%
Level 5
4%
Level 2
4%
Product realization processes
Level 4
54%
Level 3
36%
Level 1
1%
Level 5
4%
Level 2
5%
Process monitoring and measurement
Level 4
47%
Level 3
39%
Level 1
1%
Level 5
3%
Level 2
10%
Product nonconformities
Level 4
28%
Level 3
50%
Level 1
1%
Level 5
5%
Level 2
16%
Cause analysis/corrective action
(19) Internal audits are being carried out according to plan
and are effective.
(20) Management reviews are being carried out according
to plan and are effective.
(21) The organization has a focus on preventing
nonconformities.
(22) Customer feedback and customer complaints handling
mechanisms are appropriate.
|23) 1he MS s provdng condence n the orgunzuton's
ability to consistently provide product that meets customer
and applicable statutory and regulatory requirements.
(24) The organization has a culture of continual improvement
of the effectiveness of its QMS.
Level 4
44%
Level 3
39%
Level 1
2%
Level 5
3%
Level 2
12%
Internal audits
Level 4
49%
Level 3
34%
Level 1
2%
Level 5
5%
Level 2
10%
Management review
Level 4
30%
Level 3
47%
Level 1
1%
Level 5
3%
Level 2
19%
Preventive action
Level 4
48%
Level 3
40%
Level 1
1%
Level 5
4%
Level 2
7%
Customer feedback/complaints
Level 4
52%
Level 3
38%
Level 1
0%
Level 5
5%
Level 2
5%
Ability to provide conforming product
Level 4
34%
Level 3
43%
Level 1
1%
Level 5
5%
Level 2
17%
Continual improvement
Weak areas
1he veukest ureus o mpementuton thut vere dented durng the vsts vere:
- Luck o ocus on preventve ucton |ltem 2l o the checkst),
- Poor use of the Plan-Do-Check-Act approach (ISO 9001 Clause 4.1) to manage the
quuty munugement system processes |ltem 8 o the checkst),
- loor cuture o contnuu mprovement |ltem 24 o the checkst),
- Lack of adequate cause analysis and effective corrective action for process, product and
system nonconormtes |ltem l8 o the checkst),
- Little use of the process approach throughout the organization (Item 7 of the checklist).
lnterestngy, no muor concerns vere dented n the customer compunts hundng process,
consderng thut ths hud been dented us one o the mun crtcsms mude by nsttutonu purchusers
ubout ther lSC 900l-certed suppers.
Correlation between the system performance and the length of time the organization
had been certifed
As mght be expected, mproved eves o condence n the overu perormunce o certed
orgunzutons vere observed us u uncton o the ength o tme certed und the muturty o the
quuty munugement system. 1hs cun be seen n gure 42.
Correlation between the system performance and the time taken to implement the system
Figure 43 shows that organizations claiming to have implemented their quality management system
und ucheved certcuton n ess thun 6 months hud u tendency not to perorm us ve on the overu
|25) 1he certcuton process hus been conducted eectvey
by the certcuton body.
|26) Cveru condence n ths orgunzuton's mpementuton
of ISO 9001.
Level 4
40%
Level 3
44%
Level 1
1%
Level 5
2%
Level 2
13%
Certifcation process
Level 4
43%
Level 3
43%
Level 1
1%
Level 5
6%
Level 2
7%
Overall confdence
condence score us orgunzutons thut hud tuken onger to mpement the system |regurdess o the
ength o tme they hud been certed). 1he utter uso ucheved u hgher percentuge o Crude 5
scores, ndcutng un exceent |benchmurk) eve o condence, thun dd the ones vho hud
ucheved certcuton n u short ength o tme.
figurc 42. OvcraII IcvcIs of condcncc in thc quaIity managcmcnt systcm as a function
of thc Icngth of timc thc organization had bccn ccrticd (bascd on thc visits to
S61 ccrticd organizations)
figurc 43. OvcraII IcvcIs of condcncc as a function of thc Icngth of timc takcn by thc
organization from thc dccision to impIcmcnt a QMS untiI thc achicvcmcnt of
ccrtication (bascd on thc visits to S61 ccrticd organizations)
Over 10 years
410 years
Less than 3 years
0 10 20 30 40 50 60 70 80 90 100
Percentage of organizations achieving confdence level 1 to 5
Level 1
Level 2
Level 3
Level 4
Level 5
Overall
confdence
level

L
e
n
g
t
h

o
f

t
i
m
e

c
e
r
t
i
f
e
d
Overall confdence level as a function of system maturity
Over 2 years
13 months to 2 years
712 months
46 months
Less than 3 months
0 10 20 30 40 50 60 70 80 90 100
Level 1
Level 2
Level 3
Level 4
Level 5
Overall
confdence
level
Overall confdence level as a function of time taken to implement system
T
i
m
e

t
a
k
e
n

t
o

i
m
p
l
e
m
e
n
t

Q
M
S
Correlation between the system performance and the size of the organization
lgure 44 conrms unecdotu evdence thut suggests thut smu orgunzutons |vho uso tended to
mpement ther systems qucky) demonstruted u over overu eve o condence thun urger
organizations. This can also be explained by the fact that larger organizations usually have more
resources available for the implementation of their quality management system.
figurc 44. OvcraII IcvcIs of condcncc as a function of thc organization sizc (bascd on thc visits
to S61 ccrticd organizations). Micro" is Icss than 10 cmpIoyccs, smaII" is 11 to
S0 cmpIoyccs, mcdium" is S1 to 2S0 cmpIoyccs, and Iargc" is ovcr 2S0 cmpIoyccs
(Scc, for cxampIc, furopcan Commission Rccommcndation 2003/361/fC[18])
Performance of the accreditation bodies
Requirements for IAF membership
In order to be recognized under the IAF MLA, accreditation body members of the IAF are required to
comply with the requirements of ISO/IEC 17011:2004 Conformity assessment General
requirements for accreditation bodies accrediting conformity assessment bodies[10] as well as a
series of IAF mandatory documents for the consistent application of that standard. The objective is to
ensure thut u uccredted certcuton bodes ure meetng the requrements dened n lSC/lLC l702l,
und thut the eves o condence n orgunzutons hodng lSC 900l certcutes ssued by certcuton
bodies accredited by different IAF accreditation body members are comparable, thereby facilitating
world trade.
1here ure, hovever, stuutons vhere certcuton bodes v seek more thun one uccredtuton, or u
oregn uccredtuton, und t s the certcuton body's rght to do so. Nevertheess, lAl strongy
encouruges certcuton bodes to huve ocu country uccredtuton vhenever t s uvuube. ln muny
o the Asun deveopng economes, there s u strong presence o certcuton bodes uccredted by
Large
Medium
Small
Micro
0 10 20 30 40 50 60 70 80 90 100
Level 1
Level 2
Level 3
Level 4
Level 5
Overall
confdence
level
Overall confdence level as a function of organization size
O
r
g
a
n
i
z
a
t
i
o
n

s
i
z
e
foreign accreditation bodies, due in some cases to the non-existence or relatively recent
establishment of the local accreditation body, the perception that foreign accreditation is more
prestgous, or the uct thut some mutnutonu certcuton bodes |qute egtmutey) operute u
their branches globally under a single accreditation, normally issued in their home country.
The IAF Guidance Document GD3 [24] (Guidance on Cross-Frontier Accreditation), which, in
spite of its title, was deemed a mandatory document at the time of publication, is aimed at ensuring
transparency and collaboration between accreditation bodies operating in any given country.
Clause 2.1.1 of IAF GD3 states that:
IAF MLA signatory ABs should record the countries in which each CB accredited by them issues
certcutes under ther uccredtuton. 1hs ncudes:
- Countres nto vhch uccredted certcutes ure ssued drecty rom the C's heud or other
oce, und
- Countres n vhch the C operutes rom ocu oces, vhutever the egu reutonshp o
such oces vth the purent C.
Availability of information from accreditation bodies
During the planning stages of the project, all the IAF accreditation body members were contacted and
requested to provde detus o ther uccredted certcuton bodes vho operuted n the Asun economes
within the project scope. Of the 56 accreditation body members of the IAF, only 24 responded formally.
\hst some members vho dd not respond vere unkey to huve uccredted certcuton bodes
operating in this region (for example, South American accreditation bodies), that was not the case for
some European accreditation bodies who failed to respond, even after follow-up requests.
1he eve o detu provded by the uccredtuton bodes uso vured sgncunty:
- Some uccredtuton bodes muntuned on-ne dutubuses o u certcutes |vth detus o
the respectve certcuton bodes) thut ure under ther uccredtuton. 1hs trunspurency s
to be applauded and goes beyond what is required by IAF GD3.
- Some accreditation bodies were able to provide details of the countries in which their
uccredted certcuton bodes ure operutng |vthn the sprt o lAl CD3).
- One large accreditation body responded that they did not have data readily available.
- Cne uccredtuton body vth u sgncunt presence n Asu reerred us to ts vebste to
collate these data ourselves but with the warning that its probably out of date.
1here vus uso u consderube vuruton n the vngness o certcuton bodes uccredted by
different accreditation bodies to provide the information required by ISO 17021 Clause 8.3 (see
later), and in the level of support from their respective accreditation bodies to require or encourage
them to do so. 1hs s reected n the oovng dutu or three o the nternutonu uccredtuton bodes
(ABs) with a major presence in the Asian developing economies.
- AB1 Provided all the data required by Clause 8.3 on its publicly-accessible website.
- A2 lrompted u ts uccredted certcuton bodes to provde the dutu requested, und
uter much oov-up by the A over u perod o 6 veeks, l00% o ts certcuton bodes
provided the information.
- A3 Less thun hu o the uccredted certcuton bodes provded the normuton
within two months of the request, despite numerous contacts and follow-ups.
1hese vurutons conrm the unecdotu evdence thut the requrements o lAl CD3 ure st not uy
embruced or mpemented by the lAl's uccredtuton body members, ths needs to be uddressed by
the IAF.
Overall performance of organizations certifed under diferent accreditations
lgures 45 und 46 shov the overu condence eves o the quuty munugement system runked
uccordng to the derent uccredtuton bodes thut hud uccredted the orgunzutons' certcuton
bodes. ln cuses vhere the certcuton body n queston hed tvo or more uccredtutons, both |or
all) those accreditation bodies were included in the analysis.
Figure 45 sorts the accreditation bodies in order of the percentage of the organizations exhibiting
poor perormunce |condence eves l + 2), rungng rom best to vorst, vhst lgure 46 provdes
the same data, sorted in order of the percentage of organizations exhibiting good performance
|condence eves 4 + 5), ugun rungng rom the best to vorst.
Note: Cny uccredtuton bodes |As) vth l5 or more certed orgunzutons vsted vere ncuded
in this analysis.
figurc 4S. OvcraII condcncc IcvcIs as a function of AB in ordcr of thc pcrccntagc of thc
organizations cxhibiting poor" pcrformancc (condcncc IcvcIs 1 + 2), ranging
from bcst to worst (bascd on thc visits to S61 ccrticd organizations)
0 10 20 30 40 50 60 70 80 90 100
Level 1
Level 2
Level 3
Level 4
Level 5
Overall
confdence
level
Overall performance of certifed organizations as a function of AB
A
c
c
r
e
d
i
t
a
t
i
o
n

b
o
d
y

c
o
d
e
AB9
AB8
AB7
AB2
AB6
AB3
AB5
AB4
AB1
figurc 46. OvcraII condcncc IcvcIs as a function of AB in ordcr of thc pcrccntagc of thc
organizations cxhibiting good" pcrformancc (condcncc IcvcIs 4 + S), again
ranging from thc bcst to worst (bascd on thc visits to S61 ccrticd organizations)
lt s ceur thut there ure vurutons n perormunce o orgunzutons thut huve been certed by
certcuton bodes operutng under derent uccredtutons. Codes huve been used to preserve the
anonymity of the accreditation bodies, but all the accreditation bodies mentioned have been informed
of their individual codes, and it is expected that each analyse their own performance and take the
appropriate actions.
lerormunce o the certcuton bodes
Requirements for accreditation
ln order to be uccredted, certcuton bodes huve to meet the requrements o lSC/lLC l702l:20062
|Conormty ussessment Requrements or bodes provdng uudt und certcuton o munugement
systems), which is based on six core principles impartiality, competence, responsibility, openness,
condentuty, und responsveness to compunts us ve us u seres o lAl mundutory documents
for the consistent application of that standard. The objective is to ensure that the organizations being
certed by uccredted certcuton bodes ure meetng the requrements o lSC 900l und uchevng
the expected outcomes of that standard. These expected outcomes are explained in detail in the Joint
lSC/lAl Communque Lxpected outcomes or uccredted certcuton to lSC 900lll und
essentuy ocus on the certed orgunzuton's ubty to consstenty provde product thut meets
customer and applicable statutory and regulatory requirements.
It is worth quoting verbatim from ISO/IEC 17021:2006 on the principles on which the credibility of
certcuton depends, und purtcuury thut o openness, vhch vus ound to be uckng n muny
cases during this project. (Note: bold has been added for emphasis.)
0 10 20 30 40 50 60 70 80 90 100
Level 1
Level 2
Level 3
Level 4
Level 5
Overall
confdence
level
Overall performance of certifed organizations as a function of AB
A
c
c
r
e
d
i
t
a
t
i
o
n

b
o
d
y

c
o
d
e
AB7
AB5
AB6
AB2
AB9
AB1
AB3
AB4
AB8
4 Principles
4.1 General
4.1.1 1hese pt|nc|p|es ate |he bas|s |ot |he subsequen| spec||c pet|otmance and desct|p||ve
tequ|temen|s |n |h|s ln|etna||ona| S|andatd. 1h|s ln|etna||ona| S|andatd does no| g|ve spec||c
requirements for all situations that can occur. These principles should be applied as guidance for
|he dec|s|ons |ha| may need |o be made |ot unan||c|pa|ed s||ua||ons. It|nc|p|es ate no|
requirements.
4.1.2 1he oveta|| a|m o| cet|||ca||on |s |o g|ve con|dence |o a|| pat||es |ha| a managemen|
sys|em |u|||s spec||ed tequ|temen|s. 7hc valuc ol ccrlcalon s lhc dcgrcc ol ublc condcncc
and trust |ha| |s es|ab||shed by an |mpat||a| and compe|en| assessmen| by a |h|tdpat|y. Iat||es
|ha| have an |n|etes| |n cet|||ca||on |nc|ude, bu| ate no| ||m||ed |o
(a) |he c||en|s o| |he cet|||ca||on bod|es,
(b) |he cus|omets o| |he otgan|za||ons uhose managemen| sys|ems ate cet|||ed,
(c) governmental authorities,
(d) non-governmental organizations, and
(e) consumers and other members of the public.
4.1.! It|nc|p|es |ot |nsp|t|ng con|dence |nc|ude
- |mpat||a|||y,
- compe|ence,
- tespons|b||||y,
- openness,
- con|den||a|||y, and
- tespons|veness |o comp|a|n|s .
4.5 Openness
4.S.1 A ccrlcalon body nccds lo rovdc ublc acccss lo, or dsclosurc ol, aroralc and
lmcly nlormalon aboul ls audl roccss and ccrlcalon roccss, and aboul lhc ccrlcalon
slalus (.c. lhc granlng, cxlcndng, manlanng, rcncwng, suscndng, rcducng lhc scoc ol,
or wlhdrawng ol ccrlcalon) ol any organzalon, n ordcr lo gan condcncc n lhc nlcgrly
and crcdblly ol ccrlcalon. Ocnncss s a rnclc ol acccss lo, or dsclosurc ol, aroralc
nlormalon."
Cuuse 8.3 o lSC/lLC l702l:2006 speccuy requres certcuton bodes to muntun u drectory o
vud certcutons, us shovn beov.
8.! O|tec|oty o| cet|||ed c||en|s
7hc ccrlcalon body shall manlan and makc ublcly acccssblc, or rovdc uon rcqucsl, by
any means || chooses, a drcclory ol vald ccrlcalons that as a minimum shall show the name,
te|evan| notma||ve documen|, scope and geogtaph|ca| |oca||on (e.g. c||y and coun|ty) |ot each
cet|||ed c||en| (ot |he geogtaph|c |oca||on o| |he headquat|ets and any s||es u||h|n |he scope o|
a mu|||s||e cet|||ca||on).
!o|e: 1he d|tec|oty tema|ns |he so|e ptopet|y o| |he cet|||ca||on body
Availability of information from certifcation bodies
Cveru, out o u totu o l53 certcuton bodes contucted, 57 |37%) ether voud not or coud not
provde normuton ubout ther certed cents |the normuton requested vus no more thun thut
mentoned n lSC/lLC l702l:2006 Cuuse 8.3). A vurety o reusons vere gven by certcuton
bodies for not providing the information requested in a timely manner, and a number of very
disappointing and disturbing incidents occurred:
- Some ocu brunches und runchsees o overseus certcuton bodes ueged thut ths
normuton s strcty condentu und uggressvey decned to provde uny detus o ther
certed cents.
- Some ocu brunches und runchsees o overseus certcuton bodes responded by suyng,
the information is only available in hard copy at our overseas corporate headquarters.
- 1hree nternutonu certcuton bodes ugreed to provde the normuton requested but
subect to u condentuty ugreement. lt s mportunt to note, hovever, thut uccordng to
ISO/IEC 17021 Clause 8.3, this is information that is supposed to be publicly-accessible,
or available on request.
- Cne certcuton body ugreed to shov ther dutubuse o certed cents n the reevunt
countries but would only provide the information requested for a sample of these, which
were selected during a visit to their corporate headquarters.
- Severu certcuton bodes smpy reused to provde the normuton, uegng thut lSC/
IEC 17021 does not require them to provide this information, and their accreditation body
(AB3) was not willing or able to enforce this.
lt s vtu to the credbty o the uccredted certcuton process thut these nconsstences be unuysed
by both the IAF and ISO/CASCO and a common interpretation and application of the requirements
of ISO/IEC 17021:2006 Clause 8.3 be agreed upon and effectively implemented by all.
lt s uso mportunt to recognze thut the mun concerns expressed by certcuton bodes ubout
providing the requested information were related to commercial sensitivities, and that, by making
normuton uvuube, other certcuton bodes mght try to pouch ther cents. Regurdess o the
implications in terms of ISO/IEC 17021 Clause 8.3, this reaction was disappointing and serves to
hghght the uct thut certcuton hus become u very compettve busness.
These concerns were subsequently discussed within the IAF, and a formal decision was made during
the March 2011 IAF Technical Committee meeting that Accreditation Bodies members must ensure
thut ther uccredted Certcuton odes compy vth the requrements o lSC/lLC l702l:2006/20ll
Cuuse 8.3. Accredted Certcuton odes must be ube to demonstrute to ther Accredtuton ody|s)
how they have chosen to make this information accessible or available on request, keeping in mind
the lAl 1echncu Commttee decson 08/03/03 thut provdng ony u meuns to conrm the vudty
o u gven certcuton does not conorm to the requrement o cuuse 8.3.
1hs uck o vngness o some overseus certcuton bodes to provde the normuton requested s
the main reason why the results of this project have to be interpreted with caution, since it was not
possbe to ncude cents o these certcuton bodes vthn the sumpe o certed orgunzutons
thut vere chosen to be vsted. Cveru, u totu o 57 certcuton bodes |ncudng 25 oregn
certcuton bodes uccredted by A3) dd not provde the normuton requested. Snce the totu
number o certcutes ssued by these certcuton bodes n the regon s not documented, the eects
of excluding them from the survey are unknown and unknowable, but undoubtedly will have skewed
the results to some extent towards a more favourable conclusion than a truly random sample. It
may also reinforce the perceptions that were expressed by some participants during the initial
vorkshops thut the operuton o u number o runchsees und brunches o oregn certcuton bodes
is not as transparent as it should be and is inconsistent with the overall principle of openness on
vhch the credbty o certcuton s bused. 1hs s unortunute becuuse othervse the resuts o the
survey huve shovn the outcomes o uccredted certcuton n u very postve ght.
Despte the uck o cooperuton rom these certcuton bodes, uternutve methods to coect
normuton ubout lSC 900l-certed orgunzutons vere expored, ncudng normuton rom
industry associations, chambers of commerce and general marketing materials. This meant that some
orgunzutons certed by non-couborutve certcuton bodes vere st seected or vsts, though
not all agreed to be visited.
At the time of preparation of this report, one proposal that is currently being discussed within ISO and
the lAl envsuges the deveopment o u snge, nternutonu dutubuse o certed orgunzutons, und
the experiences of the current project would certainly endorse the need for such a database. During
the local feedback workshops, it also became apparent that such a database would serve as a strong
deterrent to those ssung ruuduent certcutes snce t voud provde un eusy, trunspurent mechunsm
to very the uuthentcty o uny certcutes presented.
Overall performance of organizations certifed by diferent certifcation bodies
lgures 47 und 48 shov the vurutons n the overu condence eves n the quuty munugement
system or orgunzutons certed by derent certcuton bodes.
lgure 47 shovs the resuts or certcuton bodes vhere the number o certed cents vsted
runged rom 5 to 20, und gure 48 the resuts or certcuton bodes vhere the number vsted vus
over 20. Certcuton bodes vth ess thun 5 cents thut vere ncuded n the survey vere not
considered in this analysis but were grouped together under the category others.
Note: Codes huve been used to preserve the unonymty o the certcuton bodes, but ndvduu
codes cun be provded to certcuton bodes on request.
It can be seen that there are some variations in performance, but the following comments should be
taken into consideration:
- No uttempt hus been mude to provde u runkng o the certcuton bodes, n vev o
the fact that there are many variables that affect the overall performance. This analysis, for
exumpe, ncudes certcuton bodes operutng throughout the regon, us ve us some
certcuton bodes operutng n ony one specc country.
- lt s mportunt to remember thut muny certcuton bodes dd not provde normuton ubout
ther drectory o certed cents n the regon, und so vere not ncuded n ths unuyss.
- The number of organizations visited was not necessarily proportional to the overall
number o certcutes ssued by euch certcuton body n the regon.
- lt s nterestng to note thut the certcuton bodes n the cutegory o others, representng
u totu o 56 certcuton bodes, euch vth ess thun 5 certed cents vsted durng ths
proect |vth u totu o ll3 certed cents vsted betveen them), hud umong the ovest
eves o condence n the perormunce o ther certed cents. 1hs cun be seen n gure 48,
vhere 2l.5% o these orgunzutons exhbted un overu condence eve o l or 2.
figurc 47. Variations in thc ovcraII condcncc IcvcIs in thc QMS for organizations ccrticd
by diffcrcnt ccrtication bodics whcrc thc numbcr of ccrticd cIicnts visitcd was
from S to 20
figurc 48. Variations in thc ovcraII condcncc IcvcIs in thc QMS for organizations ccrticd by
diffcrcnt ccrtication bodics whcrc thc numbcr of ccrticd cIicnts visitcd was ovcr 20
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10 11 12
P
e
r
c
e
n
t
a
g
e

o
f

o
r
g
a
n
i
z
a
t
i
o
n
s

w
i
t
h

c
o
n
f
d
e
n
c
e

l
e
v
e
l

1

t
o

5
Level 5
Level 4
Level 3
Level 2
Level 1
Overall
confdence
level
Overall confdence levels (CBs with 5 to 20 clients visited)
Certifcation body code
13 14 15 16 17 18 19 20 21 Others
P
e
r
c
e
n
t
a
g
e

o
f

o
r
g
a
n
i
z
a
t
i
o
n
s

w
i
t
h

c
o
n
f
d
e
n
c
e

l
e
v
e
l

1

t
o

5
Level 5
Level 4
Level 3
Level 2
Level 1
Overall
confdence
level
Overall confdence levels (CBs with 20 or more clients visited)
Certifcation body code
0
20
40
60
80
100
During the March 2011 meeting of the IAF Technical Committee, comments (based on anecdotal
evidence) were made to suggest that one key area of concern about the credibility of ISO 9001
certcuton reutes to the operuton o runchsees |subcontructed bodes vho perorm uudts on
behu o oregn certcuton bodes). 1hese ure usuuy not consdered to be loregn Crtcu Locutons
|us dened n lAl CD3 24) und ure rurey subect to vsts by the runchsor's uccredtuton body.
y usng the dutu obtuned durng the present proect, t vus possbe to conrm ths tendency. As
shovn n gure 49, t s ceur thut, vhst the perormunce o orgunzutons certed by ocu
certcuton bodes or by brunch oces o oregn certcuton bodes ure smur, the percentuge o
orgunzutons vth ov eves o condence |scores o l or 2) s much hgher or orgunzutons
certed under runchsng urrungements.
The IAF is currently initiating a project to investigate these trends further and to take the appropriate
actions to address these concerns.
figurc 49. Pcrformancc of organizations ccrticd by franchiscd" ccrtication bodics, comparcd
with thosc ccrticd by IocaI ccrtication bodics or by branch ofccs of forcign
ccrtication bodics (bascd on thc visits to S61 ccrticd organizations)
Effectiveness of one-day market surveillance visits
One very important outcome of this project was the excellent correlation demonstrated between the
resuts o the one-duy vsts to certed orgunzutons und the perormunce o these orgunzutons us
perceived by their customers.
The 28 local consultants employed to carry out these visits were unanimous in their conclusion that
u one-duy vst s more thun sucent to be ube to muke u udgment ubout the vudty o un
Local CB
Local branch
of foreign CB
Franchisee
of foreign CB
0 10 20 30 40 50 60 70 80 90 100
Percentage of organizations with level of confdence 1 to 5
Level 1
Level 2
Level 3
Level 4
Level 5
Overall
confdence
level
Overall confdence in organizations broken down by type of CB
orgunzuton's lSC 900l certcuton. 1hs coud orm the buss or u nev murket surveunce
upprouch to montorng the eectveness o uccredted certcuton, n ne vth the lSC/lAl Strutegc
lmperutve thut Cutput mutters!ll.
The following considerations are important when analysing the results:
- All the UNIDO consultants had been subjected to intensive (5-day) training courses,
vhch ncuded vsts to certed orgunzutons or the purpose o cubruton und to
ensure that consistent criteria were applied during the subsequent project visits.
After each of the calibration visits (carried out in three separate groups of
approximately nine participants per group), feedback was provided to the consultants
on their own results compared to those of their peers.
It was found that there was an excellent degree of consistency among the consultants
independent marking of the organizations performance in most of the 26 topics that
were included in the check-list.
Any individual tendencies to be excessively rigid or excessively lenient (compared
to the instructors and peers evaluations) were discussed among the group in order
to ensure homogeneous performance during the subsequent one-day visits.
- \th reerence to the resuts thut ure presented n gures 50 und 5l, the oovng
observations may be made:
lntutvey, the contro sumpe o good suppers thut hud been dented by the
knowledgeable purchasing organizations during the face-to-face interviews can
be seen to demonstrate consistently better scores on a wide range of parameters
during the one-day visits.
Lkevse, the contro sumpe o bud suppers thut hud been dented by the
purchasing organizations had consistently lower scores on a wide range of
parameters during the one-day visits.
Statistical analysis on the visit results was carried out by the Institute of Applied Statistics and
uuty Munugement n Lucknov, lndu, n order to very the stutstcu sgncunce o the
differences observed between the good and bad suppliers. Because the visit results are
based on attribute (rather than variable) data, parametric inference analysis such as Students
t-test was not possible. Non-parametric inference (Moods Median Test) was considered to be
the most upproprute or ths knd o study. Athough the eve o condence n the resuts s not
as high as for parametric inference analysis, a number of the items of the check-list were found
to huve stutstcuy sgncunt derences betveen the good und bud suppers. 1hese tems
ure hghghted n boxes n gures 50 und 5l.
It is important to note, however, that out of the control sample of good and bad suppliers
selected for the one-day visits, the sample size of bad suppliers who agreed to receive a visit
was very small (8 organizations) compared to the sample of good suppliers (33 organizations).
Athough the Mood's Medun test ndcuted stutstcuy sgncunt resuts, the smu sumpe sze
did not facilitate an extremely robust analysis.
1he topcs thut shoved u stutstcuy sgncunt derence shoud thereore be prortsed n
any future applications of this market surveillance methodology. These include the
following items of the check-list:
Item 10 The organization has established and deployed meaningful objectives at relevant
functions and levels.
Item 12 QMS documentation is being used and is properly controlled.
Item 13 The organization has adequate resources (competent personnel, equipment,
etc.) to support its system.
Item 14 The work environment is appropriate.
Item 15 ley product reuzuton processes ure dented und munuged.
Item 17 lroduct nonconormtes ure dented und deut vth uccordng to documented
procedures.
Item 19 Internal audits are being carried out according to plan and are effective.
Item 21 The organization has a focus on preventing nonconformities.
Item 22 Customer feedback and customer complaints-handling mechanisms are
appropriate.
Item 23 1he MS s provdng condence n the orgunzuton's ubty to consstenty
provide product that meets customer and applicable statutory and regulatory
requirements.
Item 25 1he certcuton process hus been conducted eectvey by the certcuton
body.
Item 26 Cveru condence n ths orgunzuton's mpementuton o lSC 900l.
This result has far-reaching implications and strongly supports the validity of using short market-
surveunce type vsts o ths knd to montor the eectve mpementuton o lSC 900l n certed
orgunzutons und the perormunce o uccredted certcuton bodes. Athough the phruse murket
surveunce s trudtonuy ussocuted vth compusory product certcuton, the concept cun
equuy be upped to |vountury) munugement system certcuton by veryng the eectveness o
the uccredted certcuton process through u sumpng o the certcuton body's product |n ths
cuse, the eve o condence n u certed orgunzuton's quuty munugement system).
This in turn could lead to a more results-focused approach to accreditation, rewarding those
certcuton bodes vhose cents perorm ve by reducng the ntensty o uccredtuton surveunce
|oce vsts und vtness uudts), und conversey by ncreusng the ntensty o uccredtuton
surveunce or those certcuton bodes vhose certed cents perorm budy. 1hs voud be totuy
consstent vth und embruce the concept thut Cutput mutters!
figurc S0. Comparison bctwccn good" and bad" suppIicrs for thc S61 onc-day visits
(Iow scorcs)
01. Initial audit duration by the CB
02. Surveillance sudit duration and frequency
03. Scope statement on the organizations certifcate
04. Justifcation for any exclusions (ISO 9001:2008 Clause 1.2)
09. Quality policy
11. Quality manual as a true refection of the QMS
12. Appropriate use of QMS documentation
18. Cause analysis for nonconformities and efective corrective action
19. Internal audits
21. Prevention
23. Overall confdence in the ability to produce consistent, conforming products
24. Continual improvement of QMS efectiveness
25. Overall confdence in the CBs processes
26. Overall confdence in the organizations implementation of ISO 9001
0 10 20 30 40 50 60 70
Percentage of sample with score of 1 or 2
Percentage of good and bad suppliers with
a score of 1 or 2 compared to overall sample
Good suppliers
Bad suppliers
Overall results
figurc S1. Comparison bctwccn good" and bad" suppIicrs for thc S61 onc-day visits
(high scorcs)
01. Initial audit duration by the CB
02. Surveillance sudit duration and frequency
03. Scope statement on the organizations certifcate
04. Justifcation for any exclusions (ISO 9001:2008 Clause 1.2)
09. Quality policy
11. Quality manual as a true refection of the QMS
12. Appropriate use of QMS documentation
18. Cause analysis for nonconformities and efective corrective action
19. Internal audits
21. Prevention
23. Overall confdence in the ability to produce consistent, conforming products
24. Continual improvement of QMS efectiveness
25. Overall confdence in the CBs processes
26. Overall confdence in the organizations implementation of ISO 9001
0 10 20 30 40 50 60 70 80 90
Percentage of sample with score of 4 or 5
Percentage of good and bad suppliers with
a score of 4 or 5 compared to overall sample
Good suppliers
Bad suppliers
Overall results
lt s strongy recommended thut lSC, the lAl und other stukehoders dene hov to mpement such u
market surveillance programme, in order to further enhance the effectiveness and credibility of
uccredted certcuton. lt s recognzed thut ths perormunce-bused upprouch s key to meet vth
sgncunt resstunce rom some o those nvoved, but ths proect hus ceury demonstruted the
vudty o such u methodoogy n dentyng sub-stundurd perormunce o certed orgunzutons
und ther certcuton bodes. luctors thut voud need to be tuken nto consderuton ncude:
- Who would be responsible? (ISO and/or the IAF, or some other independent entity (to be
dened):)
- Hov coud such un ntutve be nunced:
- 1he contructuu mpcutons or uccredtuton bodes und certcuton bodes to ensure
uccess to certed cents these vere seected or such murket surveunce vsts.
- 1he benets thut certcuton bodes mght derve rom such un upprouch ther certed
cents demonstrute good eves o condence durng the murket surveunce vsts.
- The sanctions (increased accreditation surveillance?) that might be imposed for
certcuton bodes vhose certed cents perorm budy.
80
REFERENCES AND BIBLIOGRAPHY
1. ISO 9001:2008 ua|||y managemen| sys|ems Requ|temen|s, lnternutonu Crgunzuton
for Standardization, Geneva, Switzerland.
2. ISO/IEC 17021:2006 Con|otm||y assessmen| Requ|temen|s |ot bod|es ptov|d|ng aud|| and
cet|||ca||on o| managemen| sys|ems, lnternutonu Crgunzuton or Stundurdzuton, Cenevu,
Switzerland.
3. 1he lSC Sutvey o| Cet|||ca||ons 2660, lnternutonu Crgunzuton or Stundurdzuton,
Geneva, Switzerland.
4. UNIDO Proposal SAARC/SEA 20080917 for Project TE/RAS/09/003 Implementation of ISO
0661 ua|||y Managemen| Sys|em |n s|an deve|op|ng coun|t|es: Sutvey covet|ng sys|em
deve|opmen|, cet|||ca||on, accted||a||on and econom|c bene||s, Lnted Nutons lndustru
Development Organization, Vienna, Austria (Internal document).
5. ISO 9004:2009 Manag|ng |ot |he sus|a|ned success o| an otgan|za||on qua|||y managemen|
approach, lnternutonu Crgunzuton or Stundurdzuton, Cenevu, Svtzerund.
6. ISO 10002:2004 ua|||y managemen| Cus|omet sa||s|ac||on Cu|de||nes |ot comp|a|n|s
handling in organizations, lnternutonu Crgunzuton or Stundurdzuton, Cenevu, Svtzerund.
7. ISO 10019:2005 Cu|de||nes |ot |he se|ec||on o| qua|||y managemen| sys|em consu||an|s and
use of their services, lnternutonu Crgunzuton or Stundurdzuton, Cenevu, Svtzerund.
8. IAF MD5:2009 lI Manda|oty Oocumen| |ot duta||on o| MS and IMS ud||s, lnternutonu
Accredtuton lorum, uvuube or dovnoud rom vvv.u.nu.
9. ISO 9000:2005 ua|||y managemen| sys|ems Iundamen|a|s and vocabu|aty, lnternutonu
Organization for Standardization, Geneva, Switzerland.
10. ISO/IEC 17011:2004 Con|otm||y assessmen| Ceneta| tequ|temen|s |ot accted||a||on bod|es
accted|||ng con|otm||y assessmen| bod|es, lnternutonu Crgunzuton or Stundurdzuton,
11. Joint ISO/IAF Communiqu Ixpec|ed ou|comes |ot accted||ed cet|||ca||on |o lSC 0661,
available for download from www.iso.org and www.iaf.nu.
12. Croft N. H. and Dougherty R. Itesetv|ng |he cted|b||||y o| lSC 0661:2666 cet|||ca||on, lSC
Management Systems SeptemberOctober 2007, International Organization for Standardization,
REFERENCES AND BIBLIOGRAPHY 81
13. lSC 0666 Cet|||ca||on Sutvey Iutchasets petcep||ons o| |he|t supp||ets, 2006 Report,
Instituto Nacional de Metrologia, Normalizao e Qualidade (Brazil).
14. 1he lSC Sutvey o| Cet|||ca||ons 2668, lnternutonu Crgunzuton or Stundurdzuton,
15. Croft, N. H. Supp|y cha|ns and lSC 0661 \ha| |o expec|, hou |o ge| ||, lSC locus+, Apr
2010, International Organization for Standardization, Geneva, Switzerland.
16. Informative document lSC 0661 uha| does || mean |n |he supp|y cha|n:, uvuube or
download from www.iso.org.
17. ISO/IAFAuditing Practices Group website www.iso.org/tc176/ISO9001AuditingPracticesGroup.
l8. Luropeun Commsson Commsson Recommenduton 2003/36l/LC concernng the denton
o mcro, smu und medum-szed enterprses, Ccu }ournu o the Luropeun Lnon, 20 Muy
2003.
19. Heras, I., Dick, G. P. M., and Casedess, M. ISO 9000 registrations impact on sales and
pto||ab||||y. |ong||ud|na| ana|ys|s o| pet|otmance be|ote and a||et accted||a||on, lnternutonu
Journal of Quality & Reliability Management, Vol. 19 No. 6, 2002, pp. 774791.
20. Corbett, C. J., Montes, M. J., Kirsch, D. A., Alvarez-Gil, M. J., Ooes lSC 0666 Cet|||ca||on
Iay:, lSC Munugement Systems }uyAugust 2002, lnternutonu Crgunzuton or
Standardization, Geneva, Switzerland.
21. OHSAS 18001:2007 Cccupa||ona| hea||h and sa|e|y managemen| sys|ems Requ|temen|s,
British Standards Institution, London, United Kingdom.
22. ISO/IEC 27001:2005 ln|otma||on |echno|ogy Secut||y |echn|ques ln|otma||on secut||y
managemen| sys|ems Requ|temen|s, lnternutonu Crgunzuton or Stundurdzuton,
23. ISO/TS 16949:2009 ua|||y managemen| sys|ems Iat||cu|at tequ|temen|s |ot |he app||ca||on
of ISO 9001:2008 for automotive production and relevant service part organizations,
International Organization for Standardization, Geneva, Switzerland.
24. IAF GD3:2003 IAF Guidance* on Cross Frontier Accreditation, lnternutonu Accredtuton
lorum, uvuube or dovnoud rom vvv.u.nu.
* Athough the tte reers to Cudunce, ut the tme ths document vus pubshed |2003) t vus mundutory n nuture or u
IAF MLA members.
82
ANNEX: ACKNOWLEDGEMENTS FOR
EACH OF THE PARTICIPATING
COUNTRIES
The active contributions of the following people in each of the participating countries is gratefully
acknowledged.
Country UNlDO ConsuItant Support
Bangladesh Mr A.T. Lutful Kabir
Mr A.M. Mainuddin Khan
Mr A.K. Fazlul Ahad (BSTI)
Mr David Holbourne (BQSP)
Mr Mostafa Anwar (BQSP)
Mr Abu Abdullah (BAB)
Bhutan Mr Ashok Kumar Jain Mr Pelden Dendup (SQCA)
India Mr Ashok Kumar Jain
(Lead Consultant)
Mr A.N.S.P. Shastry
Mr V. J. Brahmaiah
Dr Kavindra Baliyan
Mr P.K. Krishnan
Mr Subbaramaiah Petluri
Mr Shashank Sheth
Dr Ujjwal Manohar Rao
Mr B Venkataraman (NABCB)
Dr Girdhar Gyani (QCI)
Mr P K Gambhir (BIS)
Dr A K Goyal (BIS)
Mr N Banik (BIS)
Lt-Gen. H Lal (QCI)
Mr Philippe Scholtes (UNIDO)
Mr Vinaj Vij (UNIDO)
Ms Ayumi Fujino (UNIDO)
Indonesia Mr Dulbert Tampubolon
Dr Ria Pardede
Ms Widad Baraba
Ms Fenny Widjaya
Dr Bambang Setiadi (BSN)
Ms Arini Widyastuti (BSN)
Dr Sunarya (KAN now retired)
Dr Zakiyah (KAN)
Mr Imran Farooque (UNIDO)
Mr Johannes Verhelst (UNIDO)
Malaysia Mr Harbans Singh
Mr Kwan Loong Ng
Dr Norlinda Mohd. Zawawi
Ms Fadilah Baharin (Standards Malaysia)
Mr Ridzwan Kasim (Standards Malaysia)
Mr Shaharul Sadri Alwi (Standards Malaysia)
Ms Norhuzu Arn |Stundurds Muuysu)
Maldives Dr Subadra Jayasinghe Mr Solih Hussain (MSMU/MED)
ANNEX: ACKNOWLEDGEMENTS FOR EACH OF THE PARTICIPATING COUNTRIES 83
Country UNlDO ConsuItant Support
Nepal Ms Samita Shrestha
Mr Prakash Oli
Dr S.R. Joshi (NBSM)
Mr Indu Bikram Joshi (NBSM)
Mr Poorna Manandhar (UNIDO)
Pakistan Mr Saif Ur Rahman Mr Muhammad Junaid Qureshi (PSQCA)
Mr Najamuddin (PNAC)
Mr Zawdu Felleke (UNIDO)
Philippines Ms Cirila Botor
Ms EmmalineVitug
Mr Victorio Mario Dimagiba (BPS)
Ms Daisy Omila (BPS)
Mr Jaime Lasaro L. Olmos (PAO)
Dr Suresh C. Raj (UNIDO)
Ms Jillian de Paz Revadulla (UNIDO)
Sri lanka Mr Didul Kodagoda
Ms Subadra Jayasinghe
Dr L. N. Senaweera (SLSI)
Mr Wasantha Meewaddana (SLSI)
Mr Gamini Dharmawardena (SLSI)
Mr Sanath Medis, (SLAB)
Mr Sarath Abeysundara (UNIDO)
Thailand Ms Saraporn Jomsang
Mr Narit Lerkmoung
Ms Witchar Pichainarong
Ms Ratanaporn Chuengsanguansit
(TISI now retired)
Mr Chaiyong Krittapholchai (TISI)
Mr Weerasak Phenglong (TISI)
Ms Rachada Isarasenarak (NSC)
Mr Ekanit Romyanon (NSC)
Ms Gloria Adapon (UNIDO)
Viet Nam Ms Minh ThuHa
Mr Le Dinh Ban
Mr Le Quang Vu
Ms Le Thi Viet Hong
Dr Ngo Quy Viet (STAMEQ)
Dr Vu Van Hong (STAMEQ)
Ms Nguyen Thi Thanh Van (STAMEQ)
Mr Vu Xuan Thuy (BOA)
Ms Nilgun Tas (UNIDO)
Printed in Austria
V.11-88335April 20122,600
UNITED NATIONS INDUSTRIAL DEVELOPMENT ORGANIZATION
Vienna International Centre, P.O. Box 300, 1400 Vienna, Austria
Telephone: (+43-1) 26026-3859, Fax: (+43-1) 26926-6840
E-mail: L.Goonatilake@unido.org, Internet: www.unido.org

ISO 9001 Impact Survey-eBook - Ver2

Cargado por

Información del documento

Descripción original:

Título original

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

ISO 9001 Impact Survey-eBook - Ver2

Cargado por

Copyright:

Formatos disponibles

Based on Project TE/RAS/09/003

A survey covering quality management system

También podría gustarte