T i i Plan Training Pl

Title: Acceptance Sampling by Attributes of System/ Software Development

Acceptance Sampling Inspection by Attributes

Learning Objectives: Upon completion, the trainee will be able to: Exhibit a personal familiarity with the concepts and practices of Acceptance Sampling (Objective: Knowledge) State one or more advantages and disadvantages of Acceptance Sampling, including discussion of Deming Demings s criticism of Sampling (Objective: Skill/ Attitude) Demonstrate ability to perform Acceptance Sampling process, including use of Acceptance Sampling Worksheet as well as the Z1-4 look-up tool (Objective: Skill)

Acceptance Sampling Inspection by Attributes

Materials Required: The following training materials are required: Training g classroom w/ projector p j Access to Z1-4 sampling materials Sampling bead bowl, red and white beads (separated) w/ scoop Copies of Sampling Worksheet handouts Folder #1 ~ Records sampling exercise

Acceptance Sampling Inspection by Attributes

Introduce self/ subject Create general interest Sampling is an allowable practice under CMMIsm Specifically p y authorized by y many y of todays y SQPPs Provides a means of getting work done Create personal interest Sampling is an enabling SQE skill Provides useable/welcome management information

Acceptance Sampling by Attributes of f System/ S t / Software S ft Development D l t

W. L. Bill Trest

Agenda

What is attribute sampling? Why not use 100% inspection? Attribute sampling vs statistical process control Type I and type II errors Operating characteristic (OC) curve Population vs sample Simple random sample Stratified random sample Systematic sampling

Examples be made topresentation, software process artifacts. Note: Thiswill is not an SPC nor will any Math above elementary school level be required, No math beyond elementary school but examples of software process will be cited.

Agenda (Continued)
Single vs double sampling Multiple p and sequential q sampling p g Sampling inspection by attributes ANSI/ASQC Z1.4 Inspection levels Switching rules Demings condemnation of sampling Identification of key features and objectives (KFOs) Application/exercise

What is Attribute Sampling?

Attribute sampling provides quality management information to determine whether to accept or reject a lot of material that has already been produced, based upon less than 100 percent inspection.

Attribute sampling provides characteristic information about the population as a whole, based upon an examination i i of f a truly l representative i sample l (<100%) ( 100%) drawn from the material of interest (lot), after product is sufficiently y complete. p

Why Not Use 100% Inspection?

Typically, there are six situations where 100% inspection is often not practical When inspection/testing is destructive When costs are prohibitively high When time or technology gy are constraints When population sizes are large and the probability of inspection errors are high When history or application justifies 100% inspection When potential liability risks are sufficient to warrant continuous monitoring

Advantages of Sampling

The number of required auditors is relatively small Less disruption of normal business Results can be obtained in a short time Processes can be evaluated by looking at a small number of areas Opportunities for improvement can be quickly identified

Disadvantages of Sampling
Sampling cannot guarantee the identification of all deficiencies Significant deficiencies may not be in sample Deficiencies taken may be exceptions Corrective actions may y be misdirected Little/ no organization-wide info in sample

Samples taken from discrete process artifacts say very little about anything other than the process or product p oduct from o which c t the e sample sa p e is s ta taken e

Acceptance Sampling vs SPC

Statistical process control (SPC) is preferred over attribute sampling when near near real time time monitoring of a process artifact or products key characteristic (KC) is a requirement

Type I and Type II Errors

Because lot disposition is based on sample inspection results there is a probability of making an incorrect disposition Incorrectly rejecting a lot that is really acceptable is a Type I error Incorrectly accepting a lot that is really unacceptable is a Type II error

When using a statistically valid sampling plan, any probabilities of Type I and II errors are accounted for via use of automated tools, based on sampling tables from ANSI/ASQ standards

Operating Characteristic Curve (OC Curve)

An OC curve is a graphic Showing the probability of accepting lots (Pac) of Type I and II errors Showing the quality level of the lot/process when using sampling plans The randomly sample number of units to be taken from lot population as a whole (n) is used to determine lot disposition

OC Curves (continued)
Operating Characteristic Curve
120% Pa = Probability of Acceptance 100% 80% 60% 40% 20% 0%
0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 10% 11% 12% 13% 14% 15% 16% 17% 18% 19% 20%

Lot Fraction Nonconforming (p)

The shape and look-up values at points along an OC Curve indicates the ability and power to discriminate between acceptable and unacceptable lots

Population vs. Sample

A sample is considered representative of the population as a whole in acceptance sampling A sample is a subset of f the population or lot We determine the disposition of the lot based upon examination of the sample

Population sample

Population vs. Sample (Continued)

Overview of Z1.4 Tool

Simple Random Sample

For sampling plans to be statistically valid, each sample must be randomly selected from the population

Stratified Random Sample

When the population consists of a mixture of more than one element with a common linking characteristic, a stratified random sampling can assure the sample is representative of the population

Systemic Sampling

Systemic sampling is the selection of samples from a population according to a set schedule of plan For example every third item may be systematically selected. Statistical validity is enhance when the actual every every n item is chosen at random in accordance with a set plan.

Single vs. Double Sampling

In single sampling the decision to accept or reject the lot is made based on the results of examining one and only one single sample When there is not a clear accept/reject decision from inspection of the first sample, a second sample is taken and the decision is made based upon the results of examination of combined samples

Multiple & Sequential Sampling

Multiple and sequential sampling procedures are the same as for double sampling, except in multiple sampling, the number of samples to make a decision may exceed two

Sampling Inspection by Attributes ANSI/ASQC Z1.4

Individual items in a random sample are inspected ~ using bi binomial i l criteria it i applied li d to t entire ti sample l ~ and d judged j d d as acceptable or not acceptable Information needed (From Z-1.4 tool): Lot size Inspection Level (Level II is default) AQL ( Level 4.0 is default per p DCMA) ) Type of Sampling (Single is default)

Inspection Levels
Inspection level depends upon lot size and degree of discrimination needed There are three general levels: I, II and III. Unless otherwise specified, Level II shall be used. Inspection Level I is used when less discrimination is needed. Level III is used for greater discrimination. There are several special levels: S-1, S-2, S-3 and S-4. Typically, S-3 and S-4 may be used when small sample sizes are necessary and large risks are tolerated

ANSI Z1.4 (Batch Size 2000) Example Change of Inspection Level ~ AQL not changed
Example ~ On Next slide

Inspection Level I
Single/Double AQL Normal/Tightened/ Reduced Sample Size Accept/Reject Single 4.0% Normal 50 5-6

II
Single 4.0% Normal 125 10-11

III
Single 4.0% Normal 200 14-15

S-1
Single 4.0% Normal 3 0-1

S-2
Single 4.0% Normal 13 1-2

S-3
Single 4.0% Normal 13 1-2

S-4
Single 4.0% Normal 32 3-4

Inspection levels ~ established via agreement, govern sample sizes more than AQL. AQL

Sampling Plan Worksheet

Sampling Plan Worksheet
Date of Worksheet: _____________ 12 Never 2007 S/SQ Evaluation Database Activity Record Number: ___________________ FXX-F-XXX-E

Lot Size (Units)

Sample Size

AQL

Accept (Ac) (Plan/A ) ctual)

Not Accept (Re) (Plan/Actual)

Nominal First Error Rate Percent1 (Actual Re divided y Sample p Size by x 100)

Worst Case Error Rate Percent2 (OC Curve ( Review)

Actual Number of Errors

2000

125

4.0%

10/2

11/2

1.6 %

~ 2.0 to 3.0 %

1. Nominal 1st Case error rate is the best (lowest) estimate of percent of errors remaining in the Lot Size (Units) examined. 2. Worst case error rate is the worst (highest) error rate that the OC curve shows remaining in the Lot Size (Units) examined.

ANSI Z1.4 (Batch Size 2000) Example Change of AQL ~ Holding Inspection Level
Inspection Level II
Single/Double AQL Normal/Tightened/ Reduced Sample Size Accept/Reject Single 1.0% Normal 125 3-4

II
Single 1.5% Normal 125 5-6

II
Single 2.5% Normal 125 7-8

II
Single 4.0% Normal 125 10-11

II
Single 6.5% Normal 125 14-15

II
Single 10% Normal 125 21-22

II
Single 15.0% Normal 125 21-22

Inspection levels ~ established via agreement govern sample sizes more than AQL AQL, but Accept/ Reject limits varies proportionately as AQL increases

Switching Rules for ANSI Z1.4

Normal to Reduced
Prior 10 Lots Accepted, with Total Nonconforming less than Limit (Re) Number Number, and Production steady, and Management Approved

START

Normal to Tightened
2 out of 5 or fewer consecutive lots examined are not acceptable

REDUCED

NORMAL

TIGHTENED Discontinue

Reduced to Normal
Lot Not Accepted while on reduced inspection, p , or Lot Appears Acceptable, but Nonconformities found lie between Plan Ac and Re, or Production is Irregular, or Other Conditions Warrant

Tightened to Normal
5 consecutive lots Acceptable

Discontinue inspection when 5 lots not acceptable while on tightened inspection Corrective Action Required

Switching not required when further information is not needed/necessary

Switching Rules (Back-Up)

Use normal inspection and continue unchanged except as designated below. Normal to Tightened -- When 2 out of 5 (or fewer) consecutive lots are not acceptable Tightened to Normal -- When 5 consecutive lots are considered acceptable. Normal to Reduced -- When all of the following conditions are satisfied: 1. The preceding 10 lots are acceptable 2 2. Total findings from preceding 10 lots is equal to or less than the AQL for normal single inspection. (If double or multiple sampling is in use, all samples inspected should be included, not "first" samples only) 3 3. P d ti Production is i at t a steady t d rate t 4. Reduced inspection is considered desirable by the responsible authority Note: Reduced inspection returns to normal when defects are the same or more than the AQL for normal single

Demings Condemnation
Deming criticized acceptance sampling plans as techniques that guarantee some customers will get defective products Acceptance sampling is appropriate when used for a product or process artifact whose state of control is unknown or known to be out-of-control

Acceptance A t sampling li is i not t as significantly i ifi tl valuable l bl when process variation is within control limits

Identification of key features and objectives

Each process produces artifacts/ evidence of work accomplished Ap process without evidence probably p y didnt happen pp A process without evidence prescribed by approved planning probably wasnt performed well (or planning is suspect) Some process evidence is more indicative of process health health as a key feature of acceptable performance, than less important evidence indicating process objectives are not satisfied

To take advantage of sampling principles, samples are inspected for conformity to only a few; e.g., no more than five, salient, key process or product features of a binomial nature, that are both agreed-to by management as well as documented/ retained as an inspection aid and record

Application/ Exercise
Optional Activity #1 Are process records complete?

Sampling Plan Worksheet

Date of Worksheet: _____________ S/SQ Evaluation Database Activity Record Number: ___________________

Lot Size (Units)

Sample Size

AQL

Accept (Ac) (Plan/Actual)

Not Accept (Re) (Plan/Actual)

Nominal First Error Rate Percent1 (Actual Re divided by Sample Size)

Worst Case Error Rate Percent2 (OC Curve Review)

Actual Number of Errors

1. Nominal 1st Case error rate is the best (lowest) estimate of percent of errors remaining in the Lot Size (Units) examined. 2. Worst case error rate is the worst (highest) error rate that the OC curve shows remaining in the Lot Size (Units) examined.

Lot Identification/ Type/ Description/ Other Information (Optional)