Statistical Approaches for PPQ

Options and Outcomes

Tara Scherder
Managing Director, Arlenda, Inc

 During the process qualification (PQ) stage of

process validation, the process design is evaluated
to determine if it is capable of reproducible
commercial manufacture
A successful PPQ will confirm the process design
and demonstrate that the commercial
manufacturing process performs as expected
.State a clear conclusion as to whether the data
indicates the process met the conditions
established in the protocol and whether the

process is considered to be in a state of control

Reference: 2011 FDA Guidance Document: Process Validation

Our Session Today

Choice of Statistical Method
Understand Conclusions and Details
(assumptions, requirements)
Combine Statistics and Process Knowledge
Part of Continuum of Process Understanding

Two examples
1. Content Uniformity
2. Packaging Quality Measurements
3

Methods
Exploratory Methods
Variance Components
Monte Carlo Simulation
ASTM E2709 and E2810
Control Charts & Capability
Tolerance Intervals
Bayesian Prediction Interval
ANOVA
ANSI Acceptance Sampling for Variables &
Attributes
Percent Non-Conformance

Example: Content Uniformity

Four samples drawn at each of
15 locations across the batch,
three batches

Graphical methods
Multi-Vari Chart for CU by Batch - Location
CU Sampling Plan 2

CU Sampling Plan 2
each batch has 4 samples at each of 15 locations

each batch has 4 samples at each of 15 locations

110

110

105

100

CU

CU

105

Batch
A
B
C

100

95

95
90
1

8
9
Location

10

11

12

13

14

15

7
8
9
Location

10

11

12

13

14

15

Variance Components

Batch A
Parameter

Between-location
SD
Within-location
SD
Total

Batch B

Point

% of total

Point

Estimate

(%LC)

Estimate

2.57
1.25
2.86

80.9
19.1
100

1.00
3.39
3.54

% of
total
(%LC)

8.1

Batch C
Point

% of total

Estimate

(%LC)

Parameter

3.39

90.2

Overall Mean

91.9Between-batch
1.12 9.8
SD
100 Between3.57 100
location[Batch]
SD
Within-location
SD

Total

Point
Estimate

% Total

100.1

1.25

12.2

2.52

50.2

2.18

37.6

3.56

1007

Monte Carlo Simulation

Probability of passing a method/procedure
based on user provided inputs
Provides a way of evaluating a
method/procedure or process using
computer generated data in place of
collecting actual data

Monte Carlo Simulation

Percentage of Batches Passing UDU Test
Batch A
Parameter

Between-location
SD
Within-location
SD

Total

Batch

Batch B

Batch C

Point

% of total

Point

% of total

Estimate

(%LC)

Estimate

(%LC)

2.57

80.9

1.00

8.1

1.25

19.1

3.39

91.9

2.86

100

Point Estimate

3.54

Point

% of total

Parameter
Estimate
(%LC)
Overall Mean
3.39
90.2

% Total

100.1

Between-batch SD

1.25

12.2

Betweenlocation[Batch]
3.57
100 SD

2.52

50.2

Within-location SD

2.18

37.6

Total

3.56

100

1.12

100

Point Estimate

9.8

Confidence
Limit

S1

S1 & S2

S1

S1 & S2

100

100

99.5

99.8

99.7

100

87.4

98.4

100

100

98.6

99.1

Point Estimate

S1
100.
0

S1 and S2
100.0
9

ASTM E2709/E2810: General Strategy

1. Determine Sampling Plan
Sample Size (# Locations & # per location)
Use Prior Knowledge
OC curves, examination of acceptance limit tables

2. Construct/Choose Acceptance Limit Table

Select Confidence Level (Usually 90 or 95%)
Select Coverage (usually 95%): Desired Probability of
Samples passing Testing Standard (eg USP UDU)

3. Collect Data & Compute Summary Statistics

4. Compare to appropriate Acceptance Limit Table

Acceptance Limit Table Example

for USP UDU Test
Result

Location

97.08

99.72

98.37

97.50

99.72

100.32

101.01

100.29

99.90

98.27

98.88

97.96

98.78

98.17

98.94

97.78

96.32

96.61

99.66

97.20

100.97

102.17

99.06

98.80

97.02

97.35

98.65

99.98

99.39

98.81

98.63

98.06

99.59

97.80

97.67

98.95

10

97.97

98.54

100.26

98.74

11

96.09

98.61

97.49

97.50

12

98.87

97.81

97.28

98.80

13

101.10

102.60

100.48

98.62

14

100.80

100.34

98.49

100.93

15

99.70

100.09

100.14

99.20

Acceptance Limit Table Example

for USP UDU Test
Descriptive Statistics
Overall Mean
SE (within-location Std Dev)
Standard deviation of Location
Means
90%CI/95%Cov

98.93
1.07
1.06

Standard Deviation of Location Means

0.9

1.0

SE

LL

UL

0.9

88.1

1.0

1.1

1.2

LL

UL

LL

UL

LL

UL

111.9

88.5

111.5

88.9

111.1

89.3

110.7

88.2

111.8

88.6

111.4

89.0

111.0

89.4

110.6

1.1

88.4

111.6

88.7

111.3

89.1

110.9

89.5

110.5

1.2

88.5

111.5

88.9

111.1

89.2 110.8

89.6 110.4

1.3

88.7

111.3

89.0

111.0

89.4 110.6

89.7 110.3

Predicting Probability of Passing

ASTM 2810
Across Batches
Overall Mean(%LC)
Variance Components (Std Dev)
Between Batch (%LC)
Between Location (%LC)
Within Location (%LC)
Total (%LC)

100.1
1.25*(p=0.02)
2.52*(p<0.0001)
2.18
3.56

Simulations
P(Batches Passing ASTM E2810) Using Point
Estimates
90%CI/95% Cov
N=10
32.7%
N=20
73.7%
N=30
91.8%
N=60
99.4%
95%CI/95% Cov
N=10
17.9%
N=20
55.8%
N=30
82.0%
N=60
98.5%
13

Control Charts: Two Subgroup Charts

Xbar-R Chart of CU by Batch
B

110

Sample M ean

105
1

1
8

1 5

100

5
1

95

_
U
_ C L=101.59
X=99.91
LC L=98.22

Subgroup Mean

110

I-MR-R/S (Between/Within) Chart of CU by Batch

13

17

21
25
Sample

29

33

41

45

16

13

10

21

25

29

33

37

41

45

C
UCL=9.28

__
MR=2.84
0

LCL=0
1

13

17

12

17

21

25

29

33

37

41

45

16

U C L=5.29
_
R=2.32

LC L=0
1

13

17

21
25
Sample

29

33

37

41

45

Sample Range

Sample Range

_
X=99.91

100

37

UCL=107.46

LCL=92.35

MR of Subgroup Mean

C
1

90

UCL=5.29
_
R=2.32
LCL=0

0
1

13

17

21

25

29

33

37

41

45

Sample

14

Tolerance Interval*
Assures with a pre-specified confidence level, that there is
at least a pre-specified probability (called coverage) that
the individual results will fall within the interval endpoints
k depends on distribution, coverage and confidence
PTS-TI
95% confidence that the percentage of tablets outside the range of
(85%, 115%) label claim (LC) is less than 12.5%

PTI-TOST
95% confidence and 87.5% coverage of the 85% to 115% LC limiting
the percentages of tablets below 85% and above 115% LC are both
less than 6.25% of the batch

*computationally complex for multi location sampling plan

15

Bayesian Prediction Interval

Bayesian solution provides true prediction of failure of
future batches
Incorporates uncertainty in parameter estimation
Can incorporate between batch, between location, and
within location variance components
Fit model
Perform simulations to obtain posterior distribution of
parameters
Obtain predictive distribution of CU
Assess
probability that .025 and .975 percentiles will be outside specification

Statistical statement : X% probability that 95% of

batches will pass UDU, or fall within some assay
range
16

ANOVA
Fixed locations or batches: detect
significant variation in group means
Random locations: detect significant
variation among groups

17

ANOVA
Batch and Location
Treated as Fixed Effects

Batch and Location Treated

as Random Effects

Parameter

Point
Estimate

Overall Mean
Between-batch
SD
Betweenlocation[Batch]
SD
Within-location
SD

100.1

Total

% Total

1.25

12.2

2.52

50.2

2.18

37.6

3.56

18
100

Example: Package Fill Volume

Packaging run time = 170 minutes

Fill rate = 180 bottles/minute
Total bottles filled = 30,600
Filler has 6 nozzle heads
Specifications: LSL = 99.5; USL = 100.5 ml
AQL (acceptance quality level) = 0.1%
Shift in mean after 26,000 bottles from 100.2100.35 and Std Dev from 0.08 to 0.09 ml
19

ANSI Z1.9
For Lot size = 30,600; AQL = 0.1%; Tightened, Level II
Inspection, Variability Unknown, sample size of 100 is
required.
One simulation , estimate of % non-conforming (ncf)
was 0.052 (table B-5 of standard); maximum allowable
% ncf is 0.218

Decision: ACCEPT the BATCH

Actual % ncf was 0.14%, which is higher than the AQL
of 0.1%.
Based on the OC curve, if 100 samples are drawn from a
lot of this size with 0.14% ncf, the lot will be accepted
75 % of the time. That is, a bottle with a defect will
randomly be found in the sample only 25 % of the time
20

Control Charts & Capability

21

Example: Critical Defect

ANSI Z1.4 Attribute Acceptance
50,000 bottles; AQL=0.065%; ANSI Z1.4 Tightened,
Inspection level II (no specification for Rejectable
Quality Level, or Lot Tolerance Percent Defective)
Using Tables from standard, the required sample size is
1250, and the lot will be accepted if 1 or less
nonconforming bottles are found.
Beta error is high; for instance, there is a 19% chance of
accepting a lot that has 0.246 % nonconforming.
Consumer risk not controlled.
No statistical statement can be made regarding quality
of lot
22

Example: Major Defect

ANSI Z1.4 Attribute Acceptance
50,000 bottles; AQL=1%; ANSI Z1.4 Tightened, Inspection
level II
Using Tables from standard, the required sample size is 500;
lot will be accepted if 8 or less nonconforming bottles are
found
Can identify other sampling plans using software that might be
more efficient.
Operating Characteristic (OC) Curve
n c
275 4
500 8

Probability of Acceptance

1.0
n
c

0.8

sample size
acceptance number

0.6

0.4

0.2

0.0
0

2
3
Lot Percent Defective

23

Example: Critical Defect

Proportion Non-conforming
For any pass/fail sampling results, a statistical
statement regarding the quality of the lot can be
made. For instance, a confidence statement can be
made regarding the bounds for the percent non
conforming
Example: assume no failures are found in a sample of
1250. This allows the following statistical statement:

With 95.0% confidence, the population

nonconformance rate will be no more
than 0.0050 (~0.5%) (1)
(1) Agresti/Coull interval from Lawrence D. Brown, T. Tony Cai and Anirban DasGupta, Interval
Estimation for a Binomial Proportion, Statistical Science, 2001, Vol. 16, No. 2, 101133

24

Contributors

James Bergum
Richard Montes
Helen Strickland
Jennifer Walsh

25

26