A Case for Health Indicators vs.

Condition Indicators in Mechanical Diagnostics Eric Bechhoefer1, Al Duke2, Eric Mayhew3 Goodrich Fuel and Utility Systems Vergennes, VT 05491
1

Eric.Bechhoefer@Goodrich.com
2

Al.Duke@Goodrich.com

Eric.Mayhew@Goodrich.com
(CI) that tell us something about the condition of the machine. The CI have a lot of variance (i.e. they are noisy). Since comprehensive seeded fault testing is not practical, MD evaluates the normality of the component being assessed. When one or more CI has a statistically significant deviation from the norm, we can begin to make a diagnosis of a problem. The art of MD is to set thresholds that alert users to impending problems at the right time. Some HUMS systems are focused on improving drive train safety. They aim to issue an alert only if continued safe flight is at risk. This late stage alerting is easier because when a component is close to failure, the CI are dramatically different than normal. The Goodrich IVHMS MD capability is designed to provide early detection of impending problems far in advance of airworthiness concerns. We believe that for operators to embrace HUMS it needs to provide operational cost savings. Early detection enables anticipatory maintenance. Replacement parts can be ordered, shafts can be realigned, and so on when it is convenient for the maintainer. This improves aircraft availability (ref.3 ). Early detection, however, brings with it the challenge of discerning when elevated CI are just due to noise or are truly a significant difference and worthy of operator attention. A given part may have several failure modes and each mode may drive different CI. What the operator needs to know is: Is this part becoming unserviceable? How long can I use this part before I need to take it out of service? Without early detection, prognostics is impossible.

Abstract1 Health and Usage Monitoring Systems (HUMS) have been proven to be indispensable in improving aircraft safety, readiness and reduced maintenance cost. While there is some consensus in the helicopter community as to what constitutes HUMS functionality, there is still a question as to what mechanical diagnostics (MD) is and how MD information is displayed to the user. MD has at least two paradigms for the evaluation of a monitored components condition. The first paradigm will be identified as Condition Indicator (CI) based thresholding, and the second paradigm will be called Health Indicator (HI) based thresholding. In most HUMS systems, a vibration signature is processed into a set of Condition Indicators (CI). For many components, the CI have physical meaning. As an example, shaft order 1 (SO1) is the magnitude of the shaft vibration in inches per second of the first harmonics. Other CIs have limited physical meaning, such at gear distributed fault. In the first paradigm, these CI can be viewed and trended on a ground station. In some cases, the original equipment manufacturer will supply limits for the CI. As an example, SO1 limits may be given for the engine input drive shaft. In most cases, however, there are no established limits and a statistical procedure is used (e.g. a threshold is set such that the probability of the component being normal, when exceeding the threshold, is small) to trigger maintenance In the second paradigm, multiple CI for a given component are fused into one HI. The paper outlines the advantages and disadvantages of these methodologies and makes the case for the HI. Introduction MD is fundamentally a statistical process. We measure and analyze dynamic signals from machinery to extract features Presented at the American Helicopter Society 63th Annual Forum, Virginia Beach, VA, May 1-3, 2007. Copyright @ 2007 by the American Helicopter Society International, Inc. All rights reserved.
1

Maintainer/Operator Considerations A HUMS system is only as good is the information that it provides to the Maintainer or Operator. Given this, the systems utility is improved by providing a simple, intuitive interface. A natural way to view of the drive train components is to highlight those components graphically that require attention. For example, components that are healthy are nominally green. Component with more damage or wear are yellow, while components that have significant wear and are in need of maintenance are highlighted at red (Figure 1).

Components such as bearings have a number of failure modes relating to their component parts (ball, cage, inner, and outer race). Gears are perhaps the most complex of components with a number of failure modes. Test stand results indicate that certain CI are better at early fault detection, while others are better at late stage fault detection. The point is, no single CI captures all modes of failure for a given component. Further, while some CI have an intuitive meaning (such as SO1), other are more difficult to interpret (example FM4). This again suggests that mapping to a common scale for health improves the interpretation and acceptance of the information presented.

Figure 1 HI Display To implement a simple and intuitive graphical interface such as this in effect is fusing the CI into an HI. One must also consider that the maintainer has to exercise judgment in the interpretation of the information they are viewing. Since each component will have a unique CI range of values for good, warning and alarm, it is impractical to assume that anyone can look at any given CI value and discern its meaning. For example, a given components shaft order one (SO1) value 1 inch per second (ips) might be perfectly acceptable, but for a different shaft might be a sign of impending failure. This leads to the concept of normalized HI values, or more precisely, mapping a components CI to a HI whose values are common across all components. For example, the mapping could take a range from zero to 1, where 0 is new, 0.7 infers a state of warning (suggesting, perhaps time to schedule maintenance) and 0.9 implies a state of alarm (maintenance should be performed). This paradigm of rolling up a number of component CIs and using them to represent graphically the state of the component empowers the maintainer. It provides a simple, direct measure of component health which is easily interpreted. This facilitates the acceptance and use of mechanical diagnostic information to new users and gives confidence to more experienced users. CI Short Comings Of coarse, there would be no need for multiple CIs if there was a Silver Bullet CI. Such a CI would be sensitive to all modes of failure and damage. In our experience, no such CI exists. As an example, consider a shaft component. A shaft can be out of balance (SO1 gets large), it can be bent (typically SO1 and SO2 gets large), or there is mechanical looseness or cracking in the coupling (resulting in SO1, SO2 and SO3 increasing). Even with relatively simple failure modes, the shaft requires three CI to capture its health state.

Thresholds The helicopter manufacturer, in some cases, may have established limits for CI (example, engine input shaft, SO1), but in most cases there are no limits for most CI. The lack of limits means that thresholds are set statistically, such that the probability of a normal component being identified as damaged is small (i.e. PFA: probability of false alarm). Of course, the threshold on a CI can be set such that the PFA is effectively zero. This strategy results in reducing the sensitivity of detecting a fault (that is, the probability of detection, PD). The goal is to set the threshold high enough to reduce PFA, while maintaining good PD. Consider the system in which the threshold for a CI is set at 3 standard deviations. For a normally distributed CI, the PFA would be 0.0013, or a little more than 1 in a thousand trials would be an indication of fault when in fact the component is healthy. This is a natural, statistical process that cannot be avoided. However, one would say, 1 in a 1000 trials is not a problem, as a thousand trials is a large number.. Consider, though, a drive train on a helicopter that might have 25 shafts, each shaft having three CI (25 * 3 = 75 CIs to evaluate). The drive train has 27 gears (assume 6 CI for each gear: 27 * 6 = 162 CIs), and 65 bearings (assume 5 CI for each bearing: 65 * 5 = 325 CIs). This gives a total of 562 CIs, each with a PFA of 0.0013. In a single full set of acquisitions, the probability of one false alarm per acquisition is then: 1-(1-0.0013) 562 = 0.53 This indicates that with each full set of acquisitions there is a 53% probability of a false alarm! For the present, assume that in one hour we have n independent acquisitions which represent a series of Bernoulli trials, with a PFA as given above (0.53). This approximately obeys the Poisson probability law with parameter = n*PFA.

p(y ) = e

y!

Eq (1)

Assuming that we can process 6 acquisitions per hour, the probability for y number of false alarms can be calculated: y 0 1 2 3 4 5 6 p(y) 0.04 .13 .21 .22 .18 .11 .06 The expected number of false alarm per hour is simply = n*p = 3.19. This indicates that a naive approach to threshold setting will result in more false alarms than expected.. Now consider a similar system in which the CIs are rolled up by some function into an HI, and that HI has a PFA of 0.0013. Using similar logic, we see that the probability of one false alarm (only 117 HIs) is: 1-(1-0.0013) 117 = 0.14. The expected number of false alarms per hour is then 0.87, or about a quarter that expected when using CIs. Because CIs tend to be correlated (i.e. not completely independent) and non-Gaussian, the number of false alarms observed would likely be higher. As it will be shown, HIs allow the system engineer to control false alarms.

variance an a priori PFA can be defined. This gives assurance that when a component is identified by HUMS as failing or damaged, the component in fact is in need of maintenance. This concept of setting thresholds to account for betweenaircraft and within-aircraft variance sets an absolute and global threshold. This is in contrast to systems that might set threshold dynamically or be dependent on changes in absolute CI levels. While ultimately, setting thresholds dynamically might result in more sensitivity (i.e. ability to detect a change in a component), it has been our experience that in fact the systems are too sensitive and that such a system has to potential to drive unwarranted maintenance actions. Further, dynamic thresholds have a potential to not alert on components that are assembled incorrectly or have significant damage when the system is installed. For many HUMS applications, the system will be retrofitted onto an aircraft that may have questionable service history. Such aircraft could have components that are damaged or in need of maintenance. A system with an absolute or global threshold will identify these components as bad. A system that uses dynamic threshold setting would, by definition, require a period of time to define the threshold. During this period of time, the aircraft could be at risk because a component is damaged.

Why are CIs So Noisy? Each time an acquisition of component is made, a number of CIs are calculated for a component. The CIs are complex functions of a number of variables: Torque, RPM, material stiffness, mass, etc. These variables influence the vibration response resulting in a stochastic process. What we observe is variance in the measured CI. Typically we observe within-aircraft variance (example: SO1 values changing from one acquisition to another) which reflect the random, sample to sample variance. Additionally, we measure between-aircraft variance (e.g. the mean value and variance between two aircraft is different). For shafts, this variance is due to difference in shaft eccentricity due to balance, looseness/cracking in couplings, and trueness of (see ref 7). For gears, and bearings, wear and damage similarly result in differences between aircraft.

Statistics Aspect of Using Multiple CI In setting thresholds globally, a large enough sample of data needs to be analyzed in order to ascertain the between aircraft and within aircraft variance. It is implied that when setting these thresholds that there exists a certain level of confidence that the statistics generated are true. If one assumes Gaussian distributions, the required sample size needs to be large enough so that:

Z =
1

has an approximately standard normal distribution, where are the sample mean, sample and sample standard deviation respectively. When designing thresholds for release on a fleet of aircraft, initially, a small number of aircraft will be equipped with HUMS to gather data. The confidence of the distribution of CI becomes better with a larger number of samples. We assume that the aircraft (AC1, AC2, .., ACn) represent a random sample of n selected from the fleet of aircraft. If we assume that the CIs from a normal distribution with
1 2 3 4 5 6 A/C Number 7 8 9 10

0.8

, , and

0.6 IPS 0.4 0.2 0

Figure 2 Example of Noisy CI Data The key to a successful HUMS system is to account for the variance within and between aircraft. By accounting for this

unknown mean and variance, and we let C and represent the CI sample mean and standard deviation, then,

T = C C

Is a t distribution with n-1 degrees of freedom.

We can use this knowledge in determining an appropriate sample size for the number of aircraft required for training. For example, if we can compare the 95 percentile confidence level of a t distribution vs. a standard normal and note that the threshold values converge as n approaches 21 (Figure 3). Alternatively, we can look at the rate of change of a threshold value for the t-distribution vs. n and get, in essence, a marginal change in confidence when adding an aircraft as a sample.
6 Threshold 5 4 3 2 1 Marginal Change in Threshold 5 10 15 20 25 T-Distribution Gaussian

the critical value (threshold for a given PFA), mean and variance can be calculated. These values give information on the behavior of the algorithm at various conditions, and give a priori information for the design of tracking or smoothing filters. In the Goodrich IVHMS, the HI is typically designed such that the PFA is 10-6. This threshold has been arbitrarily set at .7 (warning) on a scale of 0 to 1, with 0 being new. A maintenance practice is set such at the HI of .7 suggest that maintenance should be planned (e.g. order parts) and .9 (alarm) suggest that maintenance should be performed (ref 8). There are a number of functions that the HI could assume. If the maximum of n number of identical distribution CI were taken (ref. 2), this function would be an order statistic. Alternatively, if through service history the distribution of nominal and damage components are known, a Bayesian classifier could be implemented (ref. 5). However, due to the robust design of the aircraft, the service history has provided few component failures. Most of the service history provides CIs from nominal (i.e. serviceable) components. This suggests identifying a HI function that is sensitive to deviations away from the distribution of normal components. One such function is based on the Nakagami distribution (ref 4). As an example, we will derive an HI function based on a Nakagami distribution. The Nakagami distribution is similar to the Rayleigh, Rice and Chi-Square distributions. The Chi-Square distribution of n degrees of freedom is a transformation of the Gaussian distributed random variable where

0 -2 -4 5 10 15 20 25

Figure 3 Effect of Sample Number on Thresholds This analysis indicates that for CI that are approximately Gaussian, the same mean estimation approaches the true mean with a sample size of 21, which is the ideal number of aircraft we would like to set thresholds. The marginal rates of change in the thresholds suggest that the change in threshold (confidence) does not greatly change when the sample size is greater than 5. This means that the practical, minimum number of aircraft required is 6. This suggests that there should be two opportunities to build configuration for the aircraft: an initial configuration using the data on 6 aircraft, and a final fleet configuration based on 21 aircraft. HI Based Thresholding In HI based thresholding, the CIs for a given component are fused into one HI. Formally, the HI is a function of distributions (ref 7), as the CI are treated as a random variable from some distribution. Ideally, the HI should use the information available from the CI to be sensitive to all failure modes. Additionally, there should be some normalization and scaling of the HI so that all HIs have common meaning. In practice, the HI is designed such that the PFA of the HI exceeding a given threshold and generating an alert is small and constant across all components. It is important to consider the HI as a function of distribution because it allows the use of statistical tools to quantify HI performance. Through statistical techniques (ref 6), it is possible to derive the HI probability density function (PDF) and cumulative distribution function (CDF). From the CDF,

If the distribution of X is zero mean, R is a central ChiSquare. The Rayleigh distribution is the square root of the central Chi-Square distribution for n = 2 degrees of freedom. In the case where X is not zero mean, Eq. 1 is a non-central Chi-Square distribution. The square root of Y is a Rice distribution. This is the distribution seen with damaged shafts (i.e. centrality of SO1, SO2 and SO3 > 0) or damaged bearings. The Nakagami is a generalized case of the square root of the central Chi-Square distribution. The Nakagami is most commonly used to characterize the statistics of signals transmitted through multipath fading channels (ref 9). The function measures a normalized distance:

R=

This is ideal for an HI in which n number of CIs are used to cover all potential failure modes of the component, and where explicit knowledge of CI sensitivity to a failure mode is not completely known. The PDF for this distribution is given by Nakagami (ref 7) as:

f (R) = 2 ()( ) R 2 1e R
2

R2 =

n i=1

X i2
2

(1)

n i=1

X i2

(2)

(3)

where is defined as

= E [R 2 ]

R 2 = 1 (2 2)
(4)

n i=1

2 2 X1i + X 2i 2 i

and m is defined as the ratio of moments:

= 2 E (R 2 ) , 1 2
2

By setting = 1 Eq 3 reduces to a Rayleigh PDF. Here it must be highlighted that Eq. 4 has great significance in the development of the HI. Observe that is the expected value of R2, where R2 is a Chi-Square distribution of n degrees of freedom. HI Algorithm for Rayleigh Based CIs It can be shown that CIs based on magnitude such as SO1, SO2 and SO3 have Rayleigh distribution for the nominal component (e.g. no out of balance, no misalignment, looseness or bent shaft). Similarly, it can be shown that bearing CIs such as bearing envelope energies are also Rayleigh (ref 4). In sampling theory, a representative population is sampled and statistics are gathered that describes some underlying phenomena. In the case of a nominal shaft or bearing which are undergoing forced vibration, the statistic that is estimated is , which is the underlying standard deviation of the Rayleigh distribution. Ideally, one would have available some large number of shafts from which SO1, SO2 and SO3 would be calculated. From this, the sample SO1, SO2 and SO3 standard deviation could be calculated and estimated by: (10) = 2 2 With the estimated , one could use the Rayleigh CDF and set a threshold for any PFA. During MD acquisitions, the measured CI could be compared against the threshold and a recommendation made. Again, from a system perspective, this is not ideal because it gives more opportunities to generate a false alarm. Alternatively, the CIs could be summed together, in which case the distance is compared to a threshold. In order to weight each CI equally, the CIs are normalized by their standard deviations (e.g. multiplication by the inverse covariance, which is the information matrix). Consider the following, where CI is a vector of CI representing the measured SO1, SO2 and SO3, and is the sample covariance from a set of nominal shafts: R 2 = CIT 1CI (6) Note that equation (6) is equivalent to equation (1). If the CIs were zero mean Gaussian, the expected value of R2 is the number of degrees of freedom. By normalizing by the covariance, one has:

= 1 (2 2) X
2n i=1

(9)

X N(0,1)

(5)

where X is a Normal Gaussian zero mean with standard deviation of one. A well know statistical property is: E a f (r) = a E f (r) (10) It can them be shown from Eq 9 and Eq 10, that the expected value of the normalized sum of n Rayleigh distribution is 1/(2-/2)*2*n.

This result gives an absolute measure of normality for any magnitude based HI. With this, is found from Eq (4) to be 1/(2-/2)*2*n and is found from Eq (5) to be n. These parameters, coupled with the inverse Nakagami CDF will give the threshold value for any PFA. This also allows calculation of the HI mean value and variance.

An HI Algorithm It is a simple matter to now scale the distance function Eq (2) by the threshold derived from the inverse Nakagami CDF (defined here as v) (11) In the case of the Rayleigh distributions with three CIs used for shaft: = 13.979 , = 3, v = 9.44, and mean = 3.58, variance = 1.113 In this example, the mean nominal shaft would have a HI of 0.26, and the standard deviation would be (0.7/9.44) x 1.1130.5 = 0.078.

HI = CI T 1CI * 0.7 v

Example: Generator Shaft Goodrichs IVHMS has been installed on over 50 UH-60 aircraft. Currently over 180,000 data acquisitions have been gathered in a period of two years. IVHMS monitors twentyfive shafts, in addition to gears, and bearing components. IVHMS has identified a number of shafts that have been in the process of degradation (generally a rare event.) As a component type, the generator shafts have provided the richest set of training data as it has the highest observed wear rate. The generator shaft is geared through the main transmission accessory module. The shaft has a phenolic coupler that is a sacrificial connection between the generator shaft and the accessory gearbox. When the shaft adapter fails, the generator shaft no longer transmits torque to the generator resulting in a power failure. The crew is then required to land at the next opportunity causing a mission to abort. Figure 4 shows SO1, SO2 and SO3 inches per second (IPS) for the general population of aircraft and for AC545. AC545 is observed in the process of shaft coupling failure (red marks).

By substitution (see Notes on Rayleigh Distribution) this is expanded to:

R2 =

n i=1

CIi2 (2 2) i2

(8)

components currently in the fleet are serviceable and the CIs are generally representative of nominal components. It was stated previously that damaged shaft and bearing components generate CIs with a Rice distribution (e.g. no assumption of zero mean Gaussian). One can see the effect changing the eccentricity of a Rice distribution when is constant (Figure 6).

Figure 4 SO1, SO2 and SO3 for AC Population and AC545 AC545 initially is in a state not significantly different from the other aircraft. At some point spline cracking results in looseness, followed by fretting. This results in an increase in SO1, SO2 and SO3. The most statistically significant change was observed in SO2. In figure 5, the PDF of the population of aircraft shaft HIs is plotted vs. AC545s shaft HI. The line plot (right side) plots the HI vs. acquisition. Observe that AC545 HI started at 0.25 and at approximately acquisition 75 progressed to an HI of 1. The shaft coupling, on removal, was found to be heavily fretted, cracked, and with evidence of heating.

Figure 6 Effect of Eccentricity on Mean and Standard Deviation of a Rice PDF Here was set to 1/(2-/2), which corresponds to a standard deviation of 1. The eccentricity varied from 0 (Rayleigh) to 10. Note that the mean value of the Rice asymptotically approaches the eccentricity, while the standard deviation asymptotically approaches 1/(2-/2). This is again a significant result. Because the sample is not dependent on the mean, it is relatively insensitive to changes in eccentricity or model violations due to damage. This model is dependent almost exclusively on the assumption that the mass and stiffness of the component is similar between aircraft (i.e. is constant). Due to the close tolerance of aviation manufacturing, this is a good assumption. Even in the presence of slightly damaged components, the sample covariance is representative of nominal components and the resulting HI algorithm will be optimally suited to detect anomalous components. Other Considerations We believe that the HI concept can additionally reduce risk by exposing unknown or anomalous failure modes in a component. While a number of CI have been developed, it is difficult to anticipate the response of a given CI to failure mode that has not been seen due to limited service history. This can be somewhat ameliorated in an HI by designing the HI as a function of a number of CIs. Instead of having the maintainer look at a large number of disparate CI, the maintainer has one HI that encompasses the information of n CI. It must be pointed out that the HI concept does not replace CIs, but augments HUMS by providing higher quality

Figure 5 HI of AC Population vs. AC545

Effect of Model Violation on Rayleigh Assumption Through the development of this HI function, it was assumed that the sample covariance would be sampled from nominal components. This may not be the case in that the HUMs equipped aircraft are retrofitted on aircraft with an unknown service history. Essentially, it is assumed that

information. CIs will always be important to the expert user when a component is identified as failing. Under normal conditions, the HI captures all of the relevant information that a maintainer would need. Conclusions and Summary A Health Indicator paradigm provides a powerful tool to improve the system performance of HUMS. System performance means: Simple way to display and roll up component data Ease of use by maintainers: Consistent metrics across all components Reduced false alarm rate Reduced risk to anomalous and unknown failure modes Again, an HI system does not remove the ability to view and perform analysis on the raw CI. Because the CI are always available for expert analysis, the risk is future reduced if and when a component failure occurs. We believe the HI is an important tool in the understanding and effective practice of mechanical diagnostics.

[9] Proakis, John, Digital Communications, McGraw Hill, Boston, 1995.

References [1]U.S. Department of Transportation, Federal Avation Administration, Advisory Circular AC-29 MG 15: Airworthiness Approval of Rotorcraft Health Usage Monitoring Systems (HUMS) pp 986 1002 [2]Ross, S., Introduction to Probability Models Academic Press, Boston 1989, pp 135-189 [3] Wright, Johnny, Emerging Results Using IMD-HUMS in a Black Hawk Assault Battalion AHS Forum 61 Proceedings, 2005 [4]Bechhoefer, E., Bernhard, A., Use of Non-Gaussian Distribution for Analysis of Shaft Components, IEEE Aerospace, Big Sky, March 2006. [5]Wackerly, D., Mendenhall, W., Scheaffer, R., Mathematical Statistics with Applications, Duxbury press, Belmont, 1996. [6] Nakagami, M., The m-Distribution A General Formula of Intensity Distribution of Rapid Rading, Statistical Methods of Radio Wave Propagation, W. C. Hoffman (ed). Pp 3-36, Pergamon Press, NY, 1960 [7]Bechhoefer, E., Mayhew, E., Mechanical Diagnostics System Engineering in IMD-HUMS, IEEE Aerospace, Big Sky, March 2006. [8] Bechhoefer, E., Method and apparatus for determining the health of a component using condition indicators U.S. Patent 6,728,658