SAE TECHNICAL PAPER SERIES

2004-01-1631

Correlation Grading Methodology for Occupant Protection System Models

Deren Ma, Jennifer Matlack, Honglu Zhang and John Sparkman
Delphi Corporation

Reprinted From: CAE Methods for Vehicle Crashworthiness and Occupant Safety, and Safety-Critical Systems (SP-1870)

2004 SAE World Congress Detroit, Michigan March 8-11, 2004

400 Commonwealth Drive, Warrendale, PA 15096-0001 U.S.A. Tel: (724) 776-4841 Fax: (724) 776-5760 Web: www.sae.org

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of SAE. For permission and licensing requests contact: SAE Permissions 400 Commonwealth Drive Warrendale, PA 15096-0001-USA Email: permissions@sae.org Fax: 724-772-4891 Tel: 724-772-4028

For multiple print copies contact: SAE Customer Service Tel: 877-606-7323 (inside USA and Canada) Tel: 724-776-4970 (outside USA) Fax: 724-776-1615 Email: CustomerService@sae.org ISBN 0-7680-1319-4 Copyright 2004 SAE International Positions and opinions advanced in this paper are those of the author(s) and not necessarily those of SAE. The author is solely responsible for the content of the paper. A process is available by which discussions will be printed with the paper if it is published in SAE Transactions. Persons wishing to submit papers to be considered for presentation or publication by SAE should send the manuscript or a 300 word abstract of a proposed manuscript to: Secretary, Engineering Meetings Board, SAE. Printed in USA

2004-01-1631

Correlation Grading Methodology for Occupant Protection System Models

Deren Ma, Jennifer Matlack, Honglu Zhang and John Sparkman
Delphi Corporation
Copyright 2004 SAE International

ABSTRACT
Computer modeling and simulation have become one of the primary methods for development and design of automobile occupant protection systems (OPS). To ensure the accuracy and reliability of a math-based OPS design, the correlation quality assessment of mathematical models is essential for program success. In a typical industrial approach, correlation quality is assessed by comparing chart characteristics and scored based on an engineers modeling experience and judgment. However, due to the complexity of the OPS models and their responses, a systematic approach is needed for accuracy and consistency. In this paper, a correlation grading methodology for the OPS models is presented. The grading system evaluates a wide spectrum of a computer models performances, including kinematics, dynamic responses, and dummy injury measurements. Statistical analysis is utilized to compare the time histories of the tested and simulated dynamic responses. The statistical quantity measurements include the average residual, standard deviation, correlation coefficient, and 0th to 2nd moment relative differences. The correlation quality of overall kinematics and dynamic responses is scored and colorcoded from weak, marginal, adequate, good to excellent. The grading system can clearly distinguish the correlation quality for different models. The evaluation of a side curtain airbag component model and a frontal impact system model is presented as examples to demonstrate the applications of the correlation grading system.

primary means for design and optimization of the advanced OPS. To ensure the accuracy and reliability of a math-based OPS design, the correlation quality assessment of the mathematical models is essential for program success. In a typical industrial approach, correlation quality is assessed by comparing chart characteristics and scored based on an engineers modeling experience and judgment. This approach is often inaccurate, subjective and inconsistent. Due to the complexity of the OPS models and their responses, it is hard to compare the characteristics of several charts simply by reading the charts directly. An engineers experience rather than scientific analysis plays a vital role in such a model quality assessment process. Also the same engineer may apply different criteria on different models, and different engineers may have diverse scores on the quality of the same model. The inconsistency makes it hard to implement organization-wide common development process and mathematical model re-use strategy. To address these issues, many efforts have been made to provide more quantifiable criteria to judge the correlation quality of mathematical models. National Crash Analysis Center sponsored by Federal Highway Administration (FHWA) and National Highway Traffic Safety Administration (NHTSA) developed a time domain validation procedure to compare simulation results and test data for full vehicle crash simulation [3]. M. H. Ray proposed a set of criteria to validate full-scale crash simulation results using this procedure [4]. TNO Automotive and MECALOG also introduced a commercial software package ADVISER which contains a model quality-rating module with its own measurement criteria [5,6]. All these systems are trying to answer the following two questions: (1) Which measurement variables should be used to judge the model correlation quality; (2) What is the criterion for each measurement variable. Because of the technical limitations of the OPS simulation models, such as inadequate understanding of the occupant biomechanical responses in an OPS system, it is unlikely that the simulation results are very similar to the test data from a purely statistical point of view.

INTRODUCTION
The design of Occupant Protection Systems (OPS) plays an important role in vehicle development. Extensive government regulations for both frontal and side impacts [1] and consumer information programs such as New Car Assessment Program (NCAP) [2] make the design and optimization of OPS a challenging task. Pure physical tests are cost prohibitive because of the large number of crash scenarios in which the OPS has to be tested to evaluate its performance and ensure compliance of government requirements. Therefore, computer modeling and simulation have become a

Therefore, the selection of measurement variables and their criteria is vital in the application value of the correlation evaluation systems. Based on engineering experience from both physical tests and simulation models, a correlation grading system specifically tailored for OPS model evaluation was developed. The system evaluates a wide spectrum of a computer models performance, including kinematics, dynamic responses and dummy injury measurements. The correlation quality of overall kinematics and dynamic responses is scored and colorcoded from weak, marginal, adequate, good to excellent. The evaluation of a side curtain airbag component model and a frontal impact system model is presented as examples to demonstrate the applications of the correlation grading system. The grading system clearly distinguishes the correlation quality for different models with accuracy and consistency, and helps to assess confidence level of the predictive results. Additionally, the correlation grading system provides an easy-tounderstand platform for the communication of OPS modeling results among various divisions of the organization, among different organizations or between CAE analysts and hardware design engineers who do not necessarily have much background in computer simulations.

The model output variables to be evaluated are selected based on the nature of the model. In a component model of side curtain airbag free motion headform (FMH) to pole impact, for example, the head acceleration is the major variable to be assessed, while in a frontal impact system model, many more variables need to be considered, depending on the crash conditions, design regulations and customer requirements. The variable selection may also be influenced by design objectives. OVERALL KINEMATICS The kinematics is assessed by comparing the model kinematical responses with physical test videos. The kinematics variables of a typical OPS model include, but are not limited to, vehicle interior intrusions and movements, restraint system performances and dummy kinematical responses. DYNAMIC RESPONSES Dynamic responses are evaluated based on the complete time history curves of each variable from both the simulation results and the test data. Among the full range of statistical measurements of two curves, several representative measurements are selected in the correlation grading system to draw a comprehensive picture of the relationship between the curves with minimum number of statistical measurement variables. When the dynamic responses of a computer simulation model are considered close enough to those of a physical test, the following criteria need to be satisfied: 1. The overall shape and trend of the two time history curves are similar. 2. The peak values and their timing of the response curves and the corresponding occupant injury measurements are close. 3. The total areas of the curves are alike.

THE CORRELATION GRADING SYSTEM

Table 1 shows the overall structure of the correlation grading system. It consists of two major parts, the overall kinematics evaluation and the dynamic response assessment. A single overall kinematics score is granted according to multiple criteria of kinematics similarity between the model animation and the physical test videos. At the same time, several variables are graded individually for the dynamic responses. The average score of all these variables decides the general dynamic response score of the model.

Table 1 Overall Structure of the Correlation Grading System

Peak Value Grading system Overall Kinematics Timing Magnitude Statistical analysis Standard Dynamic 2nd 1st Deviation Correlation 0 Moment Avg Moment Moment Response Coefficient Diff of Residual Diff Diff Residual x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Excellent Good Adequate Marginal Weak

Excellent Good Grade Criteria Adequate Marginal Weak

x x x x x

x x x x x

To quantify the above three criteria, two key parts are included in the dynamic response evaluation section, peak value comparison and statistical analysis. For peak values, both the magnitude and the time when the peak value is reached are assessed. The statistics analysis is utilized to measure the overall shape and trend of the simulated model response time histories, as well as the total area of the curves. Average residual, standard deviation, correlation coefficient and 0th to 2nd moment relative differences are the major factors considered in the correlation grading system [7-9].
Average Residual Average Residual is the average difference between the model outputs and the reference signal, usually the test results. It is calculated as Equation 1 and is shown as a percentage of the peak value of the reference signal. Because average residual is a mathematical mean of the difference of all data points along the curves, the positive and negative differences at various data points may cancel each other. As a result, it is not unusual to have a low average residual and a high standard deviation when comparing curves with oscillatory residuals.

r=

(s
i =1

s )( f i f )

(s
N i =1

) (f
2 N i =1

(3)

where r is the correlation coefficient of two time histories,

s and f are the mathematical means of the two time history curves. The correlation coefficient can range from -1.00 to +1.00. The value of -1.00 represents a perfect negative correlation while a value of +1.00 represents a perfect positive correlation and a value of 0.00 represents a lack of correlation. A perfect correlation indicates that the model output can be linearly transformed to match the reference signal. For the purpose of OPS model correlation, this means that the shapes of the curves are very similar even if the magnitudes are offset. It is possible and common to have a very good correlation coefficient and still have significant scaling and bias errors.
0 2nd Relative Difference of Moments The relative moments describe the characteristics of the
n

R=

(s
i =1

fi )
(1)

time history curves. The nth moment M of discrete time histories si and f i can be expressed as:

where R is the average residual, si and f i are the two time histories being compared and N is the data sample size. Standard Deviation of Residual The standard deviation of residual is defined as the square root of the variance of the residual. It is calculated in Equation 2:

M n ( si ) = tin si t
i =1 N

(4)

M n ( f i ) = tin f i t
i =1

(5)

where ti is time duration at sample data point i and t is the constant time step. Moments for n=0, 1 and 2 correspond to the area under the curve, area moment of the curve, and area moment of inertia of the curve, respectively. The relative nth moments of discrete time histories si and f i can be expressed as:

s N 1 =

(R
N i =1

R)

N 1

(2)

where s N 1 is the standard deviation of the residual, Ri is the residual of the two time histories at sample data point i and R is defined in Equation 1. The standard deviation of residual is also represented as a percentage of the peak value of the reference signal. It describes how spread out the residual is around its mathematical mean. Correlation Coefficient The correlation coefficient is a measurement of the degree of linear relationship between two time histories. It is given by the following expression:

M rn ( si ) =

t
i =1

n i

si t
n +1

N ti i =1

(6)

M ( fi ) =
n r

t
i =1

n i

f i t
n +1

N t i i=1

(7)

The relative difference of the moments is calculated in Equation 8:

= M rn (si ) M rn ( f i )

(8)

si and f i are considered very similar if the relative difference of moments of si and f i is smaller than a prespecified value. The statistical analysis of average residual, standard deviation, correlation coefficient and 0th to 2nd moment relative differences, together with the peak value magnitude and timing, depicts an overall picture of the similarity between the simulation output and the reference data. The grading criteria of the measurement variables are another important component of the correlation grading system. The criteria vary from case to case. For example, for a relatively simple component model, the criteria are more stringent, while for a complex system model, the criteria are looser. Even among system models, the criteria for frontal systems and side systems are different. A whole set of criteria for each scenario were developed in the correlation grading system based on the engineering experience with the reliability and predictability of each category of models. Once decided, the grading criteria are applied constantly to all the models in the same category to ensure the consistency of the methodology.

The main objective of the model is to predict the headform acceleration under various airbag pressures at different impact locations. Several impact locations and airbag pressures were tested physically. The simulation model was intended to match the headform accelerations in all non-bottom-out cases. Here bottom out means the minimum cushion thickness between the two layers of the airbag fabric becomes zero. Once the airbag bottoms out, the interaction between the pole and the headform plays the major role, not the interaction between the airbag and the headform. In this model, the main criterion for kinematics correlation is whether the airbag bottomed out. The chief variable to consider in dynamic responses is the headform acceleration time history. To evaluate the overall correlation quality of the model, all non-bottomout cases were assessed simultaneously with the correlation grading system. Figures 2-6 illustrate the comparison of headform acceleration curves in all the five non-bottom-out cases (non-bottom-out in the physical tests). Table 2 contains the actual values of the dynamic response measurements, and Table 3 is the output of the correlation grading system, with the set of grading criteria for component level FMH impact simulations applied. In Table 3, 5 is the highest grade, while 1 is the lowest grade based on the evaluation criteria.

CASE 1
Test Simulation

Head Acceleration (g)

EXAMPLE ONE: CORRELATION QUALITY ASSESSMENT OF SIDE CURTAIN AIRBAG COMPONENT LEVEL MODEL
The correlation grading system is applicable for simulation models with various complexity levels. To demonstrate the usage of the system, the correlation quality assessment of a relatively simple component level model of side curtain airbag FMH to pole impact is presented as the first example. Figure 1 is a snapshot of the model.

120 100 80 60 40 20 0 0 0.02 0.04 0.06 0.08 0.1 Time (s)

Figure 2 Head Acceleration Curves for Case 1

Figure 1 Side Curtain Airbag FMH Pole Impact Model

CASE 2
Test
Simulation

CASE 5
Test
Simulation

100
Head Acceleration (g)

80
Head Acceleration (g)

80 60 40 20 0 0 0.02 0.04 0.06 0.08 0.1 Time (s)

60 40 20 0 0 0.02 0.04 0.06 0.08 0.1 Time (s)

Figure 3 Head Acceleration Curves for Case 2

Figure 6 Head Acceleration Curves for Case 5 Since this is a relatively simple component level model, an experienced engineer can tell the correlation quality by directly comparing the curves. Therefore, this model was also used as a benchmark for the correlation grading system. In Case 1, the headform acceleration curve measured from the physical test is fairly smooth. There is no spike on the test curve. This indicates that the airbag did not bottom out in the physical test. However, in the simulation, the head acceleration curve shows a small spike, as an indication of slight bottom out. Thus the overall kinematics was graded as Marginal. As far as peak value is concerned, the magnitude of the peak value of the model output differs obviously from the test data because of the spike. So the grade for peak value magnitude is 2 according to the grading criteria. Except for the spike, the timing of the peak values of the two curves is very close, with a grade of 5. Statistical analysis wise, the spike results in a large standard deviation of residue and a moderate 2nd moment difference. The insufficiency of the model correlation is confirmed by a few criteria simultaneously. The overall grade of dynamic response is Good from the grading system. The correlation of Cases 2, 3 and 4 is much more satisfactory than that of Case 1, resulting in a grade of Excellent in each of the cases. In Case 5, even though the kinematics of the model and the physical test are close enough, the total areas of the two curves differ apparently. The overall dynamic response score is just Adequate from the grading system. Among all 5 cases, Case 2, 3 and 4 have the best correlation, the correlation of Case 1 is the worst. The correlation grading system clearly distinguishes the correlation quality of each case. With all 5 cases considered, the overall correlation quality for this side curtain airbag FMH pole impact model is Good.

CASE 3
Test Simulation

60

Head Acceleration (g)

50 40 30 20 10 0 0 0.02 0.04 0.06 0.08 0.1 Time (s)

Figure 4 Head Acceleration Curves for Case 3

CASE 4
Test Simulation

60

Head Acceleration (g)

45 30 15 0 0 0.02 0.04 0.06 0.08 0.1 Time (s)

Figure 5 Head Acceleration Curves for Case 4

Table 2 Correlation Evaluation for Side Curtain Airbag FMH to Pole Impact Model
Peak Value Grading system Overall Kinematics Timing Magnitude Statistical analysis Standard Dynamic 2nd 1st 0 Deviation Correlation Avg Moment Moment Moment Response Coefficient of Residual Diff Diff Diff Residual 0.88% 0.21% 0.96% 0.51% 7.9% 8.03% 3.84% 3.67% 4.62% 4.2% 0.994 0.998 0.999 0.998 0.996 0.017 0.004 0.016 0.009 0.159 0.013 0.005 0.015 0.009 0.145 0.056 0.024 0.005 0.005 0.155

Case 1 Case 2 Cases Case 3 Case 4 Case 5

Marginal 2 ms Excellent 1 ms Excellent 1 ms Excellent 1 ms Excellent 1 ms

27.5% 4.5% 3.7% 4.5% 4.9%

Table 3 Correlation Grades for Side Curtain Airbag FMH Pole Impact Model
Peak Value Grading system Overall Kinematics Timing Magnitude Statistical analysis Standard Dynamic 2nd 1st 0 Deviation Correlation Avg Response Moment Moment Moment Coefficient of Residual Diff Diff Diff Residual x x x x x 5 5 5 5 3 5 x x x x x 3 5 5 5 5 5 x x x x x 5 5 5 5 5 5 x x x x x 5 5 5 5 2 4 x x x x x 5 5 5 5 3 5 x x x x x 4 5 5 5 2 4 Excellent Good Adequate Marginal Weak Good Excellent Excellent Excellent Good Good

Excellent Grade Criteria Good Adequate Marginal Weak Case 1 Cases Case 2 Case 3 Case 4 Case 5 Overall Correlation Marginal Excellent Excellent Excellent Excellent Good

x x x x x 5 5 5 5 5 5

x x x x x 2 5 5 5 5 4

EXAMPLE TWO: CORRELATION QUALITY ASSESSMENT OF FRONTAL IMPACT SYSTEM MODEL

The most important applications of the correlation grading system are for the complex system models, with complicated kinematics and multiple time history outputs. Figure 7 shows a typical frontal impact restraint system. The left picture is the simulation model, and the right one is the test video.

Figure 7 A Typical Frontal Impact System

To assess the correlation level of kinematics, vehicle and interior intrusions and movements, restraint system kinematics and dummy kinematical responses are considered. A few examples of vehicle interior intrusions and movements are seat pitching or dropping, steering wheel bending or column collapsing and instrument panel or knee bolster intrusion. The restraint system kinematics consist of the airbag deployment shape and timing and the seat belt geometry and spool out characteristics. The dummy kinematical response correlation concentrates on the pelvis, knee, chest and head motion. Contact timing is also a very important factor in the correlation, especially for the contact between the knees and the knee bolster, between the airbag and the windshield, and between the head/chest and the airbag. The next step is to evaluate the dynamic responses of the model. The time histories of all the variables of interest were analyzed. Figures 8, 9 and 10 represent the occupants head, chest and pelvis resultant accelerations and accelerations in X direction. Both the timing and magnitude of the peak value of each variable were compared with test data and scored. The statistical analysis of the average residual, standard deviation, correlation coefficient and 0th to 2nd moment relative differences completes the picture of the correlation. The scores of these statistic analyses were averaged. The combination of the scores of peak values and the statistical analysis decides the overall dynamic response score. Table 4 shows the actual values of the dynamic response evaluation variables, and Table 5 lists the outputs of the correlation grading system for this model. Notice that due to the complexity of the system model, the grading criteria in Table 5 are different from those in Table 3. The final grade for this frontal impact system model is Excellent, indicating a high quality of model correlation.

Resultant Sled Data Resultant Model Data Head Acceleration (G) 100 50 0 -50 -100 0 0.02 0.04

X Sled Data X Model Data

0.06

0.08

0.1

Time (s)

Figure 8 Time History of the Head Acceleration

Resultant Sled Data Resultant Model Data Chest Acceleration (G) 100 60 20 -20 -60 -100 0 0.02 0.04 0.06 0.08 0.1 Time (s) X Sled Data X Model Data

Figure 9 Time History of the Chest Acceleration

Resultant Sled Data Resultant Model Data X Sled Data X Model Data

Pelvis Acceleration (G)

100 60 20 -20 -60 -100 0 0.02 0.04 0.06 0.08 0.1 Time (s)

CONCLUSION
The correlation grading system has been successfully used to quantify the correlation quality of OPS models and assess confidence level of the predictive results with accuracy and consistency. It also helps to communicate OPS modeling results in an easy-to-understand platform. The system has been proved reliable and efficient for complex system models as well as relatively simple component models.

Figure 10 Time History of the Pelvis Acceleration

Table 4 Correlation Evaluation for Frontal Impact System Model

Peak Value Grading system Overall Kinematics Timing Magnitude Statistical analysis Standard Dynamic 2nd 1st Deviation Correlation 0 Moment Avg Moment Moment Response Coefficient Diff of Residual Diff Diff Residual 5.04 % 4.74 % 1.17 % 0.91 % 3.16 % 2.31 % 10.19 % 10.33 % 17.13 % 17.62 % 14.67 % 18.31 % 0.981 1 0.927 0.92 1 1 0.142 0.154 0.036 0.029 0.109 0.086 0.135 0.134 0.102 0.105 0.163 0.096 0.121 0.113 0.171 0.184 0.21 0.077

Head R Acc Variables Chest R Acc Chest X Acc Pelvis R Acc Pelvis X Acc Excellent Head X Acc

1 ms 2 ms 8 ms 8 ms 5 ms 5 ms

3.9% 0.1% 0.9% 0.9% 3.6% 12%

Table 5 Correlation Grades for Frontal Impact System Model

Peak Value Grading system Overall Kinematics Timing Magnitude Excellent Good Grade Criteria Adequate Marginal Weak Head R Acc Excellent Variables Head X Acc Chest R Acc Chest X Acc Pelvis R Acc Pelvis X Acc x x x x x 5 5 3 3 4 4 x x x x x 5 5 5 5 5 3 Statistical analysis Standard Dynamic 2nd 1st Deviation Correlation 0 Moment Avg Moment Moment Response Coefficient Diff of Residual Diff Diff Residual x x x x x 4 4 5 5 5 5 x x x x x 5 5 4 4 5 4 x x x x x 5 5 4 4 5 5 x x x x x 5 5 5 5 5 5 x x x x x 5 5 5 5 4 5 x x x x x 5 5 5 3 5 Excellent 5 Excellent Good Adequate Marginal Weak

REFERENCES
1. U.S. Department of Transportation, National Highway Traffic Safety Administration, Federal Motor Vehicle Safety Standards and Other Regulations. (http://www.nhtsa.dot.gov/cars/rules/standards/) 2. U.S. Department of Transportation, National Highway Traffic Safety Administration, New Car Assessment Program, 1978. (http://www.nhtsa.dot.gov/cars/testing/ncap) 3. FHWA/NHTSA National Crash Analysis Center, FHWA Quantitative Validation Procedure, 1997. (http://www.ncac.gwu.edu/archives/software/) 4. M. H. Ray, "Repeatability of Full-Scale Crash Tests and a Criteria for Validating Finite Element Simulations", Transportation Research Record 5. 6. 7. 8.

9.

No.1528, Transportation Research Board, Washington, D.C., December, 1997. TNO Automotive, Research and Technology Development News ADVISER, May, 2003. TNO Automotive, Presentation Script, ADVISER, May, 2003. J.L. Devore, Probability and Statistics for Engineering and the Sciences, Brooks/Cole Publishing Company, 1982. S. Basu and A. Haghighi, Numerical Analysis of Roadside Design (NARD): Validation Manual, Volume III, Report FHWA-RD-88-214, Washington D. C., Federal Highway Administration, September, 1988. The Technology Administration of the U.S. Department of Commerce, NIST/SEMATECH, eHandbook of Statistical Method. (http://www.itl.nist.gov/div898/handbook/index.htm)