DEVELOPMENT OF NUMERICAL EVALUATION SYSTEM FOR OPERATORS

AT LNG RECEIVING TERMINAL

Suguru Kitagaki, Hisato Inoue, Masanori Oki

Osaka Gas Co.,Ltd :

Keywords: operator, training, simulator, evaluation

1. Abstract

Osaka Gas Co., Ltd. Senboku LNG terminals, “multi-energy distributors” are productive terminals that not
only manufacture city gas from received LNG but also supply natural gas to power plants in the terminals and
make use of the cryogenic properties of LNG.
In addition, we have succeeded in training our operators to conduct integrated operation of these businesses
regarding LNG terminals. Therefore, Senboku LNG terminals can be steadily and effectively operated by a
small group.
It is essential, of course, for steady and effective operation to have sufficient operators in terms of both
quality and quantity. So, we have given serious consideration to criteria of skills and effective training methods
for rapid and definite encouragement of operators.
We, at Senboku LNG terminals, have developed a highly-functional simulator that has a numerical
evaluation system for training programs that can numerically evaluate abilities of operators.
In addition, we have actually proven the effectiveness of these developments through actual training of our
operators. The following text provides further details.

2. Background

2.1 Outline of Osaka Gas Co.,Ltd

Japan introduced LNG back in 1969 and Osaka Gas Co.,Ltd introduced it in 1972. Now LNG use has
spread all over the world and we have great confidence in our 37 years’ experience of handling LNG. Osaka
Gas Co.,Ltd, located in the Kansai region, the western part of Japan, is mainly in the business of
manufacturing, transporting and selling city gas produced from LNG to end users. The number of customers
is approximately 6.88 million. The amount of city gas sold by Osaka Gas Co.,Ltd is approximately 8.8 billion
cubic meters (unit: 45 MJ/m3). Figure.1 shows our LNG receiving terminals and our trunk lines and our
terminals, consisting of Senboku terminal 1, Senboku terminal 2 and Himeji terminal.

1
 Osaka Gas’ Service Area

Kyoto

HIMEJI Kobe
Osaka

TerminalⅠ
SENBOKU
TerminalⅡ

customers : 6.88million
pipeline extension: 57,931km

Figure.1 Our service area

2.2 Outline of Senboku LNG terminals

The Senboku LNG terminals are important energy-producing terminals that supply about 70% of the city
gas supplied by Osaka Gas Co.,Ltd. The Senboku LNG terminals consist of Terminal 1 and Terminal 2.
Terminal 1 started service operation in 1971 and has been receiving LNG since 1972. Terminal 2 started
operation in 1977. Both terminals have been operated for over 30 years. In the Senboku LNG terminals, we
manage various businesses. This is a characteristic of our terminals. In addition to manufacturing natural gas
for end users, we supply natural gas for power plants in the terminals, following deregulation of the electric
power market, and also make use of the cryogenic properties of LNG in cryogenic power generation or to
make cold products such as liquid nitrogen. Moreover, we make use of our neighboring plants, sharing some
utilities with them or supplying LNG cold energy. Thus, we have developed a lot of businesses associated with
our LNG receiving terminal.

2.3 Background of the development of this numerical evaluation system

On the other hand, we are now facing some difficulties. Firstly, various kinds of know-how are necessary for
operating our LNG terminal. It is hard for operators to adapt to multi-energy manufacturing. Secondly, we
need rapid promotion of young operators to replace retiring veteran experts. In 2004, the proportion of expert
operators who are over 45 years old in our LNG terminal was 63 percent. We estimate, however, that this
proportion will decrease to 12 percent in 2015. Figure.2 shows a prediction of this transition made in 2004.

2
25

20

15 Start-up
Mid-level
10 Expert

0
<20 <25 <30 <35 <40 <45 <50 <55 <60

In 2004: actual data

25

20

15 Start-up
Mid-level
10 Expert

0
<20 <25 <30 <35 <40 <45 <50 <55 <60

In 2010: estimated data

25

20

15 Start-up
Mid-level
10 Expert

0
<20 <25 <30 <35 <40 <45 <50 <55 <60

In 2015: estimated data

3
 Figure.2 The transition of the age of the operators
in one of our terminals, as predicted in 2004

Thirdly, we should try to improve individual ability to manage the operation by small groups. Of course,
where possible reduction of the number of operators is also required for cost cutting purposes.
To solve these problems, we have developed a human resources management system. In this program, we
define three levels of operators. Thus, all shifts can be comprised of sufficiently well-trained operators, and we
can continue to conduct reliable operations.
To assess how well-skilled an operator is, it is quite important to clarify the level of the knowledge/skill that
should be obtained. Numerical evaluation is one way to effectively judge the skill of operators. We asked
Toyama University to collaborate in development of our training system because it has extensive experience
of developing process simulators with quantitative learning-level evaluation methods. We developed our own
original numerical system for evaluation of training results and took out a patent in October 2007.

3. Development of a highly-functional simulator (Ref.1 and Ref.2)

3.1 features of the simulator

The simulator that we developed has the following features.

・ It can execute extremely precise simulation of actual operation

・ It has an infinite number of malfunction modes
・ It can produce a training program from both the viewpoint of a panel operator and a field operator
・ It can be introduced at small cost

Figure.3 shows the first feature of this simulator. It simulates interactions of facilities in our LNG terminals so
comprehensively, for example operation consoles, control systems and circuits of local facilities, that incidents
including various factors can be produced. Even the timing when a facility is in operation can be precisely
similar to an actual one. Thus, we can conduct highly effective training. In addition to training, the simulator
can be used for verification of reactions of process in various conditions or to study the suitability of operations
planned for the first time, because of its high-level functions.

4
 Plant
(Local equipment)

Relay circuits
(Local equipment control)
Operation computers
(Central control room)

Power system Control system

(Power receiving/transformation room) (CPU room)

Figure.3 The layout of the simulator

The second feature is its number of malfunction modes. The simulator can, of course, produce trainings
based on various kinds of normal operations. In addition, it has a function that can replicate various kinds of
malfunctions. Therefore, we can also enhance operators’ ability to respond to emergency cases.
In this simulation model, all kinds of facilities such as pumps, heat exchangers, control valves, and so on, are
also included in the malfunction modes. So, all kinds of malfunctions can be replicated and the number of
combinations of malfunctions is infinite. Figure.4 shows the example of the malfunctions replicated by the
simulator.

Electrical shut down

LNG pump trip Earthquake

Figure.4 the example of the malfunctions in the simulator

5
 The third feature is that it can display from both the viewpoints of a panel operator and a field operator. If we
try to operate some facilities in the field in the simulator, we can operate them on the simulator display.
Therefore, not only remote operation in central control room but also local operation in the field can be
practiced. Figure.5 shows an example of the display in field operation training mode .

Figure.5 An example of the display in field operation training mode

In this training mode, trainees try to move in the field through a controller on the display. They can feel as if
they are in the real field and they can also understand the way to a point in the field and how much time it
takes to move from one point to another point in the field.
Figure.6 shows another example of the display in field operation training mode . It shows a gauge and a
valve in the field. If trainees click the picture of a gauge on display, the value of the gauge is displayed above
the gauge. If they click the picture of a valve on display, the controller of the valve is displayed and they open
or close it by the controller. Of course, these operations influence the other process parameters in the
simulator.

Figure.6 Another example of the display in field operation training mode

The fourth feature is price. Since we made use of regular-type, not specially-ordered computer and software,
the simulator can be introduced at reasonable cost.

6
3.2 Development of numerical evaluation system for operators

In training with a conventional type of simulator, it is true that trainers can fully evaluate trainees through
definition of skill levels needed for operation but it is hard to say that they can evaluate numerically or
objectively.
We developed a numerical evaluation system for operators by comparing similarity with operation by a
model operator. In this evaluation, we adopted some mathematical methods.
Our numerical evaluation is based on two parameters. One is ‘procedure’, similarity to a skilled operator’s
manipulation process. The other is ‘stability’, coincidence of type of alarms (temperature, pressure and
flow-rate etc.) during the trainee’s operation. First we need to assemble data on a skilled operator’s (expert A)
manipulation process and alarm information (what we call ‘stability’). When examinees to be evaluated
(trainee B) carry out operations, the system automatically compares information from expert A to that of
trainee B. It can quantify the degree of coincidence of the expert’s data with the trainee’s results.
Figure.7 shows an output image of this evaluation. The horizontal axis shows coincidence score of procedure.
The vertical axis shows stability. These two parameters enable the system to achieve more precise
evaluation.

Good
100
Stability

Bad

Error Correct

0
0 100
Procedure

Figure.7 Output image of this evaluation

3.3 How to evaluate

We notified two differences. One is a difference in record of manipulation process. The other is a difference
in distribution of alarms or square configuration of alarms. Figure.8 and Figure.9 show examples. These
graphs are from a case of electric power failure. The horizontal axis shows elapsed time (minutes) and the
vertical axis shows the number of alarms (piece/minutes) and operation frequency (times/ minutes). The area
graph shows the number of alarms and the bar graph shows operation frequency. Figure.8 shows the result
by a skilled operator and Figure.9 shows the result by a trainee operator.

7
 Skill is high. Skill is low.

140 12 140 12

130 130
10 10
Warning(Piece/minutes)

Warning(Piece/minutes)
frequency(Times/minutes)
120

frequency(Times/minutes)
120
110 8 8
110

Operation

Operation
100 Warning Warning
6 100 6
90 Operation Operation
90
80 4 4
70 80
2 2
60 70

50 0 60 0
1 3 5 7 9 11 13 15 17 19 21 23 1 3 5 7 9 11 13 15 17 19 21 23
Elapsed time(minutes) Elapsed time(minutes)

Figure.8 The result of a skilled operator Figure.9 The result of a trainee operator

Next, we compared alarm data made by a trainee to that by a skilled operator. Then, we numerated the
similarity by the following mathematical methods.

・Comparison based on correlation coefficient

・Comparison based on Euclidean norm
・Comparison based on hidden Markov model

3.4 How to define an ideal model

Thus, a definition of the model that is compared is essential in this evaluation system. Some people may say
that an ideal model is one made by a computer because of its accuracy and quickness. Human operators,
however, cannot conduct the same operation as computers. Computers can conduct such an impossible
manipulation from human viewpoint as a parallel processing. Therefore, we chose some skilled operators and
compared the results by the operators with an ideal model made by computers. Then we determined the best
model from the operators with this numerical evaluation method. Figure.10 shows the ideal result made by
computer, in the case of “LNG pump trip”. On the other hand, Figure.11 and Figure.12 show results made by
two skilled operators. In this case, if the score of the result by computer is 100, that of the result by operator A
is 75.8 and that by operator B is 74. Thus, we judged that the result by operator A is the best model in the
case of “LNG pump trip”.

Control by computer

50 12
45
10
frequency(Times/minutes)

40
Warning(Piece/minutes)

35
8
Operation

30
Warning
25 6
Operation
20
4
15
10
2
5
0 0
1 2 3 4 5 6 7 8 9 10 11
Elapsed time(minutes)

Figure.10 The ideal result made by computer

8
 Skill is high. Skill is high.

50 12 50 12
45 45
10 10

frequency(Times/minutes)

frequency(Times/minutes)
40 40

Warning(Piece/minutes)
Warning(Piece/minutes)

35 35
8 8

Operation

Operation
30 30
Warning Warning
25 6 25 6
Operation Operation
20 20
4 4
15 15
10 10
2 2
5 5
0 0 0 0
1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11
Elapsed time(minutes) Elapsed time(minutes)

Figure.11 The result by operator A Figure.12 The result by operator B

In other cases, we use the same method to define the best model in this numerical evaluation system.

4. Application for operator training

We have applied this system for operator training. Up to the present date, 32 patterns of training by the
simulator have already been arranged that trainees can be evaluated numerically. Besides, about 50 patterns
will be added in a year. At all levels, we impose them on monthly and yearly training menus.

4.1 Training for start-up operators

We impose start up operators on 11 patterns of training by the simulator in a year. They self-train
themselves using the simulator. We introduce this system’s effectiveness by the case-study of ‘LNG pump
trip’. In this case, operators are expected to implement the following four steps after the occurrence of a
simulated alarm.

1. Noticing of pump trip

2. Start-up spare pump
3. Adjust the pump load to stabilize liquid pressure
4. Confirm stability of liquid pressure

Figure.13 shows the improvement process of a new operator and an example of quantitative evaluation
results. Eight examples are included in figure 1. Both procedure and stability scores become slightly better
and you can see how it functions.

9
 100 Result
8ｔｈ
80

Stability
60

40 ２nd

20 1st

0
0 20 40 60 80 100
Procedure
Figure.13 Improvement process of a new employee and example of quantitative evaluation results

4.2 Training for mid-level operators

Figure.14 shows the examples of quantitative evaluation results of eight operators with less than 5 years of
experience. We require start-up operators to undergo 14 patterns of training in a year. Thus, they are
repeatedly trained by use of the simulator.
On the first time, both procedure and stability scores are scattered. But from the second time, we can see
the scores of operators A and B improve as a result of training with this system.

First time Second time

100.0
100.0

90.0
90.0

H
80.0
80.0 E

F
C 70.0
70.0

D
G
After
Stability
Stability

60.0 training 60.0

B
Mr.A 50.0
50.0

40.0
40.0

30.0
30.0

20.0
20.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0

Procedure Procedure

Figure.14 Examples of quantitative evaluation results of operators with less than 5 years of experience

4.3 Training for experts

Figure.15 shows examples of quantitative evaluation results of operators with over 10 years of experience.
We require start-up operators to undergo 17 patterns of training including patterns which mid-level operators
are required to undergo, in a year.
There is much difference of proficiency among skilled operators from previous results. But operation
procedure is similar among them. On the other hand, there is difference in stability scores. This is because
timing of operation or number of manipulations is different, since skilled operators each have their own
operation style. You can see well-marked improvement of stability scores.

10
 100.0 100.0

90.0 90.0

80.0 80.0

70.0 70.0

After
Stability

Stability
60.0
Training 60.0

50.0 50.0

40.0 40.0

30.0 30.0

20.0 20.0
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0

Procedure Procedure

Figure.15 Examples of quantitative evaluation results of operators with over 10 years of experience

5. Successful results in utilization of this system

5.1 Application for qualification test using this numerical evaluation system

We define three levels of operators: start-up, mid-level, and expert. We confirm the knowledge/skill level
that each has reached, by conducting qualification tests. In one of the tests, applicants are judged objectively
by this numerical evaluation system. In this test, a pattern of training is selected at random. So, applicants
cannot know what will happen on the simulator in advance.
Of course, not only evaluations by simulator but also those by supervisors are added to assess the
qualification. Practical tests and public qualifications are also needed for promotion. In the earliest case,
operators can qualify for start-up level in a year, mid-level in two years and expert level in five years. Thus,
qualification tests including this numerical evaluation system assist rapid promotion of operators.

5.2 Successful results in utilization of this system

Thus, we have developed and utilized this system. Consequently, we have attained the successful results in
training operators. Figure.16 shows actual data of the number of operators according to skill level in 2004 and
in 2009. In 2004, the proportion of expert operators was 63 percent (Figure.2). Then the proportion in 2010 is
estimated to decrease to 40 percent. The proportion in 2009, however, was 54 percent because of utilization
of this system (Figure.16). It is true that there is a difference of the number of operators between 2004 and in
2009, but rapid promotion of young operators (under 40) is noticeable in the figure.

11
 10%

Expert
27%
Mid-level
Start-up
63%

In 2004: actual data

13%

40% Expert
Mid-level
Start-up

47%

In 2010: estimated data

23%

Expert
54% Mid-level
Start-up
23%

In 2009: actual data

Figure.16 Successful results in utilization of this system

12
6． Conclusion
In order to manage various businesses at LNG receiving terminals, we have developed a quantitative
learning level evaluation system. It enables us to evaluate various levels of operators in detail and measure
the effectiveness of training. It is very useful for clear assessment of each operator’s knowledge/skill level.
Through application of this system in our terminals, we have actually trained our operators rapidly and
steadily.
In conclusion, the key features of our quantitative learning level evaluation system are represented by the
following three points.

1. It can rapidly and steadily improve operation skills by comparing less-skilled operators with skilled
personnel.
2. It takes little time to establish the model. (Just operate to make a model, no parameter setting is necessary)
3. It provides a convenient means of preserving operation know-how for the next generation

Thus, our human resource management system is efficient in promoting a safe and cost-efficient (in other
words, sustainable) supply of multi-energy. We will continue to expand our terminal business.

References
1. Taketoshi KUROOKA, Hisato INOUE, Yasushige WATABANE, (2008), A New Metric For Evaluation Of
Plant Operator’s Proficiencies. FOCAPO 2008
2. Hisato INOUE (2007), Development of an LNG Receiving Terminal Process Simulator and Construction of
an Education/Training. LNG15

13