Machine Vision Technology :

Past, Present, and Future

Masakazu Ejiri
Central Research Laboratory, Hitacki, Ltd.
Kokubunji, Tokyo 185, Japan

Abstract
.
This paper discusses the past, present, and
future of machine vision technology,
emphasizing its practical aspects. Four types of
machine vision technology based on pattern
matching, feature parameter, window, and slitlight methods are now being widely used in
industry. However, recent applications require
more effective use of knowledge-based
processing, combined with application-specific
methods. Study is fius required for the future
progress of machine vision, especially in such
areas as sensor fusion, real-time processing, and
visualization/enhancement technology. Some
new future applications are also discussed.
c

1 Introduction
Japanese industry has been taking the initiative in
creating the "machine vision" field and has played a
major role in its advancement since its inception. In
the past 20 years, a large number of industrial
processes have been automated using machine vision,
most prominently in the semiconductor industry
where production places heavy demands on precision
and efficiency. However, many processes remain
unchanged due to the difficulties in reliable
recognition.
Although "computer vision" research has also
advanced greatly in that time, it is rather
methodology-oriented, and unfortunately does not
suggest pragmatic solutions to difficult application
problems. m a t is, research and development seems to
be stagnant from the application point of view. To
look at the future of machine vision research, it is
worthwhile to look back to what we have done, to
understand where we are now, and to think about
what direction we have to move in.

2 Brief History of Machine Vision

The first vision-based intelligent robots in Japan

appeared in 1970, one from the Electro-technical
Laboratory of MITI, and the other from the Central
Research Laboratory of Hitachi Ltd. Each was
controlled by a minicomputer whose capability was
far less than today's microcomputers. The
configuration of the Hitachi robot is shown in Figure
1. [1,2] This robot is typical of the "intelligent
machines", as shown in Figure 2. The outstanding
feajure of this robot was a macroscopic instruction, in
this case assembly drawings. This enables the robot to
be controlled without any further microscopic
instructions corresponding to each robot movement.
The concept of the intelligent robot is still alive. [3,4]
Since these developments, considerable work has
been done in Japan, especially for industrial
applications, and unique, basic, and applied research
on industrial machine vision systems has been carried
out. The first successful application of image

tor

---,Objects

Figure 1.

An intelligent robot that assembles

objects from plan drawings

processing to industrial automation was the defectinspection machine for printed circuit boards in 1972.
[ 5 ] In 1973, an automatic wire-bonding system with
time-sharing vision was developed for transistor
assembly. 161 This was later extended to the
development of systems for automatic assembly of
ICs and LSIs. A robot for bolting molds for concrete
piles and poles, developed in 1974, was the first
application of the dynamic recognition technique of
moving objects. [7] Through these pioneering
studies, the importance of vision techniques is now
recognized in many sectors. In particular, the
successful developments of a local pattern matching
method [6] and the time-sharing vision systems based
on it made a great contribution to the semiconductor
industry. At the same time, it encouraged researchers
towards further development of machine vision
technology.
In the computer vision fields at that time,
researchers were struggling with the more general
methods of vision using imaging devices looking
obliquely at the object. A research paper that showed
the possibility of categorizing objects using a
vertically-set imaging device did not get much
attention at first. [8] However, since the SRI
algorithm based on the feature parameter method [9]
was proposed, vertical observation gradually
obtained much attention as a simple but effective
method for industrial object-recognition. This is now
a standard vision method for industrial applications.
As for the hardware for a machine vision
system, a costly minicomputer was first used in
combination with a special-purpose image processor.
To lower the cost/performance ratio, a large-scale
manufacturing system with 50 transistor assembly
machines was developed, where one minicomputer is
shared by five image processors, each servicing ten
cameras attached to the machines. [6] As the
availability of microcomputers increased, stand-alone

Monocular

Figure 3 .

Shape-from-shading
Photometric stereo
Shape-from-texture

Methods for machine vision

assembly machines became dominant, combining a

microcomputer and an image processor. Nowadays,
these machines are again being connected with each
other, through a LAN system, to form an integrated
manufacturing system with distributed intelligence.
Machine vision was first implemented as
application-specific, special-purpose vision systems
using binary image processing techniques based on
local-parallel image processors. However, as the
availability of image-processing LSIs increased,
gray-scale image processors became a cost-effective
reality, and are now being effectively applied in
industry.

3 The state-of-the-art of Machine Vision

ronment

Environment
understanding

Figure 2.

Spot I i gti t
Slit-light
Grid-light
Time-of-flizht

o+fJr ;o
Intelligent
intedace

Pattem matching
Feature parameter
Window

Passive

Visual, auditory, tactile senson

A model of intelligent machines

xxx

3.1 Outline
Many types of machine vision technology have
been developed to date as summarized in Figure 3.
The passive technology in 3-D vision typically
includes binocular vision and the "shape-from-X"
method where X implies such words as shading,
texture, and motion. However, these methods have
not yet been fully applied to industrial use due mainly
to restrictions on processing speed and reliability.
Instead, the practical machine vision systems
commonly used in industry are based on a few
simplified, fundamental technologies. These are
passive, monocular, 2-D vision technologies:
* pattern matching method,
* feature parameter method, and
* window method.
The active technology typically uses a structured light
method or a time-of-flight method to give the distance
between the object and imaging device. One useful
structured light method is:
* slit-light method,
which is based on the triangulation principle.
These four methods are effective in industrial

applications, on their own or in combination.

However, the technology level attained so far in
machine vision applications is still low and can be
summarized as follows:
Recognition of geometric features,
supplementary features, and a few restricted
cases of qualitative features of simply-shaped
objects. These objects must be within a
predicted area or positioned in a
predetermined area. Usually, the illumination
is carefully controlled.
At present, machine vision techniques cannot be
applied to:
Recognition of most of the qualitative features
of even simply-shaped objects, and the
geometric and supplementary features of
complex objects, especially when the
illumination changes drastically such as in
outdoor, day-and-night use in all-weather
conditions.
The feature parameter method and the window
method are flexible enough for general-purpose
vision systems, while the others are mainly used in
special-purpose systems. More than 300 generalpurpose vision models have been developed in Japan,
bved roughly on these two methods. However, only
100 models survive nowadays in terms of active sales,
although the business is still small and progressing
slowly. It is estimated that more than 7,500 machine
vision systems are being successfully operated in
Japanese industry. The main uses of machine vision
arc in the assembly field where part identification and
determining position/orientation are the main areas.
Machine vision is also used in the inspection field
where pattem alignment, dimension measurement,
and defect detection are the main areas.
One recent trend in machine vision is to improve
machine vision systems using the knowledge of
objects and environment (mainly factual knowledge),
and the knowledge of the methods of image
processing and decision-makiig (mainly rules). These
attempts focus on the effective use of knowledge to
automatically generate optimal procedures in preprocessing, feature extraction and structural analysis
for a given application. The following two features
are critical [lo];
(1) Knowledge separation,
where the knowledge is stored separately
with the processing procedure. Thus,
functions can be altered by replacing the
knowledge. This also pro'vides a basic
framework for automatic knowledge
acquisition.

(2) Knowledge or model driven,

where the results from pre-processing,
feature extraction, and structural analysis
are compared to the knowledge and the
decisions are fed back to the preceding
processes. Flexible processing thus
becomes possible. The knowledge can take
various forms such as explicitlyrepresented if-then rules and models, or
implicit forms that can be derived from
these explicit representations.
In industrial applications, however, a simpler
s-ystem will be more cost-effective, faster and
reliable. In past applications, object types and imaging
methods have been somewhat restricted to meet these
cost, speed, and reliability conditions. For example,
recognition of complex 3-D objects is simplified to a
2-Drecognition problem by constraining a certain
object surface to a standard base plane., Then, a
number of algorithms are developed, and the best is
selected and implemented into a procedural highspeed image processor. Thus, "image understanding"
technology based on knowledge has seldom been used
in attual industrial applications, although it has been
useful in simulations in algorithm development. One
reason is that knowledge-based processing is still
slow. Another is that feedback from the matching
process may disturb the rhythm of the manufacturing
process.
However, the importance of knowledge
separation from processing procedures, and of the
knowledge-driven or model-driven techniques for
controlling the image processing is gradually
increasing, leading to more general and more
expansible machine vision. The following sections
briefly discuss the four basic machine vision methods
from the viewpoint of their knowledge-related
aspects. A minimal framework for knowledge
separation and a knowledge-driven feature can be
seen in these methods.

3.2 Pattem matching method

A typical example of this method is the

recognition of characters on the object surface, where
the unknown input image is directly compared with
standard pattems or templates for each character.
These templates can be regarded as factual knowledge
of characters, and thus separation of the knowledge
from the procedure is in common use. Another
example of pattem matching is the recognition of the
position of complex objects such as transistors, ICs,
and =Is, where a few local pattems are extracted in
advance from an object pattem as standard pattems,

XXXI

and are used to detect the best matching positions in

the input image. [6] This method is shown in Figure
4. The distances and angles between the detected
positions are then verified. Thus, the knowledge used
in this method is the local pattems themselves, and the
relationships between them. By changing the
knowledge,, the system can be adapted to various
objects. When the geometric check using distances
and angles is verified, the positions for subsequent
wire-bonding are calculated. The local patterns
should be unique to each other in this method, and the
selection of the local patterns from a given object
image has also been automated. [ 111 This automatic
selection can be regarded as a means of automatic
knowledge acquisition.

Instead of complex local pattems having unique

shapes, unique combinations of much simpler pattems
in the image can be used to find the position of the
object. [12]. Figure 5 shows such a method.
Positional relation among simple local patterns (such
as comer patterns) distributed in an entire image is
represented as a feature grid pattern consisting of
their pattern codes. Thus, structural matching of
pattems can also be implemented by conventional
pattern matching hardware. The knowledge used is
the feature grid pattern that represents the structure
of the standard object pattem from which an inputting
feature grid pattern is searched. The exchange of
knowledge enables the system to be adapted to new
objects.

3.3 Feature parameter method

Temp1 a 1c

This method extracts several geometric features

from an object pattern. The features include the area,
peripheral length, number of holes, and moments.
Thus the object can be represented as a point in ndimensional feature space, and compared with the
standard regions of each object to find the closest
object category. Standard regions can be taught by a
teaching-by-showing method. Thus, the knowledge in
this method is a combination of pattern features of the
objects.
Another method uses a decision tree derived
from the measured parameter distributions of each
object in a teaching process. The feature parameter
that distinguishes the distributions most is the first
parameter to be examined, as shown in Figure 6.
Thus, the method divides the problem into subdecision problems, and generates a decision tree

pal terns

Emitter

Figure 4. Local pattem matching method

Res11I t,s

Decision t r e e

I
L i

'---Feature grid pattern ( f o r e n t i r e

patterns

corner

1-1

imagp)

Parameter x ,

1 2 4 8

Figure 5.

Structural matching method for

feature pattems

Figure 6.

XXXE

Decision-tree type feature parameter

method

automatically. This generation is one means of

automatic knowledge acquisition, and the tree is
secondary knowledge derived from the primary
knowledge of the object parameter distributions.
One recent application of this method using a
gray-level image is to a defect-detection machine for
light-emitting diodes. [ 13J
The projection
distributions of brightness in two orthogonal
directions are calculated from a gray-level image of
the radiation pattern of the diodes. The Fourier
coefficients of the distributions are then used as
feature parameters for finding small brightness
irregularities. The empirical decision criteria of a
skilled human inspector can be learnt by showing
normal diodes to the system, and can be installed as a
parameter space distribution. The decision is made in
terms of Mahalanobis distances to the distribution of
normal diodes.
General-purpose machine vision systems based
on this method are already widely available, and are
mainly used for object discrimination in various
industrial processes.

(c) Shape discrimination

Figure 7.

Center point

3.4 Window method

( b ) Timing finding

(a) Position finding

Window method

Corner point

Edge line

(a) Primitive features

Observation of selected portions of an objett

image can be used to recognize object category,
position, and orientation. [3] Object areas in the
windows are the basis for the decision when the
windows are appropriately placed in the image field.
Some variations of the window method are &own in
Figure 7. Relative positions and window sizes can be
varied to search dynamically for different objects.
Thus, the knowledge used in this method is the sizes
and shapes of the windows, the pattern features
(mostly areas) in the windows, and the relative
positions between windows.
Such a machine vision system detects an object's
comer point, center point, or an edge line from each
window as shown in Figure 8, and then combines
these primary points and lines to give compound
features such as distances, angles, and cross-points.
[I41 These are used to find the object category,
position, and orientation. This vision system has been
implemented as a general-purpose system, and is
being applied effectively in such fields as VCR
assembly, as shown in Figure 9.

1.

Line vs. point

Point v s . point

Line vs. line

( b ) Compound features

Figure 8.

Features used in a window method

3.5 Slit-light method

This method projects a slitqight beam from one
direction and observes the object's reflection from
another direction. Triangulation gives the distance
between the object and the observation point. One

xxxm

Figure 9.

An assembly cell for VCRs

typical application of this method is a sealing robot

with a slit-light projector and an image sensor on its
wrist. A slit-light beam is projected vertically to a
seam between two overlapping plates. The image
obtained from an oblique direction contains a kink in
the slit-light beam. The position of the kink in the
image is kept centered by moving the arm, and sealant
is applied to the seam through a nozzle. This gives
continuous sealing of the curved seam. [ 151
Other applications include inspecting soldering
between LSI leads and printed boards. Profiles of the
leads are detected by the combination of a scanning
spotlight and a linear sensor, and are used for
classification into normal, non-contact, mis-aligned,
or bridged leads. [ 161
This method is also used for inspecting
assembled printed boards. Slit-light beams are
projected individually onto each component on the
board, and the heights and widths are checked to
reveal any mis-placed or mis-oriented components.
Conventional gray image analysis can be used in
combination with this method to check the polarity of
capacitors and the component names by reading the
surface marks. [ 171

processing u n i t ( I ' U )

Figure 10. Basic configuration of knowledgedirected process (generic form)

Fat.al

Drfcct

Input
image

randidate

Defect

image

image

dcfert
data

3.6 Design pattem referring method

In addition to these four conventional methods,
various application-oriented, special-purpose
techniques have been developed. One recent technique
is the inspection of complicated patterns. Until
recently, a two-chip comparison method has been
widely used to find abnormalities in complex binary
patterns, for example, for mask inspection in
semiconductor manufacture. However, this method is
becoming inadequate for application to recent high-.
density pattems, particularly to gray-level images of
semiconductor wafers, and a design pattem referring
method is becoming more favored. In these cases,
pipelined image processing is necessary for providing
high-speed abnormality detection. The knowledge to
b e implemented in the pipeline image processor
includes:
(1) pattems to be inspected,
(2) substances that form the pattems, and
(3) abnormality definition.
These can be encompassed in a generic form as:
If a pixel (or a pixel within a certain blob) is at
a certain position, and if the feature value(s) of
the pixel (or blob) is within a certain
parameter range, then the pixel is regarded as
a decision and is processed by a certain
operation.

Figure 11. Basic pipeline architecture for defect

identification
An instance of this generic knowledge is:
If a pixel is on a silicon-oxide layer, and if the
brightness of the pixel is outside the range Ti
to T2, then the pixel is a part of a defect
candidate, and is flagged.
A processing unit that uses this type of knowledge can
be implemented in hardware, as shown in Figure 10.
In this configuration, an input image and a number of
design patterns are simultaneously fed into a
processing element and a control element
respectively. The design pattems are generated from
the design data in such a way that they are accurately
aligned with the input image. This alignment is
executed by a position calibration circuit, prior to
their application to the control element. The control
element outputs different control parameters
depending upon the position being scanned, and the
parameters control the processing element. Several
processing units, each of which is the combination of
a control element and a processing element, can be
connected serially or in parallel to form a pipelinestructured inspection machine. Figure 11 shows the
basic architecture of such a machine. A wafer

XXXN

::
,I

I.ighl

De l e c t

detect.ion c

Calibration
circuit

Dcsign
pattern
generator

1
circuit
Design d a t a

'Table

I
I

CAD sysien
(Nain f r a w compri k r )

Figure 12. Wafer inspection machine

(for logic ICs)
inspection machine for logic ICs based on this
principle is shown in Figure 12. [18,19] A wafer
inspection machine for high-density memories has
also been developed based on the same principle.[20]
In the candidate extraction stage of the defect
detection circuit in these machines, an optimal defect
discrimination procedure is selected for each layer
depending upon the pixel position currently being
scanned. In the defect judgment stage, the design
patterns and the input pattern are processed by the
same procedure, and the results are compared. One
example of such a procedure is illustrated in Figure
13, where pattem-widths are measured using a
distance-transform filter for checking defect seuerity.
This means that the original design patterns are
modified to generate new knowledge for comparison.
Design p a t t e r n

r-l

Input patt.ern

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

1
2
3
4
5

4 Future Machine Vision

4.1 Aspects of machine vision research

To achieve higher-level machine vision for a

wide variety of applications, it is necessary to further
research on computer vision and image processing
that form the basis of practical machine vision
technology. However, the research in these fields
seems to have the following aspects, which can easily
lead research in the wrong direction.
(1) Easy-to-start

r"-1
l==A1
1
2
3
4
5

The circuit for this operation can also be represented

by the generic form shown before in Figure 10.
The calibration circuit is used to measure both
the positional error and the pattern-width error
between the input pattem and design patterns, using a
real-time correlation calculation. The results are fed
back in real-time to the timing of design-pattern
generation and to the dilation/erosion control of
generated pattems, to ensure precisely-registered
design gatterns. This calibration circuit further
modifies control parameters such as Ti and T2 used in
the candidate extraction and defect identification
stages, by periodically measuring the frequency
distribution of brightness for each layer portion of
the pattem. Thus, this circuit serves as a mechanism
for modifying the knowledge.
The wafer inspection machines based on this
method use a model, i.e. design patterns, as the
pattern for comparison, as the signal for controlling
the process, as the basic knowledge for deriving
secondary knowledge, and as the original standard
knowledge for modifying itself to adapt to the
changed condition. Thus the method is highly reliable
and gives real-time high-density pattern inspection,
despite its being knowledge-driven.

1
2
3
4
5

1 1 1
2 2 2
3 3 3
4 4 4
5 5 5

1 1 1 1
2 2 2 2
3 3 3 3
4 4fi

1 1 1 1 1
2 2 2 2 2
2 3 3 3 3
4
4 41

5 5 3 9 3 5 5 5

9Cornpar ison

Figure 13. Pattern-width measurement for defect

fatality based on distance-transform

Anyone can start research in this area quite

easily. The analysis of only one image taken under a
certain -- usually unspecified -- condition easily
satisfies the researcher, due simply to the fact that the
result can be a beautiful picture apparently different
from the original input image. This illusion may lead
one to view the research as quite successful, but it is
still superficial, lacking the depth, durability, and
adaptability essential for practical applications.

(2) Difficult-to-evaluate
Judging the effectiveness of this technology is in
general very difficult, as the heart of the technology is

xxxv

an indefinite algorithm. The benefits of the algorithm

can only be evaluated quantitatively by actually
applying it. This is one reason why technology
transfer to the users is difficult to achieve.

++'

I+,

Ima e sensors

(3) Difficult-to-accumulate
Different assumptions are made in every
research paper. Not all of them are practical, making
technology accumulation difficult. The processing
speed problem is usually ignored, with the assumption
that it will be solved in the future as semiconductor
technology progresses. This may be partially true.
However, there is also the possibility that vision
problems currently being studied will be worthless
because they will be solved easily by other means
before the necessary processing speed is attained.

Real

1I

i t

In-process conhol
(adaptive and predictive)

Knowledge-base, model-base

Object flow
Information flow
Machine condition
System condition
configuration

External dormation

Figure 14. Intelligent manufacturing control

system

4.2 Important technologies in machine vision

Bearing these aspects in mind, continuous efforts
are needed to develop and accumulate algorithms and
apply them to various fields. In particular, the
following issues will be increasingly important in
future machine vision.
(1) Sensor fusion
Many products with different specifications will
be needed to meet the increasingly diverse needs of
individuals. Therefore, the following techniques are
becoming more important in manufacturing research.

PI1

* Product realization
* Intelligent manufacturing control

The former is a design technique starting from the

conceptualization of a product based on human needs,
and finishing with a product design through quick
prototyping and evaluation, where the intelligent use
of computers will be the key to success. The latter is a
control technique at the time that the products are
manufactured. The manufacturing system will thus be
configured as in Figure 14, where the flow of
information as well as of objects must be controlled
intelligently. Thus, machine vision will be important
for detecting the object and machine conditions.
The data sensed by these vision systems are then
integrated to keep the manufacturing system
operating optimally, Thus, the importance of sensor
fusion technology will incfease. However, the method
of integrating such sensed data is still unclear. One
possible way to obtain effective sensor fusion
technology will be through neural networks.

(2) Inspection for varying specifications

In the manufacturing process, at least two types
of machine vision system will be required: one to
continuously check for maximum efficiency of
operation in each manufacturing machine; the other
to check that each product is being correctly
manufactured. This inspection function must be
distributed to each important stage of the
manufacturing process. In the 1990s, mainstream
machine vision applications will thus change from
assembly to inspection. In particular, inspection
techniques that can adapt to a group of products made
under different specifications will be crucial.
The outputs from the sensors can directly
control the objects being processed and the machines
and equipment used in manufacture. This is a means
of in-process adaptive control. If a machine vision
finds a slight erroddeviation in the object shape being
processed, this information must be fed to subsequent
processes. Thus, the downstream processes must be
modified to adapt to the object before it becomes
seriously defective. This method is called predictive
control. Feed-forward predictive control will be an
important feature in advanced CIM (computer
integrated manufacturing) from the viewpoint of no
defective products.

(3) One-pass real-time processing

Future applications will demand real-time image
processing, as the number of operations required for
recognition is growing greatly. It is obvious, from the
technological trends in semiconductors as will be
discussed later, that adequate image memory capacity
is obtainable, even for extremely large images.
However, it is usually a waste of time to first store the

XXXB

image into an image memory and then process it by

accessing the image memory. Especially in industrial
applications, it is necessary to develop a fast method
in which the completion of image acquisition implies
the completion of the image processing (or at least
pre-processing).
In the wafer inspection machine described
earlier, a control image is first generated from the
defect candidate image, as shown in Figure 15. In this
control image, all holes and downward dents are
filled by a recursive, propagation-type distancetransform filter. Therefore, when the two scan lines
are observed, it is possible to decide whether the

llead

Tail

observed blob is the head portion, the intermediate

portion, or the tail portion of an object figure. Thus,
by controlling the state-transitions of a sequential
machine (shown in Figure 16) in response to the twobit signals (numbered 0, 1, 2, and 3) observed at the
vertical position in the two adjacent scan lines,
features of each figure can be accumulated until the
scan reaches the end. Thus, feature parameters such as
area, peripheral length, and projected length can be
found for each figure after one vertical scan. [20]
'?his one-pass real-time processing technique can
be applied to the usual defect detection and particle
analysis problems. However, if the image is a mixture
of extremely complicated pattems, this method will
not be applicable because the figures would be fused
when the control image is produced. An improved
method has thus been proposed, and its strictness has
been proven theoretically for any complicatedlypatterned figure arrangements. [22] This improved
method can measure the object features of such a
pattem as shown in Figure 17, which has two object
pattems. The feature values are output at the end of
every scan of a figure (at a point @ in the figure).
As shown in these examples, raster-scan, onepass real-time processing techniques based on on-site
processing will be increasingly important in the
future. Extensive study will be needed on the widerange of real-time processing algorithms.

Body-2

Body-1
.

Scan1 iiie Internal

number s t a t e
NIL
(SSSS)
(LRSLR)
(SLRLRLR)

Figure 15. One-pass feature extraction method

Start

, from

buffer

(LSSflR)

(LSSHR)
(LHflflflR)

NIL
Detected pattern features
PFl(123 27 5 28 11 0 0 0 0)
PFZ(256 30 3 31 2 0 0 0 0 )
Tai 1

Body

Head

Figure 17. One-pass feature extraction from a

complex image (Contains two figures
with 4-neighbor connection mode)

Figure 16. A sequential machine for one-pass

feature extraction

xxm

(4)Visualization/enhancement technique
Vision systems for complicated 3-D objects,
objects with much more complicated surface pattems,
and hard-to-see objects will also be needed. The
matching of distorted image signals in SEM images
[23] is a typical example of recent approaches.
Texture analysis is another. A self-organizing image
filter for texture separation [24] has been
demonstrated successfully in this area. A knowledgebased inspection of complicated LSI pattems [ 191 is
also an example of recent approaches to the
recognition of hard-to-see objects.
Recognition of invisible objects is also becoming
increasingly important. One example is the inspection
of soldering of LSIs surface-mounted on a circuit
board, where the LSI leads are completely hidden
under the LSI package. For such uses, visualization
techniques and image enhancement techniques are the
keys to reliable vision systems. Obviously, the use of
X-rays is one possibility for visualization of hidden
defects. Thus, sensors must be improved by
combining them with image processing techniques.
4.3 Trends of related technologies
The fundamental technologies in future machine
vision will be:
* processor technology,
* memory technology, and
* sensor technology.
All of these are core areas in semiconductor
technology. Trends in these technologies will be
briefly discussed here.

- lo6
"E
- lo5 2
3

- lo4
-

10'

- IO2
0

-10

5
m
U

- 1 F - o

- 10~'
Figure 18. Trend of processor technology

10

1-

- 10
E

"
- 100; 4"
Y)

E
E
v
W

.N

-0 E
@lG-

1 ;
L

- IO'

IO2

2
ui

Y
0

- 10'2

10

1980

1985

1990

1995

2000

Figure 19. Trend of memory technology

(1) Processor and memory technology
Recent progress in semiconductor technology
has seen certain trends, as shown in Figures 18 and
19. In the late 1990s or early 2000s, we can expect a
200 MIPS microprocessor (CISC-type) and a 1 Giga
bit memory chip if lithography limits are expanded
and other difficulties can be overcome in each
subsequent generation. This means that a memory
capacity equivalent to the human brain is attainable.
This also indicates that the present supercomputer
may eventually become a palm-top personal
supercomputer. Therefore, optimistically speaking,
almost all the hardware problems conceivable today
could be solved by the end of this decade. Most
problems exist in software/algorithms and sensors. As
for the algorithms; the robustness and the resulting
decision reliability are keys to success.
(2) Passive image sensors
As for the sensors, sensitivity, resolutiorl, and
dynamic operation range are the major concerns for
practical applications. For passive image sensors, the
image processing researchers have had to contend
with illumination problems in every application.
Current MOS and CCD image sensors, and their
successors, will not be suitable. A new image sensor
with an extremely wide dynamic range must be
developed. The lack of dynamic range in current
sensors is making image processing extremely
difficult. If a wide dynamic-range image sensor were
available, a lot of the pre-processing algorithms in
image analysis would not be necessary.
From a sensitivity standpoint, the MCP-CCD, a
charge-coupled device combined with a microchannel plate, shows promise for detecting subtle
image signals. Avalanche-multiplication image tubes
called HARPICON [25] and super-HARPICON,
developed recently for HDTV broadcasting, may also

xxm

become feasible imaging devices. These devices have

a wider dynamic range and an extremely high
sensitivity, allowing a high-resolution color image at
less than 1-lux illumination. Developing these as
solid-state imaging devices could revolutionize
machine vision applications.

1'3)Active image sensors

A range finder based on a triangulation or a
time-of-flight method is typical of an active image
sensor as described before. Unfortunately, recent
research shows little evidence that resolution can be
improved in the future.
The intended use of the range finder is the vision
of a mobile robot with an operating distance of a few
meters to few tens of meters. In industrial
manufacturing applications, however, what is needed
is a shorter-distance, higher-speed profile finder, for
example, having a resolution of 50 micrometers, an
operating range of 1 millimeter, and an operating
distance of 5 to 10 centimeters. This would be
extremely useful for surface profile detection such as
the inspection of object surfaces for pinholes.
(4) Neural network

As processor-related technologies, neural *

networks are being studied extensively. A network
simulator capable of simulating one million neurons
has already been put into laboratory use. With
hardware approaches, many neuro-chips are being
investigated. One recent topic in this field is the
prototype wafer-scale integration of a network with
576 neuro-chips on a single wafer. [26] This indicates
the possibility of integrating more than 10,000
neurons on a single wafer by the late 1990s. The
primary uses of neural networks in industry will be
vision devices to track the flow of parts and products,
and sensor fusion devices for intelligent
manufacturing control.

contribution. to the solution. Some application

examples in this concem may be:
For a clean and attractive environment
* Environmental measurement and analysis
* Land and seabed cleaning
* Garbage treatment
For a flourishing society
* Food engineering such as agricultural
automation, seabed cultivation, and
automated animal breeding
*'Efficient physical distribution control
* New traffic systems
For a safe and stable society
* Traffic safety management
* Disaster prevention and rescue
* Security management
* Customs inspection automation
These applications may include the following new
challenges in machine vision technology. These are
the recognition of:
* Irregularly shaped objects and their
defects
* Shape-changing objects and their
degrees of deformation
* Quickly moving objects and their
absolute/relative speeds
* Hiddenhard-to-see objects and the
seriousness of their defects
* Flexible/soft/untouchable objects and
qualitative features expressed in such
abstract words as matured, tender,
colorful, lovely, and beautiful
* Individualsfliving-things and their
existence, numbers, faces, and facial
expressions
* Groups of living things and their
statiddynamic behavior
The right directions to solve these problems must be
set, and clues to the solutions must at least be
ascertained within the 1990s.

4.4 Future application areas

The 1990s is neither discontinuous with the

1980s nor with the 21st century. Thus, consideratian
of the 21st century will be important in determining
what is necessary in the 1990s. Almost all problems
facing the world today will worsen in the coming
century if steps are not taken to solve them now.
These problems include environmental damage,
population explosion, natural resource shortages, and
natural disasters. In all countries, it+ also important
to cope with increasing social difficulties. Of course,
these are political, economic, and social problems,
and machine vision can only make a small

5 Conclusions

In this paper, the past history of the machine

vision research has been reviewed, and its state-ofthe-art has been evaluated, stressing the practical
standpoint. In the past, the main effort was focussed
on how quickly the processing can be completed,
which resulted in binary image processing with rather
simplified, local-parallel methods. Machine vision
gradually progressed to gray-scale image processing
as image processing LSIs became available.
Nowadays, the impetus is towards knowledge-based

xxm

processing. However, the heart of the industrial

machine vision is its speed, and in general knowledgebased processing is still too slow. Therefore, it is
necessary to adopt a simplified method where the
knowledge is compared with the observed facts, and
when matched, a certain decision is made and a certain
operation is added to the observed facts. This
knowledge-driven processing seems to be promising,
as shown by the wafer inspection machine. This
machine is a typical "knowledge-directed'' inspection
machine where the knowledge-driven aspects are
implemented in hardware. In the future, machine
vision may be further improved by this extensive
utilization of knowledge.
Some new application areas have also been
discussed in this paper. However, futurology is always
vague and involves many pros and cons. Like other
engineering problems, machine vision will also be
expected to somehow contribute to the prosperity of
future human society and the conservation of its
natural environment. A few such contributions are
conceivable. Among them are technologies to
alleviate disasters, to increase human happiness by
providing a comfortable living environment and the
stable supply of essential commodities. We must
occasionally think of these factors when researching
machine vision technology.

References
M. Ejiri, et al. : A prototype intelligent robot that
assembles objects from plan drawings, IEEE Trans.
Computer, C-21, 2, pp.161-170 (1972)
M. Ejiri : Industrial Image Processing, Shokodo Pub.,
Tokyo (1988) (in Japanese)
M. Ejiri : Machine Vision: A Practical Technology for
Advanced Image Processing, Gordon & Breach Science
Publishers, New York (1989)
M. Ejiri, Y. Nakano and T.Chusho : Artificial
Intelligence, Shokodo Pub., Tokyo (1988) (in Japanese)
M. Ejiri et al. : A process for detecting defects in
complicated pattems, Computer Graphics and Image
Processing, 2, pp.326-339 (1973)
S.Kashioka, M.Ejiri and Y.Sakamoto, A Transistor wirebonding system utilizing multiple local pattem matching
techniques, IEEE Trans. on Syst. Man & Cybern., SMC6, 8. pp.562-570 (1976)
T.Uno, M.Ejiri and T.Tokunaga, A method of real-time
recognition of moving objects and its application, Pattem
Recognition, 8, 4, pp.201-208 (1976)
H. Yoda, S . Ikeda and M. Ejiri : Hitachi Review, 22,9,
pp.362-365 (1973). Also in Trans. SICE, 10, 3, pp.284289 (1974) (in Japanese)
G. J. Agin : Computer vision systems for industrial
inspection and assembly, IEEE Computers, 13, 5, pp.1120 (1980)
[ 101 H. Sakouand M. Ejiri : Image understanding technologies

in industrial applications, Proc. Japan Society of Artificial

Intelligence, 4, 1, pp.43-51 (1989)
[ 111 S.Kashioka, Y.Shima and M.Ejin, Automatic template
selection technique for the local pattern matching method,
Trans. IEICE, J68-D, 5 , pp.1103-1110 (1985) (in
Japanese)
[ 121 H. Sakou, T. Miyatake, S . Kashioka and M. Ejiri : A
position recognition algorithm for semiconductor
alignment based on snuctural pattem matching, IEEE
Trans. ASSP, 37, 12, pp.2148-2157 (1989)
[ 131 H.Sakou. H.Yoda, M.Ejiri and O.Obata, An algorithm for
inspecting radiation pattems of infrared light emitting
diodes, Trans. IEEJ, 107-C, 8, pp.737-744
(1987) (in
..
Japanese)
[ 141 K.Matsuzaki, M.Isobe, S.Hata and K.Takeichi, Objectoriented industrial vision system, Trans. Information
Processing Society of Japan, 29, 12, pp.1119-1128
(1988)
[ 151 S.Sawano, J.Ikeda, N.Utsumi, H.Kiba, Y.Ohtani and
A.Kikuchi, A sealing robot system with visual seam
trackirk, Proc. Int. Conf. on Advanced Robotics, pp.35 1
358, TGkyo (1983)
[ 161 Y.Nakagawa, Y.Oshida, T.Ninomiya and H.Sakai, Shape
detection of solder joints by spotlight scanning lightsection method, Trans. SICE, 22,9, pp.982-987
(1986)
..
(in Japanese)
[ 171 Y.Takagi, D.Katsuta, S.Hata, T.Ohi, K.Murata and
T.Kobavashi. Visual insDection machine for chip-mounted
boards Gsing'slit-light mkthod, Tech. Report of image
Processing Application Committee, Japan Society of
Precision Engineers, 3, 1, pp.1-I1 (1988)
[ 181 M. Ejiri : Knowledge-based approaches to practical image
processing, Int. Workshop on Industrial Applications of
Machine Intelligence and Vision (MIV '89) Tokyo (April
1989)
[ 191 M. Ejiri, H. Yoda, H. Sakou and Y. Sakamoto :
Knowledge-directed inspection for complex multilayered
pattems, Machine Vision and Applications, 2, pp. 155- 166
(1989)
[20] H. Yoda, Y. Ohuchi, Y. Taniguchi and M. Ejiri : An
automatic wafer inspection system using pipelined image
processing techniques, IEEE Trans. PAMI, 10, I , pp.416 (1988)
[21J US-Japan Manufacturing Research Exchange, National
Academy Press (1989)
[22] H. Yoda : An algorithm for determining shape descriptors
based on video-rate one-pass image processing
techniques, Trans. IEICE, J72-D-I1,7, pp. 1017-1028
(1989) (in Japanese)
[23] H. Sakou, H. Yoda and M. Ejiri : An algorithm for
matching distorted waveforms using a scale-based
description, IAPR Workshop on Computer Vision, Tokyo
(Oct. 1988)
[24] H. Sakou, H. Matsushima and M. Ejiri : Texture
discrimination using self-organized multi-resolution
filtering, Trans. IEICE, J73-D-I1,4, pp.562-573 (1990)
(in Japanese)
[25] T. Unnai, N.Goto and Y. Takasaki : Newly developed
high-sensitivity HDTV camera tube, Hitachi Review, 38,
2, pp.121-126 (1989)
[26] M. Yasunaga, N. Masuda, M. Asai, M. Yamada, A.
Masaki and Y. Hirai : A wafer scale integration neural
network utilizing completely digital circuits, Proc.
IJCNN, Vol.11, pp.213-217 (June 1989)

xxxx

~~