Objective of Computer Vision

The field of computer vision can

be divided into two areas
Image enhancement
Image analysis

Here we concentrate on fast methods typical for robot soccer and robot
theatre applications
Binary image processing
Image with two gray levels 0 and 1
It contains the basic terms and concepts
used in machine vision.
Its techniques are used in all aspects of a
vision system.
Small memory requirements
Fast execution time








Gray level image vs. binary image

Image Enhancement

Point Operations

Region and segmentation
Region ( )
A subset of an image
Segmentation
Grouping of pixels into regions such that






Thresholding
Thresholding
Thresholding :
A method to convert a gray scale image into a binary
image for object-background separation
: Thresholded gray image
Obtained using a threshold T for the original gray
image
.
: Binary image
Equivalent to .
Three types of thresholding








where Z is a set of which elements are
integer-valued intervals.

Original image and its histogram
Thresholding

H
Histogram Equalization

Vision System
of Soccer Robot
Image notation for soccer
Image : a two-dimensional array of pixels

Pixel a[i, j]
Geometric properties
In many cases, some simple features of
regions are useful to:
determine the locations of objects,
and to recognize objects.
Geometric properties:
Size
Position
Orientation
Size and position
Given an m x n binary image,
Size (area) A : zero-order moment


Position : the center of area


Average in x
Average in y
Total size is number of
black dots
Example
The center of object :
Let



By the least-squares fit of the line,


Step 1: calculate a,b,c coefficients
Step 2: calculate angle, line orientation
How to calculate line orientation?
Size filter for Noise removal
It can effectively remove noise after component algorithm
labeling.
If objects of interest have sizes greater than T, all
components below T

are removed by changing the
corresponding pixels values to 0.
A noisy binary image and the resulting image after size filtering (T = 10)
How to get
the position and angle of robot
Get frame-grabber, color CCD camera and computer.
Understand how you can read the image data from frame-
grabber.
Find the position of a colored object in 2-D image.
Determine the robot uniform with two colored objects
Calculate the position and angle of the robot from the
positions of two colored objects.
Position of colored object
1. Setting of ranges for YUV
[Y
min
,Y
max
], [U
min
,U
max
], [V
min
, V
max
]
2. Thresholding
3. Labeling (grouping)
4. Size filtering (noise elimination)
5. Finding the center of a colored object


Calculations for soccer
field
Finding robots in the field
Robot position and heading
This cross sign is
easy to recognize
Robot color and team color
This slide explains labeling robots and teams
Window tracking method
Processing only the data within a small window
Getting a fast vision processing
Line Orientation
Orientation from of the axis of elongation
This is called line orientation
Line equation :
: the minimum distance between the line and origin
: the angle from x-axis to the line
The distance, d, from any (x, y) within the object to the line :
which satisfies


Minimize



Our task is to find values of angle theta and rho for which this formula is minimum.
This provides best fit to line equation
The center of object :
Let


Given is object. Find its line orientation
1. Calculate center
2. Calculate a, b, and c.
By the least-squares fit of the line,




1. Calculate center
2. Calculate a, b, and c.
3. Calculate theta
Binary algorithms
Several definitions
Neighbors
4-neighbors (4-connected)



8-neighbors (8-connected)
Path
A sequence of neighbors
Foreground :
The set of all unity valued pixels in an image
Connectivity
A pixel is said to be connected to if there is a
path from to consisting entirely of pixels of .
Connected components
A set of pixels in which each pixel is connected to all other
pixels.
Component labeling
Component labeling algorithm

It finds all connected components in an image and assigns
a unique label to all the points in a component.

One of the most common operations in machine vision

Recursive connected components algorithm

Sequential connected components algorithm

The points in a connected component form a candidate region
for an object.

An image and its connected component image
Recursive algorithm for connected
component labeling
Recursive connected components algorithm

1. Scan the image to find an unlabeled unity valued pixel
and assign it a new label L.

2. Recursively assign a label L to all its unity valued
neighbors.

3. Stop if there are no more unlabeled unity valued
pixels.

4. Go to step 1.
Pseudo code for the recursive
algorithm
Sequential algorithm for
connected component labeling
Sequential connected components algorithm using 4-connectivity
1. Scan the image from left to right and top to bottom.

2. If the pixel is unity valued , then
(a) If only one of its upper or left neighbors has a label, then copy
the label.

(b) If both have the same label, then copy the same label.

(c) If both have different labels, then copy the upper pixels label
and enter the labels in an equivalence table as equivalent
labels.


(d) Otherwise assign a new label to this pixel and
enter this label in the equivalence table.

3. If there are more pixels to consider, then go to step 2.

4. Find the lowest label for each equivalent set in the
equivalence table.

5. Scan the picture. Replace each label by the lowest
label in its equivalent set.
Sequential algorithm for connected component labeling
continued
Pseudo code for step 2 in the sequential
algorithm







An example for the sequential algorithm
Digital Geometry
Neighborhood
Connectedness
Distance Metrics
Picture Element or Pixel
Pixel value I(I,j) =
0,1 Binary Image
0 - K-1 Gray Scale Image
Vector: Multispectral Image
32
I(i,j)
(0,0)
i
j
Connected Components
Binary image with multiple 'objects'
Separate 'objects' must be labeled
individually
6 Connected Components
Finding Connected Components
Two points in an image are 'connected' if a path
can be found for which the value of the image
function is the same all along the path.
P1
P2
P3
P4
P
1

connected to
P
2

P
3

connected to
P
4

P
1

not connected to
P
3
or P
4

P
2

not connected to
P
3
or P
4

P
3

not connected to
P
1
or P
2

P
4

not connected to
P
1
or P
2

Algorithm
Pick any pixel in the image and assign it a label
Assign same label to any neighbor pixel with the
same value of the image function
Continue labeling neighbors until no neighbors
can be assigned this label
Choose another label and another pixel not
already labeled and continue
If no more unlabeled image points, stop.
Who's my neighbor?
Example
Neighbor
Consider the definition of the term 'neighbor'
Two common definitions:
Consider what happens with a closed curve.
One would expect a closed curve to partition the plane into
two connected regions.
Four Neighbor Eight Neighbor
Alternate Neighborhood Definitions
Neither neighborhood definition satisfactory!
4-neighbor
connectedness
8-neighbor
connectedness
Possible Solutions
Use 4-neighborhood for object and 8-
neighborhood for background
requires a-priori knowledge about which pixels are
object and which are background
Use a six-connected neighborhood:
Digital Distances
Alternate distance metrics for digital images
Euclidean Distance City Block Distance Chessboard Distance
= (i-n)
2
+ (j-m)
2
= |i-n| + |j-m|
= max[ |i-n|, |j-m| ]
Connected Components /
Image Labeling
Goal: To find clusters of pixels that are similar and
connected to each other
How it works:
Assign a value to each pixel
Define what similar values mean
e.g., 10 +/- 2
Determine if like pixels are connected


Connected Components /
Image Labeling
4- connected 8-connected
Component Labeling
Definitions
Neighbors
4-neighbors (4-connected)

8-neighbors (8-connected)



Connected components
A set of pixels in which each pixel is connected to all other
pixels.
An image and its connected
component image
Connected Components /
Image Labeling
1 1 1 1 1 1
1 0 0 1 1 1
1 1 1 0 1 1
1 2 2 0 0 1
1 2 2 0 0 1
A A A A A A
A B B A A A
A A A C A A
A D D C C A
A D D C C A
Connected Components /
Image Labeling
1 1 1 1 1 1
1 0 0 1 1 1
1 1 1 0 1 1
1 2 2 0 0 1
1 2 2 0 0 1
A A A A A A
A B B A A A
A A A B A A
A C C B B A
A C C B B A
After thresholding an image, we want to know something about the
regions found ...
Binary Image Processing
How many objects are in the image?
Where are the distinct object components?
Cleaning up a binary image?
Recognizing objects through their response to image masks
Describing the shape/structure of 2d objects
Pattern
Counting Objects
external corners internal corners
1
0 1
1 1
1 0
1 1
1 1
0 0
1 1
1 0
0 0
1 1
0 0
0 0
1 0
0 0
0 1
0
Connected Component Labeling
1
0
0
1
2
1
0
0
1
2
Algorithm
1. Image is A. Let A = -A;
2. Start in upper left and work L to
R, Top to Bottom, looking for an
unprocessed (-1) pixel.
3. When one is found, change its
label to the next unused integer.
Relabel all of that pixels
unprocessed neighbors and their
neighbors recursively.
4. When there are no more
unprocessed neighbors, resume
searching at step 2 -- but do so where
you left off the last time.
Connected Component Labeling
Summary
A simple procedure to get robot position and heading:
Thresholding,
labeling,
size filtering,
center of area calculation,
window tracking method
Trade-off between:
real-time vision processing
and robustness
Specific frame grabber required
Problems for students
1. Binary image processing and basic operations on binary
images.
2. List and explain briefly operations used in image enhancement
3. What are point operations. Give examples.
4. Explain contrast stretching operation. Write simple program.
5. Definition of image segmentation.
6. Segmentation based on thresholding
7. Types of thresholding. Give examples of each with practical
applications.
8. The general idea of histogram equalization. Show examples.
9. Geometric positions of objects in robot soccer.
10. Size and position of objects in robot soccer.
Problems for students
11. How to calculate quickly the center of the mass of an
object?
12. How to calculate line orientation?
13. Describe and program size filter for noise removal.
Emphasize speed not quality.
14. How to calculate position and angle of a robot using the
ceiling camera?
15. Use of colors in vision of robot soccer.
16. Window tracking methods in robot soccer.
17. Use of Line orientation in robot soccer.
Problems for students
18. Binary image processing algorithms in robot soccer.
19. Component labeling
20. Recursive connected components algorithm
21. Sequential connected components algorithm
22. Algorithms to find connected components.
23. Alternate definitions of neighbors in connected
components algorithms. Applications in robot soccer.
24. The concept of digital distance and its use.
25. Connected component labeling in binary images.
26. Counting objects. What are applications in robot soccer?