CHAPTER 1

Role of Artificial Intelligence and Image

Processing in Computer Vision
Industrial Machine Vision Applications
System Architecture
State of Art

Introduction
What is computer vision?
The analysis of digital images by a computer

The Three Stages of Computer Vision

Low-Level

sharpening

blurring
7

Low-Level

Canny

original image

edge image

Mid-Level

ORT

data
structure
circular arcs and line segments
8
edge image

Mid-level

K-means
clustering
(followed by
connected
component
analysis)

regions of homogeneous color

original color image

data
structure

Low- to High-Level

low-level
edge image

mid-level

high-level

consistent
line clusters

Building Recognition
10

System Architecture

Typical Computer Vision System. The figure shows the basic

components of a computer vision system. The left most component is
the scene or object of study. The next required component shown in
the figure is the sensing device used to collect data from the scene.
The third component is the computation device. The device computes
information such as visual cues and reasons on this information to
generate interpretations of the scene such as objects present or
actions being performed.

System Architecture
Typical computer vision system
involves three components,
1. Scene under study
2. Sensing device that can be used to
analyze the scene (used to collect
data from the scene)

System Architecture
3.

Computational device that can perform the analysis

of the scene based on the data from the sensor. The
computation devices generates two possible forms of
data, information such as visual cues, and interpretations
of information such as actions being performed or the
presence of objects. The two forms of data can each be
used to refine the other until the output of the vision
system is computed with a predefined amount of
certainty. The result can be the 3D location for every pixel
within and image or the certainty that a person is
performing jumping jacks. With proper selection of the
information and the interpretation algorithm, computer
vision systems can be applied in a large number of
applications.

Digital Image Terminology:

0
0
0
0
0
0
0

0
0
1
0
0
1
0

0
1
95
92
93
92
94

0
1
96
93
93
93
95

1
1
94
93
94
93
95

0
0
93
92
92
93
96

0
0
92
92
93
93
95

pixel (with value 94)

its 3x3 neighborhood
region of medium
intensity
resolution (7x7)

binary image
gray-scale (or gray-tone) image
color image
multi-spectral image
range image
labeled image
14

Spatial Filtering

Image filtering

Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

Slides from Steve Seitz and Rick Szeliski

Image filtering

Hybrid Images, Oliva et al., http://cvcl.mit.edu/hybridimage.htm

What is an image?

Images as functions
We can think of an image as a function,
f, from R2 to R:
f( x, y ) gives the intensity at position ( x, y )
Realistically, we expect the image only to
be defined over a rectangle, with a finite
range:
f: [a,b]x[c,d] [0,1]

Images as functions
A color image is just three functions pasted
together. We can write this as a vector-valued
function:

r ( x, y )

f ( x, y ) g ( x, y )

b( x, y )

Images as functions

What is a digital image?

We usually work with digital
(discrete) images:
Sample the 2D space on a regular grid
Quantize each sample (round to nearest
integer)

If our samples are apart, we can write

this as:
f[i ,j] = Quantize{ f(i , j ) }

Spatial Filtering (contd)

The word filtering has been borrowed
from the frequency domain.
Filters are classified as:
Low-pass (i.e., preserve low frequencies)
High-pass (i.e., preserve high frequencies)
Band-pass (i.e., preserve frequencies within
a band)
Band-reject (i.e., reject frequencies within a
band)

Spatial Filtering Neighborhood (or

Mask)
Typically, the neighborhood is rectangular and its size
is much smaller than that of f(x,y)
- e.g., 3x3 or 5x5

Filtering noise
How can we smooth away noise in
an image?
0

100

130

110

120

110

110

90

100

90

100

130

100

90

130

110

120

100

130

110

120

90

110

80

120

100

Mean filtering

Mean filtering
0

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

10

20

30

30

30

20

10

20

40

60

60

60

40

20

30

60

90

90

90

60

30

30

50

80

80

90

60

30

30

50

80

80

90

60

30

20

30

50

50

60

40

20

10

20

30

30

30

30

20

10

Cross-correlation filtering
Lets write this down as an equation. Assume
the averaging window is (2k+1)x(2k+1):

We can generalize this idea by allowing different weights

for different neighboring pixels:

This is called a cross-correlation operation and

written:
H is called the filter, kernel, or mask.
The above allows negative filter indices. When you
implement need to use: H[u+k,v+k] instead of H[u,v]

Mean kernel
Whats the kernel for a 3x3 mean
filter?
0

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

90

Mean of neighborhood

Effect of filter size

Mean vs. Gaussian filtering

Gaussian filtering
A Gaussian kernel gives less weight
to pixels further from the center of
the window

Linear vs Non-Linear
Spatial Filtering Methods
A filtering method is linear when the output is a
weighted sum of the input pixels.

Methods that do not satisfy the above property are

called non-linear.
e.g.,

Linear Spatial Filtering

Methods
Main linear spatial filtering methods:
Correlation
Convolution

Correlation (contd)
Often used in applications
where we need to measure
the
similarity
between
images or parts of images
(e.g., template matching).

Convolution
Similar to correlation except that the mask is
first flipped both horizontally and vertically.

g (i, j ) w(i, j ) f (i, j )

K /2

K /2

w( s, t ) f (i s, j t )

s K /2 t K / 2

Note: if w(i, j) is symmetric, that is w(i, j)=w(i,-j), then convolution is equivalent to

correlation!

Image gradient
How can we differentiate a digital
image F[x,y]?
Option 1: reconstruct a continuous
image, f, then take gradient
Option 2: take discrete derivative (finite
difference)

Image gradient

It points in the direction of most rapid change in

intensity

It points in the direction of most rapid change in

intensity

original

smoothed (5x5 Gaussian)

Why does
this work?

smoothed original
(scaled by 4, offset +128)

Gaussian

delta function

Laplacian of Gaussian
(2nd derivative operator)

More on filters
Cross-correlation/convolution is useful for, e.g.,
Blurring
Sharpening
Edge Detection
Interpolation
Convolution has a number of nice properties
Commutative, associative
Convolution corresponds to product in the Fourier domain
More sophisticated filtering techniques can often yield superior results for
these and other tasks:
Polynomial (e.g., bicubic) filters
Steerable filters
Median filters
Bilateral Filters

Filters
We will mainly focus on two types of
filters:
Smoothing (low-pass)
Sharpening (high-pass)

Smoothing Filters (low-pass)

Useful for reducing noise and eliminating small
details.
The elements of the mask must be positive.
Sum of mask elements is 1 (after normalization).

Gaussian

Smoothing filters Example

Useful for reducing noise and
eliminating small details.
input image

smoothed image

Sharpening Filters (highpass)

Useful for highlighting fine details.
The elements of the mask contain
both positive and negative
weights.
Sum of mask elements is 0.
1st derivative
of Gaussian

2nd derivative
of Gaussian

Sharpening Filters (contd)

Useful for highlighting fine details.
e.g., emphasize edges

Sharpening Filters - Example

Note that the response of sharpening might be
negative.
Values must be re-mapped to [0, 255]
input image

sharpened image

Smoothing Filters
Averaging
Gaussian
Median filtering (non-linear)

Smoothing Filters: Averaging

Smoothing Filters: Averaging

(contd)
Mask size determines the degree of smoothing
(loss of detail).
original

3x3

15x15

5x5

25x25

7x7

Smoothing Filters: Averaging

(contd)
Example: extract largest, brightest objects

15 x 15 averaging

image thresholding

Smoothing filters: Gaussian

The weights are Gaussian samples:

= 1.4

mask size is
a function of :

Smoothing filters: Gaussian (contd)

controls the amount of smoothing
As increases, more samples must be obtained to represent
the Gaussian function accurately.

=3

Smoothing filters: Gaussian (contd)

Averaging vs Gaussian Smoothing

Averaging

Gaussian

Smoothing Filters: Median Filtering

(non-linear)
Very effective for removing salt and pepper noise
(i.e., random occurrences of black and white
pixels).

averaging

median
filtering

Smoothing Filters: Median Filtering

(contd)
Replace each pixel by the median
in a neighborhood around the
pixel.

Sharpening Filters

Unsharp masking
High Boost filter
Gradient (1st derivative)
Laplacian (2nd derivative)

Sharpening Filters: Unsharp Masking

Obtain a sharp image by
subtracting a lowpass filtered (i.e.,
smoothed) image from the original
image:

=
(after contrast
enhancement)

Sharpening Filters: High

Boost
Image sharpening emphasizes edges but low
frequency components are lost.
High boost filter: amplify input image, then
subtract a lowpass image.

(A-1)

Sharpening Filters: High Boost

(contd)
If A=1, we get unsharp masking.
If A>1, part of the original image is added
back to the high pass filtered image.
One way to implement high boost filtering is
using the masks below:

Sharpening Filters: High Boost (contd)

A=1.4

A=1.9

Sharpening Filters:
Derivatives
Taking the derivative of an image
results in sharpening the image.
The derivative of an image (i.e., 2D
signal) can be computed using the
gradient.

Laplacian
The Laplacian (2nd derivative) is defined as:

(dot product)

Approximate
2nd derivatives:

Laplacian (contd)
Laplacian Mask

Edges can be found

by detect the zero-crossings

Example: Laplacian vs
Gradient
Laplacian

Sobel

FOV (Field of View)

FOV (Field of View)

FOV (Field of View)