Chapter 6

These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Image Segmentation
 Chapter 6
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

REFERENCES

Slides by Brian Mac Namee

Brian.MacNamee@comp.dit.ie
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
 Thresholding
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
 But Be Careful
If you get the threshold wrong the results can be disastrous

Threshold Too Low Threshold Too High

Thresholding Example 1
Thresholding Example 2
 Problems With Single Value Thresholding

Single value thresholding only works for bimodal histograms

Images with other kinds of histograms need more than a single
threshold
 Thresholding

T1
Thresholding
Thresholding
 Problems With Single Value Thresholding

Let’s say we want to

isolate the contents
of the bottles
Think about what the
histogram for this
image would look like
What would happen if we used a single threshold value?
 Single Value Thresholding and Illumination

Uneven illumination can really upset a single valued

thresholding scheme
 Basic Global Thresholding

Based on the histogram of an image

Partition the image histogram using a single global
threshold
The success of this technique very strongly depends on
how well the histogram can be partitioned
 Basic Global Thresholding Algorithm
The basic global threshold, T, is calculated
as follows:
1. Select an initial estimate for T (typically the average grey
level in the image)
2. Segment the image using T to produce two groups of pixels:
G1 consisting of pixels with grey levels >T and G2 consisting
pixels with grey levels ≤ T
3. Compute the average grey levels of pixels in G1 to give μ1
and G2 to give μ2
 Basic Global Thresholding Algorithm
4. Compute a new threshold value:
μ1 + μ2
T=
2
5. Repeat steps 2 – 4 until the difference in T in successive
iterations is less than a predefined limit T
This algorithm works very well for finding thresholds when
the histogram is suitable
 Basic Adaptive Thresholding

An approach to handling situations in which single value

thresholding will not work is to divide an image into sub
images and threshold these individually
Since the threshold for each pixel depends on its location
within an image this technique is said to adaptive
Thresholding Based on Boundary
 Why Region-Based Segmentation?


Segmentation
 Edge detection and Thresholding
not always effective.

Homogenous regions
 Region-based segmentation.
 Effective in noisy images.
 Definations


Based on sets.

Each image R is a set of R7
R6
regions Ri.
 Every pixel belongs to one R1
R5
region.
 One pixel can only belong to a R2 R3

single region. R4

S
R i=1
= Ri Ri intersect R j =∅
 Definations

R7
R6

R1
R5

R2 R3

R4
 Basic Formulation

Let R represent the entire image region.

Segmentation partitions R into n subregions, R1,
R2, ..., Rn, such that:

a) n R i=R
i=1 a) Every pixel must be in a region

b) Ri is a connected region, i =1 , 2 , . .. , n . b) Points in a region must be connected.

c) Ri ∩ R j=φ for all i and j , i≠ j c) Regions must be disjoint.

d) P ( Ri ) =TRUE for i =1,2 ,. .. , n . d) All pixels in a region satisfy specific properties.

e) Different regions have different properties.

e) P ( Ri ∪ R j )=FALSE for i ≠ j .
 Basic Formulation


Groups pixels into larger 
Iterative process
regions.  How to start?

Starts with a seed region.  How to iterate?

Grows region by merging
 When to stop?
neighboring pixels.
Finish

Initial Stop
Iterations
Regions Condition
Region Growing
 Region Merging


Algorithm
 Divide image into an initial set of regions.

One region per pixel.
 Define a similarity criteria for merging regions.
 Merge similar regions.
 Repeat previous step until no more merge operations are
possible.
 Similarity Criteria


Homogeneity of regions is used as the main
segmentation criterion in region growing.
 gray level
 color, texture
 shape Choice of criteria affects
segmentation results
 model dramatically!
 etc.
 Gray-Level Criteria


Comparing to Original Seed Pixel
 Very sensitive to choice of seed point.

Comparing to Neighbor in Region
 Allows gradual changes in the region.
 Can cause significant drift.

Comparing to Region Statistics
 Acts as a drift dampener.

Other possibilities!
 Region Splitting


Algorithm
R1 R2
 One initial set that includes the R1
whole image. R3 R4
 Similarity criteria.
 Iteratively split regions into
sub-regions.
 Stop when no more splittings R1 R2
R1 R2 R3
are possible.
R4 R5
R3
R4 R5 R6 R7 R6 R7
 Splitting and Merge


Combination of both
algorithms.

Can handle a larger variety of
shapes.
 Simply apply previous
algorithms consecutively.
 Detection Of Discontinuities
There are three basic types of grey level discontinuities that
we tend to look for in digital images:
– Points
– Lines
– Edges
We typically find discontinuities using masks and correlation
Detection Of Discontinuities
Detection Of Discontinuities
 Point Detection
Point detection can be achieved simply using the mask
below:

Points are detected at those pixels in the subsequent

filtered image that are above a set threshold
 Point Detection

X-ray image of a Result of point Result of

turbine blade detection thresholding
 Line Detection
The next level of complexity is to try to detect lines
The masks below will extract lines that are one pixel thick
and running in a particular direction
Line Detection
 Line Detection
Binary image of a wire bond mask

After processing
Result of
with -45° line
thresholding
detector
filtering result
 Edge
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Edges are significant local changes of intensity in a digital image.

An edge can be defined as a set of connected pixels that forms a
boundary between two disjoint regions. There are three types of edges:

Horizontal edges

Vertical edges

Diagonal edges
 Edge Detection

• Convert a 2D image into a set of curves

– Extracts salient features of the scene
– More compact than pixels
 Origin of Edges

surface normal discontinuity

depth discontinuity

surface color discontinuity

illumination discontinuity

• Edges are caused by a variety of factors

 Edge Types

Step Edges

Roof Edge Line Edges

 Real Edges

Noisy and Discrete!

We want an Edge Operator that produces:

– Edge Magnitude
– Edge Orientation
– High Detection Rate and Good Localization
 Edge Detection
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Edge Detection is a method of segmenting an image into regions of

discontinuity.
It is a widely used technique in digital image processing like

pattern recognition

image morphology

feature extraction
 Edge Detection
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Edge detection is a fundamental tool in image processing,

machine vision and computer vision, particularly in the areas
of feature detection and feature extraction.
The purpose of detecting sharp changes in image brightness
is to capture important changes in the image.
Image filters are commonly used for edge detection.
 Edge Detection
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

The process of edge detection consists of three main steps:

- Noise reduction
- Detection of edge points
- Edge localization
 Edge Detection
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari


First-order derivatives generally produce thicker edges in image


Second-order derivatives have a stronger response to fine detail,
such as thin lines, isolated points, and noise
 Gradient
• Gradient equation:

• Represents direction of most rapid change in intensity

• Gradient direction:

• The edge strength is given by the gradient magnitude

 Discrete Edge Operator
• How can we differentiate a discrete image?

Finite difference approximations:

∂I I i , j+1 I i+1 , j +1
≈ ( ( I i +1 , j+1 − I i , j+ 1) + ( I i+ 1, j − I i , j ) )
∂x
∂I Ii, j I i+1 , j
≈ ( ( I i+1 , j +1 − I i +1 , j ) + ( I i , j +1 − I i , j ) )
∂y

Convolution masks :

∂I −1 1 ∂I 1 1
∂x −1 1 ∂y −1 −1
 Discrete Edge Operator
• Second order partial derivatives: I i− 1 , j+1 I i , j+1 I i+1 , j +1
∂2 I I i− 1 , j Ii, j I i+1 , j
2
≈ ( I i , j −1 −2 I i , j + I i , j+1 )
∂y
∂2 I I i− 1 , j −1 I i , j −1 I i+1 , j −1
2
≈ ( I i −1 , j − 2 I i , j + I i +1 , j )
∂x
• Laplacian :
2 ∂2 I ∂2 I
∇ I= 2
+ 2
∂x ∂ y

Convolution masks :

0 1 0 1 4 1
2
∇ I≈
1 −4 1 4 −20 4

0 1 0 1 4 1
 Image Edge Detection Operators in Digital Image Processing

Edge Detection Operators are of two types:

1. Gradient based operator
- computes first-order derivations in a digital image like Sobel
operator, Prewitt operator, Robert operator
2. Gaussian based operator
- computes second-order derivations in a digital image like, Canny
edge detector, Laplacian of Gaussian
 Image Edge Detection Operators in Digital Image Processing
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari
 Sobel Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

It is a discrete differentiation operator.

It computes the gradient approximation of image intensity function for
image edge detection.
At the pixels of an image, the Sobel operator produces either the
normal to a vector or the corresponding gradient vector.
 Sobel Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

It uses two 3 x 3 kernels or masks which are convolved with the input
image to calculate the vertical and horizontal derivative approximations
respectively
 Sobel Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Advantages:

Simple and time efficient computation

Very easy at searching for smooth edges
Limitations:

Diagonal direction points are not preserved always

Highly sensitive to noise

Not very accurate in edge detection

Detect with thick and rough edges does not give appropriate results
 Prewitt Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari


This operator is almost similar to the sobel operator.

It also detects vertical and horizontal edges of an image.

It is one of the best ways to detect the orientation and magnitude of an
image. It uses the kernels or masks
 Prewitt Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Advantages:

Good performance on detecting vertical and horizontal edges

Best operator to detect the orientation of an image

Limitations:

The magnitude of coefficient is fixed and cannot be changed

Diagonal direction points are not preserved always
 Robert Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

This gradient-based operator computes the sum of squares of the

differences between diagonally adjacent pixels in an image through
discrete differentiation.
Then the gradient approximation is made. It uses the following 2 x 2
kernels or masks
 Robert Operator
These slides should not be used as the primary source of data. Students are encouraged to learn from the core textbooks and reference books. Contents in these slides are copyrighted to the instructor and authors of
original texts where applicable. -Mohan Bhandari

Advantages

Detection of edges and orientation are very easy

Diagonal direction points are preserved

Limitations

Very sensitive to noise

Not very accurate in edge detection
 Comparision of Operators
Good Localization
Gradient:
Noise Sensitive
Poor Detection
Roberts (2 x 2): 0 1 1 0
-1 0 0 -1

Sobel (3 x 3):
-1 0 1 1 1 1
-1 0 1 0 0 0
-1 0 1 -1 -1 1
Sobel (5 x 5):
-1 -2 0 2 1 1 2 3 2 1
-2 -3 0 3 2 2 3 5 3 2
Poor Localization
-3 -5 0 5 3 0 0 0 0 0
Less Noise Sensitive
-2 -3 0 3 2 -2 -3 -5 -3 -2 Good Detection
-1 -2 0 2 1 -1 -2 -3 -2 -1
Edge Detection Example
 Laplace Operator

Problems with Laplacian:

It generates “double edges”, i.e., positive and negative values for each edge.
It is extremely sensitive to noise.
 Laplacian and Zero Cross

Laplacian operator seeks out points in the

signal stream where the digital signal of an
image passes through a pre-set '0' value,
and marks this out as a potential edge
point.

Because the signal has crossed through

the point of zero, it is called a zero-crossing
 Laplacian of Gaussian


Works by smoothing the image with a Gaussian low-pass
filter, and then applying a Laplacian edge detector to the
result.

The LoG filter can sometimes be approximated by taking
the differences of two Gaussians of different widths, in a
method known as Difference of Gaussians (DoG).
 Laplacian of Gaussian

It is a Gaussian-based operator which uses the Laplacian to take the

second derivative of an image.
This really works well when the transition of the gray level seems to be
abrupt.
It works on the zero-crossing method i.e when the second-order
derivative crosses zero, then that particular location corresponds to a
maximum level. It is called an edge location. The Gaussian operator
reduces the noise and the Laplacian operator detects the sharp edges.
 Laplacian of Gaussian

The Laplacian of Gaussian of the image is defined by the formula:

with g the Gaussian kernel and * the convolution

This convolution can be further expanded, in the 2D case, as

 Laplacian of Gaussian

Laplacian318
of Gaussian (LoG) transfer function (Mexican hat)
EDGE DETECTION

(a) (b)
 Effect of Noise
• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal

Where is the edge??

 Solution

Where is the edge? Look for peaks in

Derivative Theorem of Convolution
…saves us one operation.
 Derivative Theorem of Convolution
2 2
∂ ∂
∂x
2
( h ∗ f
(
) =
∂x
2)h ∗f Laplacian of Gaussian

Laplacian of Gaussian (LoG)

Laplacian of Gaussian operator

Where is the edge? Zero-crossings of bottom graph !

 Laplacian of Gaussian

Advantages:

Easy to detect edges and their various orientations

There is fixed characteristics in all directions
Limitations:

Very sensitive to noise

The localization error may be severe at curved edges

It generates noisy responses that do not correspond to edges, so-called “false edges”
 Difference of Gaussian

The difference of Gaussian (DoG) of image can be written as

 Canny Edge Detection

It is a gaussian-based operator in detecting edges. This operator is not susceptible to

noise. It extracts image features without affecting or altering the feature.

Canny edge detector have advanced algorithm derived from the previous work of
Laplacian of Gaussian operator. It is widely used as an optimal edge detection technique.
It detects edges based on three criteria:

Low error rate

Edge points must be accurately localized

There should be just one single edge response
 Canny Edge Operator

• Smooth image I with 2D Gaussian: G∗I

• Find local edge normal directions for each pixel

∇ (G ∗ I )
n̄=
|∇ ( G ∗ I )|

• Compute edge magnitudes |∇ ( G ∗ I )|

• Locate edges by finding zero-crossings along the edge normal directions (non-maximum
suppression) 2
∂ (G ∗ I )
2
=0
∂ n̄
 Canny Edge Operator

• Check if pixel is local maximum along gradient direction

– requires checking interpolated pixels p and r
 Canny Edge Operator

Lenna Magnitude of Gradient

Edge
 Edge linking and boundary detection

Goal of edge detection: to produce an image containing only the

edges of the original image.
However, due to the many technical challenges (noise, shadows,
occlusion, etc,), most edge detection algorithms will output an
image containing fragmented edges.
Additional processing is needed to turn fragmented edge segments
into useful lines and object boundaries.

Hough transform: a global method for edge linking and

boundary detection.
 Hough Transform

 A mathematical method designed to find lines in images.

It can be used for linking the results of edge detection, turning potentially
sparse, broken, or isolated edges into useful lines that correspond to the actual
edges in the image.
 Hough Transform

Let (x,y) be the coordinates of a point in a binary image (containing threshold edge detection
results).
The Hough transform stores in an accumulator array all pairs (a,b) that satisfy the equation
y = ax+ b. The (a,b) array is called the transform array.

Example:, the point (x,y) = (1,3) in the input image will result in the equation b = -a + 3, which can be plotted
as a line that represents all pairs (a,b) that satisfy this equation.
 Hough Transform
 Since each point in the image will map to a line in the transform domain, repeating the process for other
points will result in many intersecting lines, one per point.
 The meaning of two or more lines intersecting in the transform domain is that the points to which they
correspond are aligned in the image.
 The points with the greatest number of intersections in the transform domain correspond to the longest
lines in the image.
Hough Transform