Image Segmentation

Subject: FIP (181102)

Prof. Asodariya Bhavesh
ECD,SSASIT, Surat
 Image Segmentation

• Image segmentation divides an image into regions that are

connected and have some similarity within the region and
some difference between adjacent regions.
• The goal is usually to find individual objects in an image.
• For the most part there are fundamentally two kinds of
approaches to segmentation: discontinuity and similarity.
– Similarity may be due to pixel intensity, color or texture.
– Differences are sudden changes (discontinuities) in any of these, but
especially sudden changes in intensity along a boundary line, which is
called an edge.
 Detection of Discontinuities

• There are three kinds of discontinuities of intensity: points,

lines and edges.
• The most common way to look for discontinuities is to scan a
small mask over the image. The mask determines which kind
of discontinuity to look for.
9
R = w1 z1 + w2 z 2 + ... + w9 z9 = ∑ wi zi
i =1
 Detection of Discontinuities
Point Detection

R ≥T
where T : a nonnegative threshold
 Detection of Discontinuities
Line Detection

• Only slightly more common than point detection is to find a

one pixel wide line in an image.
• For digital images the only three point straight lines are only
horizontal, vertical, or diagonal (+ or –45°).
Detection of Discontinuities
Line Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
 Detection of Discontinuities
Gradient Operators

• First-order derivatives:
– The gradient of an image f(x,y) at location (x,y) is
defined as the vector:
G x   ∂f
∂x

∇f =   =  ∂f 
G y   ∂y 
– The magnitude of this vector: [
∇f = mag(∇f ) = Gx2 + G y2 ] 1
2

 Gx 
– The direction of this vector: α ( x, y ) = tan  
−1
G 
 y
– It points in the direction of the greatest rate of change
of f at location (x,y)
Detection of Discontinuities
Gradient Operators
 Detection of Discontinuities
Gradient Operators

Roberts cross-gradient operators

Prewitt operators

Sobel operators
 Detection of Discontinuities
Gradient Operators

Prewitt masks for

detecting diagonal edges

Sobel masks for

detecting diagonal edges
 Detection of Discontinuities
Gradient Operators: Example

∇f ≈ G x + G y
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators

• Second-order derivatives: (The Laplacian)

– The Laplacian of an 2D function f(x,y) is defined as
∂2 f ∂2 f
∇ f = 2 + 2
2

∂x ∂y

– Two forms in practice:

 More Advanced Techniques for Edge Detection

Marr-Hildreth edge detector

Marr and Hildreth argued that
1) Intensity changes are dependent of image scale and so their
detection requires the use of operators different sizes and
2) That a sudden intensity change will give rise to a peak or trough
in the first derivative or, equivalently, to zero crossing in the second
derivative.
Two salient feature:
1)It should be differential operator capable of computing first or
second order derivatives at every point in an image
2)It should be capable of being tuned to act at any desired scale, so
that large operator can be used to detect blurry edges and small
operators to detect sharply focused fine detail
 More Advanced Techniques for Edge Detection

• Consider the function: A Gaussian function

r2
− 2
h ( r ) = −e 2σ
where r 2 = x 2 + y 2
and σ : the standard deviation
• The Laplacian of h is
r2 The Laplacian of a
r −σ  2 2 − 2
∇ h( r ) = − 
2
e
2σ
Gaussian (LoG)
 σ
4

• The Laplacian of a Gaussian sometimes is called the Mexican
hat function. It also can be computed by smoothing the image
with the Gaussian smoothing mask, followed by application of
the Laplacian mask.
 More Advanced Techniques for Edge Detection

• Marr-Hildreth edge detector

• Marr and Hildreth argued that the most satisfactory operator fulfilling these
conditions is the filter ∇2G where, ∇2 is the Laplacian operator, and G is the
2-D Gaussian function x +y 2 2
−
G ( x, y ) = e 2σ 2

∂ 2 G ( x , y ) ∂ 2 G ( x, y )
∇ G ( x, y ) =
2
+
∂x 2 ∂y 2
∂  − x − 2σ 2  ∂  − y − x + y2 
2 2 2 2
x +y

∇ G ( x, y ) =
2
e +  e 2σ 

∂x  σ 2
 ∂y  σ 2

  
x2 + y2 x2 + y 2
x 1  2 − y 1  2 −
∇ 2G ( x, y ) =  4 − 2 e 2σ 2
+  4 − 2 e 2σ 2

σ σ  σ σ 
x2 + y2
 x + y − 2σ 
2 2 2 −
∇ 2G ( x, y ) =  e 2σ 2

 σ 4

This expression is called Laplacian of a Gaussion (LoG)
More Advanced Techniques for Edge Detection
 Detection of Discontinuities
Gradient Operators: Example

Sobel gradient

Gaussian smooth function Laplacian mask

 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Hough Transforms

Hough Transforms takes the images created by the edge detection

operators
Most of the time, the edge map generated by the edge detection
algorithms is disconnected
HT can be used to connect the disjointed edge points
It is used to fit the points as plane curves
Plane curves are lines, circles, and parabolas

The line equation is y = mx + c

However, the problem is that there are infinite line passing through one
points
Therefore, an edge point in an x-y plane is transformed to a c-m plane
Now equation of line is c = (-x)m + y
 Hough Transforms

All the edge points in the x-y plane need to be fitted

All the points are transformed in c-m plane
The objective is to find the intersection point
A common intersection point indicates that the edges points which are
part of the line
If A and B are the points connected by a line in the spatial domain, then
they will be intersecting lines in the Hough Space (c-m plane)
To check whether they are intersecting lines, the c-m plane is partitioned
as accumulator lines
To find this, it can be assumed that the c-m plane can be partitioned as
an accumulator array
For every edge point (x,y), the corresponding accumulator element is
incremented in the accumulator array
At the end of this process, the accumulator values are checked
Significance is that this point gives us the line equation of the (x,y) space
 Hough Transforms

Hough Transform steps:

1)Load the image
2)Find the edges of the image using any edge detector
3)Quantize the parameter space P
4)Repeat the following for all the pixels of the image:
if the pixel is an edge pixel, then
(a) c = (-x)m + y or calculate ρ
(b) P(c,m) = P(c,m) + 1 or increment position in P
5)Show the Hough Space
6)Find the local maxima in the parameter space
7)Draw the line using the local maxima

The major problem with this algorithm is that it does not work for
vertical lines, as they have a slope of infinity
Convert line into polar coordinates ρ = x cosӨ + ysinӨ, where Ө is the
angle between the line and x-axis, and ρ is the diameter
 Edge Linking and Boundary Detection
Global Processing via the Hough Transform

• Hough transform: a way of finding edge points in an image

that lie along a straight line.
• Example: xy-plane v.s. ab-plane (parameter space)
yi = axi + b
 Edge Linking and Boundary Detection
Global Processing via the Hough Transform

• The Hough transform consists of

finding all pairs of values of θ and
ρ which satisfy the equations x cos θ + y sin θ = ρ

that pass through (x,y).

• These are accumulated in what is
basically a 2-dimensional
histogram.
• When plotted these pairs of θ
and ρ will look like a sine wave.
The process is repeated for all
appropriate (x,y) locations.
Edge Linking and Boundary Detection
Hough Transform Example
The intersection of the
curves corresponding
to points 1,3,5

2,3,4

1,4
Edge Linking and Boundary Detection
Hough Transform Example
 Thresholding

• Assumption: the range of intensity levels covered by objects

of interest is different from the background.
1 if f ( x, y ) > T
g ( x, y ) = 
0 if f ( x, y ) ≤ T

Single threshold Multiple threshold

 Thresholding
The Role of Illumination
 Thresholding
The Role of Illumination

r ( x, y ) (a) (c) i ( x, y )

(d) (e)

f ( x, y ) = i ( x , y ) r ( x , y )
 Thresholding
Basic Global Thresholding
 Thresholding
Basic Global Thresholding
 Thresholding
Basic Adaptive Thresholding
 Thresholding
Basic Adaptive Thresholding

How to solve this problem?

 Thresholding
Basic Adaptive Thresholding

Answer: subdivision
 Thresholding
Optimal Global and Adaptive Thresholding

• This method treats pixel values as probability density functions.

• The goal of this method is to minimize the probability of
misclassifying pixels as either object or background.
• There are two kinds of error:
– mislabeling an object pixel as background, and
– mislabeling a background pixel as object.
 Thresholding
Use of Boundary Characteristics
 Thresholding
Thresholds Based on Several Variables

Color image
 Region-Based Segmentation

• Edges and thresholds sometimes do not give good

results for segmentation.
• Region-based segmentation is based on the
connectivity of similar pixels in a region.
– Each region must be uniform.
– Connectivity of the pixels within the region is very
important.
• There are two main approaches to region-based
segmentation: region growing and region splitting.
 Region-Based Segmentation
Basic Formulation

• Let R represent the entire image region.

• Segmentation is a process that partitions R into subregions,
R1,R2,…,Rn, such that
n
(a) ∪ Ri = R
i =1
(b) Ri is a connected region, i = 1,2,..., n
(c) Ri ∩ R j = φ for all i and j , i ≠ j
(d) P ( Ri ) = TRUE for i = 1,2,..., n
(e) P( Ri ∪ R j ) = FALSE for any adjacent regions Ri and R j
where P(Rk): a logical predicate defined over the points in set Rk
For example: P(Rk)=TRUE if all pixels in Rk have the same gray
level.
 Region Growing

• Thresholding still produces isolated image

• Region growing algorithms works on principle of similarity

• It states that a region is coherent if all the pixels of that region are
homogeneous with respect to some characteristics such as colour,
intensity, texture, or other statistical properties

• Thus idea is to pick a pixel inside a region of interest as a starting

point (also known as a seed point) and allowing it to grow

• Seed point is compared with its neighbours, and if the properties

match , they are merged together

• This process is repeated till the regions converge to an extent that no

further merging is possible
 Region Growing Algorithm

• It is a process of grouping the pixels or subregions to get a bigger

region present in an image

•Selection of the initial seed: Initial seed that represent the ROI should
be given typically by the user. Can be chosen automatically. The seeds
can be either single or multiple

•Seed growing criteria: Similarity criterion denotes the minimum

difference in the grey levels or the average of the set of pixels. Thus, the
initial seed ‘grows’ by adding the neighbours if they share the same
properties as the initial seed

•Terminate process: If further growing is not possible then terminate

region growing process
 Region Growing Algorithm

• Consider image shown in figure:

•Assume seed point indicated by underlines. Let the seed pixels 1 and 9
represent the regions C and D, respectively

•Subtract pixel from seed value

•If the difference is less than or equal to 4 (i.e. T=4), merge the pixel
with that region. Otherwise, merge the pixel with the other region.
 Split and Merge Algorithm

• Region growing algorithm is slow

• So seed point can be extended to a seed region

• Instead of a single pixel, a node of a Regional adjacency graph (RAG)

a region itself is now considered as a starting point.

• The split process can be stated as follows:

1)Segment the image into regions R1, R2,….Rn using a set of thresholds
2)Create RAG. Use a similarity measure and formulate a homogeneity
test
3)The homogeneity test is designed based on the similarity criteria such
as intensity or any image statistics
4)Repeat step 3 until no further region exits that requires merging
 Split and Merge Algorithm

8 8 8
8 8 8
Region-Based Segmentation
Region Growing
 Region-Based Segmentation
Region Growing

• Fig. 10.41 shows the histogram of Fig. 10.40 (a). It is difficult to

segment the defects by thresholding methods. (Applying region
growing methods are better in this case.)

Figure 10.40(a) Figure 10.41

 Split and Merge using Quadtree

• Entire image is assumed as a single region. Then the homogeneity test

is applied. If the conditions are not met, then the regions are split into
four quadrants.

• This process is repeated for each quadrant until all the regions meet
the required homogeneity criteria. If the regions are too small, then the
division process is stopped.

•1) Split and continue the subdivision process until some stopping
criteria is fulfilled. The stopping criteria often occur at a stage where no
further splitting is possible.
•2) Merge adjacent regions if the regions share any common criteria.
Stop the process when no further merging is possible
 Region-Based Segmentation
Region Splitting and Merging

• Region splitting is the opposite of region growing.

– First there is a large region (possible the entire
image).
– Then a predicate (measurement) is used to
determine if the region is uniform.
– If not, then the method requires that the region
be split into two regions.
– Then each of these two regions is independently
tested by the predicate (measurement).
– This procedure continues until all resulting regions
are uniform.
 Region-Based Segmentation
Region Splitting

• The main problem with region splitting is determining where to

split a region.
• One method to divide a region is to use a quadtree structure.
• Quadtree: a tree in which nodes have exactly four descendants.
 Region-Based Segmentation
Region Splitting and Merging

• The split and merge procedure:

– Split into four disjoint quadrants any region Ri for
which P(Ri) = FALSE.
– Merge any adjacent regions Rj and Rk for which
P(RjURk) = TRUE. (the quadtree structure may not
be preserved)
– Stop when no further merging or splitting is
possible.