Digital Image Processing_4 Unit_ASRao
Digital Image Processing_4 Unit_ASRao
Laplacian Mask
ab
cd
Laplacian Mask FIGURE
(a) Laplacian kernel
used for point
detection.
(b) X-ray image
of a turbine blade
with a porosity
manifested by a
single black pixel.
(c) Result of convolving
the kernel
with the image.
(d) Result was a single
point (shown enlarged).
ASRao 16
Line Detection
Only slightly more common than point detection is to find a one pixel
wide line in an image.
Examples:
- Horizontal mask has high response when a line is passed through the
middle row of the mask with a constant background.
Apply every masks on the image
let R1 , R2 , R3 , R4 denotes the response of the horizontal, +45 degree,
vertical and -45 degree masks, respectively.
if, at a certain point in the image
|Ri | > |Rj |, for all j≠i,
that point is said to be more likely associated with a line in the
direction of mask i.
To detect lines in an image, one can use a special filter called a mask that
looks for lines in a specific direction. By applying the mask to the image
and taking the absolute value of the result, potential lines can be
identified.
Finally, by setting a threshold, or a minimum value, you can determine what is considered
a line, and detect all lines in the image that match the direction defined by the mask.
The points that are left are the strongest responses, which, for lines one pixel thick,
correspond closest to the direction defined by the mask.
Edge Detection
Segmentation by finding pixels on a region boundary.
Edges found by looking at neighboring pixels.
Region boundary formed by measuring gray value differences between
neighboring pixels
An edge is a set of connected pixels that lie on the boundary between
two regions.
an edge is a “local” concept whereas a region boundary, owing to the
way it is defined, is a more global idea.
Figure: Models (ideal representation of a step, a ramp, and a roof edge, and
their corresponding intensity profiles
Figure: A 1508 X 1970 image showing (zoomed) actual ramp, step and roof
edge profiles. The profiles are from dark to light, in the areas indicated by
the short line segments shown in the small circles. The ramp and step
profiles span 9 pixels and 2 pixels. The base of the roof edge is 3 pixels.
ab
c
FIGURE: (a) Image. (b) Horizontal intensity profile that includes the
isolated point indicated by the arrow. (c) Subsampled profile; the dashes
were added for clarity. The numbers in the boxes are the intensity values
of the dots shown in the profile. The derivatives were obtained for the
first derivative and for the second.
First column: 8-bit images
with values in the range [0,
255], and gray level profiles
of a ramp edge corrupted by
random Gaussian noise of
mean 0 and σ = 0.0, 0.1, 1.0
and 10.0, respectively.
ASRao 24
First Order derivative (Gradient Operator)
First - order derivatives:
The Gradient of an image f(x,y) at location (x,y) is defined as the vector:
f ( x, y ) f ( x, y ) f ( x, y )
f Gx G y
xy x y
where Gx is the gradient along the x-direction and Gy is the gradient
along y-direction.
The magnitude, M(x, y), of this gradient vector at a point (x, y) is given
by :
M ( x, y ) f Gx2 G y2
1
Gy
( x, y ) tan
Gx
Gradient is a non-linear operator ASRao 25
Roberts cross-gradient operators
Prewitt operators
G y zASRao
3 z6 z9 z1 z 4 z7
Z7 Z8 Z9
27
Masks of size 3X3 are preferred using either the differences along the x-
and y-directions as
f z7 z8 z9 z1 z2 z3 z3 z6 z9 z1 z4 z7
or using the cross differences along the diagonal as
f z1 z2 z4 z6 z8 z9 z2 z3 z6 z4 z7 z8
ASRao 28
cd
FIGURE:
(a) Image of size
834 × 1114 pixels,
with intensity values
scaled to the range
[0,1].
(b) |Gx|, the
component of the
gradient in the x-
direction, obtained
using the Sobel kernel
to filter the image.
(c) |Gy| , the
component in y-
direction
(d) The gradient
image, |Gx| + |Gy|.
f G x G y ASRao 29
Figure: Same sequence as previous, but with the original image smoothed
with a 5X5 averaging kernel prior to edge
ASRao detection. 30
ASRao 31
Diagonal Edge Detection
Second Order derivative (Laplacian Operator)
The Laplacian of an 2D function f(x,y) is defined as
2 f 2 f
f
2
x 2
y 2
2 f
f ( x 1, y ) f ( x 1, y ) 2 f ( x, y )
x 2
2 f
f ( x, y 1) f ( x, y 1) 2 f ( x, y )
y 2
2 f f ( x 1, y ) f ( x 1, y ) f ( x, y 1) f ( x, y 1) 4 f ( x, y )
ASRao 32
The Laplacian is a linear operator.
2 f 4 z5 z 2 z 4 z6 z8
ASRao 33
ASRao 34
The Laplacian generally is not used in its original form for edge detection
for the following reasons: (i) Second-order derivative is unacceptably
sensitive to noise and (ii) The magnitude of the Laplacian produces
double edges.
The Laplacian is often used with Gaussian smoother given by the function:
r2
h( r ) e 2 2
(d)
(c)
ASRao 37
(e) (f) (g)
ASRao 38
Sobel Edge Detector
Sobel Edge detector is one of the famous edge detector.
Consider the Sobel Edge Detector, then the image is processed in the X
and Y directions respectively.
This would result in the formation of new image, which actually is the
sum of the X and Y edges of the image.
This approach works through calculation of the gradient of the image
intensity at every pixel within the image.
ASRao 40
The next sequence is to convolute the Gx (Sobel X) and Gy (Sobel Y) over
the input image which shall enable us to calculate the value for one pixel
at a time.
Then, the shift happens and the move has to be made towards the right,
i.e., the next column shift till the column end.
ASRao 41
Convolution
The above matrix is nothing but a 5X4 image and is now being convolved
with Gx (Sobel X) operator, in the considered 5X4 matrix, we take the first
3X3 matrix, in which, the center value (i) is computed.
On the convolution, the following resultant matrix will be obtained.
ASRao 42
The Sobel-X values got convoluted with the original image matrix values.
Hence, the resultant matrix values would be as follows:
Gx 1* a 0 * b (1) * c 2 * d (2) * e 1* f 0 * g (1) * h
1*100 0 *100 (1) * 50 2 *100 (2) * 50 1*100 0 *100 (1) * 50
200 ASRao 43
Going by the same calculations we have done for the Gx, Gy can be
computed.
Gx (1) * a (2) * b (1) * c 0 * d 0 * e 1* f 2 * g 1* h
(1) *100 (2) *100 (1) * 50 0 *100 0 * 50 1*100 2 *100 1* 50
0
The above calculations shall help the reader in visualizing the edges
indirectly. In the X-axis there are changes in the cells. But in the Y-axis,
there are no changes and hence the result Gy was 0.
G Gx2 G y2
The G will be compared against the threshold and with which, one can
understand the taken point is an edge
ASRao or not. 44
Input Image
Gradient Magnitude
Edges
ASRao 45
Prewitt Edge Detector
The Prewitt Edge Detector is similar to the Sobel Edge Detector but with
minor change. The Prewitt operator shall give the values which are
symmetric around the center. But, Sobel operator gives weight to the
point which is lying close to (x,y).
ASRao 46
Input Image
Prewitt Gradient
Detected Edges
ASRao 47
Laplacian Edge Detection
The Laplacian edge detector uses only one kernel where it was two w.r.t
Sobel edge detector. Laplacian edge detector is capable and it calculates
the second order derivative in a single go.
ASRao 48
There are however, some disadvantages in this approach:
- There are pixel thick edges produced in this method.
- Very highly sensitive to noise
ASRao 49
Canny Edge Detection
This is one of most important and complex edge detector. Canny is not
like other traditional edge detectors, it is just not masking or hovering on
the input image matrix.
It was developed by John F. Canny in 1986 and is known for its optimal
edge detection capabilities.
ASRao 50
Steps to follow for Canny Edge Detection:
1. Conversion to the Gray Scale
Let us take a sample image and proceed with the conversion. We have
converted the input RGB image to a Grayscale image.
Now the Sobel operator has to applied over the input image. The
resultant Sobel operated image is shown below. This is referred as
Gradient Magnitude of the image.
ASRao 52
We preferred Sobel operator and it is the general approach. But, it is not
a mandated rule to always go with Sobel operator. It can be any gradient
operator and the result should be the gradient magnitude of the image.
G Gx2 G y2
The G will be compared against the threshold and with which, one can
understand the taken point is an edge
ASRao or not. 53
1 Gy
The formula for finding the edge detection is just
tan
Gx
4. Non-Maximum Suppression
This is the next step in the sequence. The Gradient magnitude operators
discussed in the previous stage normally obtains thick edges. But the
final image is expected to have thin edges.
North to South:
East to West:
ASRao 55
The Center cell is the region of interest for us. It is important to
understand this point.
There can be only 4 possible directions for any pixel. They are
- 0 Degrees - 45 Degrees - 90 Degrees - 135 Degrees
Any edge coming under the red range is all set to 135 degrees. (which
ASRao 57
means 112.5 to 157.5 degrees is set as 135 degrees.
After this process, direction of the edges is mapped to any of the 4
directions mentioned. The input image now shall be like the one
presented below where directions of the edges are appropriately
mapped.
ASRao 58
The 4th step in the process comes now.
The edge directions are all determined and the Non-Maximum
suppression is to be applied.
Non-Maximum suppression as the name suggests, is a process where
suppression of the pixels to zero which cannot be considered as an edge
is carried out. This shall enable the system to generate a thin line in the
output image as shown below.
These results are obtained before the thresholding and as expected the
next stage to perform thresholding and smoothing.
ASRao 59
5. Thresholding – A must do process
As one could see from the previous stage results, Non-Maximum
suppression has not provided us excellent results.
There is still some noise. The image even raises a threat in mind that
some of the edges shown may not really be so and some edges could
be missed in the process. Hence, there has to be a process to
address this challenge. The process to be followed is thresholding.
We need to go with double thresholding and in this process we have to
set two thresholds. One, a high and other is low. What is this?
Simple. Assume high threshold value as 0.8. any pixel with a value above
0.8 is to be seen as stronger edge.
- Another threshold, the lower one can be 0.2. in that case, any pixel
below this value is not an edge atASRao
all and hence set them all to 0. 60
Now comes the next question, what about the values in between?
- They may or may not be an edge. They are referred as a weak edge.
There has to be a process to determine which of the weak edges are
actual edges so as to not to miss them.
6. Edge Tracking
ASRao 61
7. The final cleansing
All the remaining weak edges can be removed and that is it. The process
is complete. Once this process is done, we could get the following output
image as the result.
ASRao 62
ASRao 63
Set of pixels from edge detecting algorithms, seldom define a boundary
completely because of noise, breaks in the boundary etc.
A good
Input Gradient Edge Edge output
Image Operation Linking image
ASRao 64
cd
FIGURE:
(a) Image of size
834 × 1114 pixels,
with intensity values
scaled to the range
[0,1].
(b) |Gx|, the
component of the
gradient in the x-
direction, obtained
using the Sobel kernel
to filter the image.
(c) |Gy| , the
component in y-
direction
(d) The gradient
image, |Gx| + |Gy|.
f G x G y ASRao 65
There are two basic approaches for edge linking:
(1) Local Processing: This is a simplest approach for linking pixels in
a small neighborhood.
(2) Global Processing via the Hough Transform: Here, we attempt
to link edge pixels that lie on specified curves. The Hough transform
is designed to detect lines, using the parametric representation of a
line.
ASRao 66
Local Processing
All points that share common properties are linked together. These are:
Strength/magnitude of the gradient
Direction of the gradient
Adjacent edge points with similar magnitude and direction are linked.
ASRao 67
The magnitude, M(x, y), of this gradient vector at a point (x, y) is given
by its Euclidean vector norm:
3. Scan the rows of g and fill all gaps in each row that do not exceed a
specified length , K.
ASRao 69
In this example, we can find the license plate candidate after edge
linking process.
ASRao 70
Global Processing via the Hough Transform
Points are linked by determining whether they lie on a curve of specified
shape.
Motivation
Given a point (xi, yj), many lines pass through this point as yi=axi+b
with different a and b.
Find all lines determined by every pair of points and subsets of
points that are close to particular lines.
ab
FIGURE
(a) xy-plane.
(b) Parameter space.
ASRao 71
Hough Transform
ASRao 72
The Procedures of Hough Transform
Step 1:
Subdivide ab- plane to accumulator cells. Let A(i, j) be the cell at (i, j) where
amin ≤ ai ≤ amax, bmin ≤ bj ≤ bmax and A(i, j) = 0
Step 2:
For every (xk, yk), find b=-xkap + yk for each allowed p
Step 3:
Round off b to the nearest allowed value bq. Let A(p, q) = A(p, q) + 1.
ASRao 75
Any point (x,y) for which f(x,y) > T is called an object point; otherwise the
point is called a background point.
When the value T changes over an image, then that process is referred
as Variable thresholding. Sometimes it is also termed as local or regional
thresholding.
ASRao 77
ASRao 78
Multilevel Thresholding
It is also possible to extract objects that have a specific intensity range
using multiple thresholds.
Image with dark
Background and two
Light objects
ASRao 79
Segmentation problems requiring multiple thresholds are best solved
using region growing methods.
T T x, y, p( x, y ), f ( x, y )
ASRao 80
A Thresholded image g(x,y) is defined as
1, f ( x, y ) T
g ( x, y )
0, f ( x, y ) T
ASRao 81
Role of Noise in Image Thresholding
ASRao 82
ASRao 83
Role of Illumination in Image Thresholding
f ( x, y ) i ( x, y ) r ( x, y )
Non-uniform illumination may change the histogram in a way that it
becomes impossible to segment the image using a single global
threshold.
z ( x, y ) ln f ( x, y )
ln i ( x, y ) ln r ( x, y )
i ' ( x, y ) r ' ( x, y )
Solution
g ( x, y ) k i ( x, y )
ASRao 85
Basic Global Thresholding
Based on visual inspection of histogram
Initial threshold must be chosen greater than the minimum and less than
the maximum intensity level in the image.
ASRao 90
Focusing on boundary pixels offers two advantages:
1. Background and object pixel probabilities are nearly equal.
2. Object and background regions have similar areas, making the histogram
symmetrical.
ASRao 92
If we assume that an image
Object is dark
Background is white.
If the gradient value is greater than or equal to some threshold T, we
assume that this point belong to edge point since points near the edge,
the gradient value will be high.
If the gradient value is less than some threshold T, we assume that this
point does not belong to edge point not even within a region near the
edge.
Laplacian is +ve for the object pixel and –ve for the background pixel.
ASRao 93
ASRao 94
Local thresholding is a technique used in image segmentation to separate
objects from the background by applying a threshold value that varies
locally across the image.
Key Concepts:
We want to apply local thresholding using a 3x3 window and calculating the
threshold value as the mean intensity value within the window.
ab
cd
(a) Original image
(b)Result of global
Thresholding
(c) Image subdivided
Into individual sub Images
(d) Result of adaptive
thresholding
ASRao 99
Figure: Zoomed version of portions shown in red square boxes in the
previous slide.
How to solveASRao
this problem? 100
Answer: Subdivision
ASRao 101
Figure: Comparison of the thresholding algorithms (a) Original image (b)
Result of adaptive thresholding algorithm (c) Result of Global thresholding
algorithm
Q(Ri) is a logical predicate property defined over the points in set Ri.
It states that a region is coherent if all the pixels of that region are
homogeneous w.r.t some characteristics such as color, intensity, texture,
or other statistical properties..
Seed point is compared with its neighbors, and if the properties match,
they are merged together.
0 1 2 3
0 0 1 2 0
1 2 5 6 1
2 1 4 7 3
3 0 2 5 1
ASRao 108
Q1: Apply region growing on the following image with seed point as 6
and threshold value.as 3.
5 6 6 7 6 7 6 6
6 7 6 7 5 5 4 7
6 6 4 4 3 2 5 6
5 4 5 4 2 3 4 6
0 3 2 3 3 2 4 7
0 0 0 0 2 2 5 6
1 1 0 1 0 3 4 4
1 0 1 0 2 3 5 4
ASRao 109
Advantages and Disadvantages of Region Growing
Advantages
Region growing methods can correctly separate the regions that have
the same properties we define.
Region growing methods can provide the original images which have
clear edges with good segmentation results.
ASRao 110
Disadvantages
Computationally expensive
Sensitive to noise.
ASRao 111
Region Splitting
Split and continue the subdivision process until some stopping criteria is
fulfilled. The stopping criteria often occur at a stage where no further
splitting is possible.
This process is repeated for each quadrant until all the regions meet the
required homogeneity criteria. If the regions are too small, then the
division process is stopped.
ASRao 112
To explain this in terms of graph theory, we call each region a node.
ASRao 113
Region Merging
Here we start from the pixel level and consider each of them as a
homogeneous region..
At any level of merging, we check if the four adjacent regions satisfy the
homogeneity property. If yes, they are merged to form a bigger region,
otherwise the regions are left as they are.
This is repeated until no further region requires merging.
ASRao 114
Region Splitting and Merging
ASRao 115
Q1: Apply Splitting and merging on the following image with
threshold value equal to 3.
5 6 6 6 7 7 6 6
6 7 6 7 5 5 4 7
6 6 4 4 3 2 5 6
5 4 5 4 2 3 4 6
0 3 2 3 3 2 4 7
0 0 0 0 2 2 5 6
1 1 0 1 0 3 4 4
1 0 1 0 2 3 5 4
ASRao 116
Allanki Sanyasi Rao
AMIE; M.Tech; (Ph.D); MISTE; MIETE
Associate Professor & HOD
Dept. of ECE
ASRao 117
Low Level Processing
Input & Output both are images
Image preprocessing for noise reduction, contrast enhancement, image
sharpening.
ASRao 119
Introduction
What is Morphology?
Morphology generally concerned with Shape and properties of objects.
Used for segmentation and feature extraction.
The word Morphology commonly denotes a branch of biology that
deals with the form and structure of animals and plants.
The fundamental use of Morphological processing is to remove the
imperfections in the structure of images.
Binary images may contain numerous imperfections. In particular, the
binary regions produced by simple thresholding are distorted by noise
and texture.
Morphological image processing pursues the goals of removing these
imperfections.
ASRao 120
Mathematical morphology as a tool for extracting image components
such as regions, boundaries, skeletons etc.
We can also use morphological techniques for pre- or post processing
images.
Preliminaries
We will use Set Theory to formalize the
operations in morphological image
{B}
processing.
Reflection:
Bˆ w | w b, for b B
Translation:
B z c | c zb, for b B
ASRao 122
Structuring Element (SE)
ASRao 124
Operators by Graphical Examples
abc
de
Figure:
(a) Two sets A and B
(b) The union of A and B
(c) The intersection of A and B
(d) The complement of A
ASRao (e) The difference between A125and B
Logical Operators for Binary Images
Figure:
Some logic operations
Between binary images.
ASRao 126
Common Morphological Operations
ASRao 127
ASRao 128
Dilation: Adds pixels to the boundaries of objects in an image.
Fills in holes.
Smoothens object boundaries.
Adds an extra outer ring of pixels onto object boundary, i.e., object
becomes slightly larger..
Dilation expands the connected sets of 1s of a binary image.
It can be used for
1. Expanding shapes:
ASRao 129
Suppose A and B are sets of pixels, dilation of A by B
A B Ax
xB
z A
A B z | Bˆ
ASRao 130
Before Dilation After Dilation
ASRao 131
Example of Dilation using three different rectangular structuring elements
ASRao 132
Common Morphological Operations
ASRao 133
Structuring Element for Dilation (Examples)
ASRao 134
Before Dilation
After Dilation
ASRao 135
Practice problem
Solution
ASRao 136
ASRao 137
Erosion reduces the number of pixels from the object boundary or
“shrink” or “thin” objects in a binary image.
The number of pixels removed depends on the size of structuring
element.
Mathematically, the erosion of a set A by B denoted AӨB is defined as
AB z | Bˆ z Ac
Typical Uses of Erosion
Removes isolated noisy pixels.
Smoothens object boundary.
Removes the outer layer of object
pixels, i.e. object becomes slightly
smaller.
ASRao 138
Example
ASRao 140
Before Erosion
After Erosion
ASRao 141
ASRao 142
ASRao 143
Structuring Element Decomposition
Structuring element decomposition is a technique used in morphological
processing to break down a complex structuring element into simpler
components. This is useful for improving the efficiency and flexibility of
morphological operations.
Imagine having a large, complex shape (structuring element) for image
processing. Breaking it down into smaller, simpler shapes can achieve the
same result. This approach is similar to dividing a large task into smaller,
more manageable steps.
Examples:
1. Decomposing a rectangle: A large rectangle can be decomposed into
smaller rectangles or even lines.
2. 2. Decomposing a circle: A circle can be decomposed into smaller circles
ASRao 144
or even lines and arcs.
Figure. (a) Decomposition of structuring element B = B1 ⊕ B2;
(b) Input and output to decomposition
ASRao windows B1 and B2. 145
Suppose we have a structuring element that is a 5x5 square:
11111
11111
11111
11111
11111
We want to decompose this structuring element into smaller
components. One possible decomposition is into four 3x3 squares:
111 111
111 111
111 111
111 111
111 111
111 111
This decomposition can be used to perform morphological operations,
such as erosion or dilation, more efficiently.
ASRao 146
Benefits:
Improved efficiency: Decomposing a structuring element can reduce the
number of operations required to perform morphological processing.
ASRao 147
ASRao 148
Combining Dilation and Erosion
Opening Operation:
An opening operation is a combination of erosion followed by dilation.
The process involves:
1. Eroding the image with a structuring element to remove small objects and
noise.
2. Dilating the eroded image with the same structuring element to restore
the original shape.
ASRao 149
Closing Operation:
A closing operation is a combination of dilation followed by erosion. The
process involves:
1. Dilating the image with a structuring element to fill small gaps and holes.
2. Eroding the dilated image with the same structuring element to restore
the original shape.
ASRao 150
Closing & Opening Processes
ASRao 151
Differences between Dilation and Erosion & Open and Close
Erosion and dilation clean image but leave objects either smaller or
larger than their original size.
Opening and closing perform same function as erosion and dilation but
object size remains the same. ASRao 152
ASRao 153
Closing is a process in which first dilation operation is performed and
then erosion operation is performed.
ASRao 154
Example:
Original Image
Structuring Element
ASRao 155
After Dilation
ASRao 156
ASRao 157
ASRao 158
Opening is a process in which first erosion operation is performed and
then dilation operation is performed.
ASRao 159
Example:
Original Image
Structuring Element
ASRao 160
After Erosion
ASRao 161
ASRao 162
ASRao 163
In morphology, hit-or-miss transform is an operation that detects a
given pattern in a binary image using a structuring element containing
1’s, 0’s and blank for don’t cares.
ASRao 164
Hit-and-miss algorithm can be used to thin and skeletonize a shape in a
binary image.
ASRao 166
Example: Hit & Miss transform is an iterative process containing repeated
steps to thin the shape by hit-and-miss method. In each iteration, some
different structuring elements are used to identify the edge pixels t be
removed.
ASRao 167
Example:
ASRao 168
ASRao 169