Computer Vision and Image processing

Chapter 4
Woldia University
IOT
March 2021
Outline
• Histograms: Find, Plot, Analyse
• Spatial domain
• Frequency domain: DFT, DCT, and HT
• Template Matching
 Histograms

 Goal:
 Learn to:
 Find histograms, using both OpenCV and Numpy functions
 Plot histograms, using OpenCV and Matplotlib functions
 You will see these functions : cv2.calcHist(), np.histogram() etc.
 Histograms

 So what is histogram ?
 You can consider histogram as a graph or plot, which gives you an overall idea about the
intensity distribution of an image.
 It is a plot with pixel values (ranging from 0 to 255, not always) in X-axis and
corresponding number of pixels in the image on Y-axis.
 It is just another way of understanding the image.
 By looking at the histogram of an image, you get intuition about contrast, brightness,
intensity distribution etc of that image.
 Histograms

 Almost all image processing tools today, provides features on histogram.

 Left region of histogram shows the amount

of darker pixels in image and right region
shows the amount of brighter pixels.
 From the histogram, you can see dark
region is more than brighter region, and
amount of midzones (pixel values in mid-
range, say around 127) are very less.
 Histograms

 1. Histogram Calculation in OpenCV

 cv2.calcHist(images, channels, mask, histSize, ranges[, hist[, accumulate]])

 2. Histogram Calculation in Numpy

 which is much faster than (around 10X) np.histogram().

 Histograms

 OpenCV function is more faster than (around 40X) than np.histogram().

 Plotting Histograms:
 1. Using Matplotlib
 Histograms

 2. Using OpenCV:
 We used cv2.calcHist() to find the histogram of the
full image.
 What if you want to find histograms of some
regions of an image?
 Just create a mask image with white color on the
region you want to find histogram and black
otherwise.
 Then pass this as the mask.
 Histograms

 See the result. In the

histogram plot, blue line
shows histogram of full
image while green line shows
histogram of masked region.
Frequency domain
 Frequency domain

 In frequency domain we don’t analyze signal with respect to time , instead with respect of
frequency.
 Difference b/n Spatial domain and Frequency domain:
 In spatial domain , we deal with images as it is. The value of the pixels of the image
change with respect to scene.
 Whereas in frequency domain , we deal with the rate at which the pixel values are
changing in spatial domain.
 Frequency domain

 Spatial domain:

 In spatial domain, we directly deal with the input matrix.

 Frequency domain

 Frequency domain:
 We first transform the image to its frequency distribution.
 Then our black box system perform what ever processing it has to performed , and the
output of the black box in this case is not an image , but a transformation.
 After performing inverse transformation , it is converted into an image which is then
viewed in spatial domain.
Frequency domain

 Transformation:
 A signal can be converted from time domain into frequency
domain using mathematical operators called transforms.
 Types of Transformation are:
1. Fourier Series
2. Fourier transformation
3. Laplace transform
4. Z transform
5. etc..
 Frequency domain

 Frequency Components:
 Any image in spatial domain can be represented in a frequency domain. But what do this
frequencies actually mean.
 Frequency components divide into two major components.
 High Frequency Components: correspond to edges in an image.
 Low Frequency Components: in an image correspond to smooth regions.
 Frequency domain

 FOURIER:
 Fourier was a mathematician in 1822.
 He give Fourier series and Fourier transform to convert
a signal into frequency domain.
 Frequency domain

 Fourier series is applied on periodic signals.

 Fourier transform is applied for non periodic signals.
 Both Fourier series and Fourier transform has two formulas. One for conversion and one
converting it back to the spatial domain.
 FOURIER SERIES
 The Fourier series can be denoted by this formula.
 Frequency domain

 The inverse can be calculated by this formula.

 The last signal is actually the sum of all the above

signals. This was the idea of the Fourier.
 Frequency domain

 Which one is applied on images, the Fourier series or the Fourier transform?
 Since the images are non periodic , so Fourier transform is used to convert them into
frequency domain.
 Fourier transform:
 DISCRETE FOURIER TRANSFORM.
 Since we are dealing with images, and in fact digital images , so for digital images we will
be working on discrete Fourier transform.
 Frequency domain

 Fourier term of a sinusoid. It include three things.

 Spatial Frequency
 Magnitude
 Phase
 The spatial frequency directly relates with the brightness of the image.
 The magnitude of the sinusoid directly relates with the contrast. Contrast is the difference
between maximum and minimum pixel intensity.
 The Phase contains the color information.
 Frequency domain

 The formula for 2 dimensional discrete Fourier transform is:

 f(x,y) denotes the image , and F(u,v) denotes the discrete Fourier transform.
 The formula for 2 dimensional inverse discrete Fourier transform is:
 Frequency domain

 Fourier Transform:
 We as a human being can easily
identify the image as airplane, but
computers can’t.
 Computers can pick a lot of details
and come up with new answers that
human might be miss.
 Frequency domain

 For instance, a computer program

can identify the differences between
this two images whereas we would
struggle with it!
 One of the tools we use in image
processing is the Fourier
transform.
 Frequency domain

 The Fourier transform is an

extension of the Fourier serious,
which we use for periodic
continuous signals.
 Frequency domain

 The Fourier transform can takecare

of the non periodic signals like this
one noise.
 Frequency domain

 Humans are good in reading Handwritten and Printed text. But Computers really struggle with it.
Frequency domain

 That’s the reason that most

websites uses this capture
images.
 To identify the real human
with the robot.
Frequency domain

 Having the computer reads

texts is used for the post office.
Frequency domain

 For transferring Books to

Electronic copies.
Frequency domain

 For Number plate recognition

Frequency domain

 For Automating input of

handwritten forms.
 Frequency domain

 One of the method the we use for recognizing characters is to use the Fourier
transform.
 When we apply transform to something we are just taking some information that is
captured in the original data, but we displayed in a different way.
 Frequency domain

If you just take the Fourier transform of the signal, we move from the time
domain to the frequency domain.

A step function in a time domain, become a

sinc function in frequency domain.

A sinc function in a time domain, become a

step function in frequency domain.
Frequency domain

 The Fourier transform relates the function’s

time domain, shown in red, to the function’s
frequency domain, shown in blue.
 The component frequencies, spread across the
frequency spectrum, are represented as peaks in
the frequency domain.
 Frequency domain

 We can also move images in the frequency domain.

 Frequency domain

 We can also move images in the frequency domain.

 Frequency domain

 We can also move images in the frequency domain.

Frequency domain
 Frequency domain

 We can also move images in the frequency domain.

 Frequency domain

 We can also move images in the frequency domain.

 This lines indicate the

legs of the tripod.
 Frequency domain

 We can also move images in the frequency domain.

 The line of the horizon.

Frequency domain
Frequency domain

 In fact, we can use the

representation of written
numbers and letters in the
frequency domain to identify
what letters or numbers in the
time domain.
 So, we use the Fourier transform
to identify letters and numbers.
Frequency domain

 We can also use the frequency

domain to remove noise from an
image.
 We can also remove white noises or
spackle by removing the low
frequency component of the signal
in the frequency domain.
Frequency domain
Frequency domain

1. DCT
2. DFT
3. HT
 Frequency domain

 1. DCT [Discrete Cosine Transform] i.e. Cosine indicates there are only real terms.
DCT uses real cosine values.
DCT
f --------------> F
[Spatial domain] ? [DFT] [Frequency domain]

 DFT has the complex term. 1D: f(x) -------> F(u)

DFT uses complex signals. 2D: f(x,y) -------> F(u,v)
 Frequency domain

• F(u)
   for 1 Dimensional, F(u,v) for 2Dimensional.
 F = C.f for 1D
 F = C.f.CT for 2D
 Where, C denote Cosine Transform Matrix
 C(u,v) = , for the 1st raw, u=0, 0v N-1
= cos[] , for the other rows, 1 u N-1
0 v N-1
 Frequency domain

 Let N = 4
 C(u,v) = u & v -> 0 to N-1
 For the first raw, = = 0.5

CT = (-1) i+j Mij

Frequency domain
Frequency domain

(CT)T = C
 Frequency domain

DCT:
 C = CT -----> Not symmetric matrix
 DCT is real term, and orthogonal.
 DFT has the complex terms, and it is also a Sinusoidal function.
 DCT also used in JPEG Compression.
 DCT also known as Sinusoidal function., which is nothing but you can manipulate in terms
of sine. That is it changes with time.
 Frequency domain

2. DFT: Discrete Fourier Transform

 1. Spatial domain --------> Frequency domain
f -------> F
DCT: using CM! DFT using FM?
 2. Represented as WNnk
Twiddle factor
 T = WNnk
= [e-j2π]nk/N
= e-j2πnk/N
 Frequency domain
k
0 1 2 3
T = WNnk 1 1 1 1
0
[e-j2π]nk/N n 1 -j -1 j
1
e-j2πnk/N 2
1 -1 1 -1
1 j -1 -j 4x4
e-j2πnk/4 3 NXN
 Frequency domain

 2D F = TfTT = TfT => is a Symmetric matrix. 1D F = Tf

 E.g. 2D: f = 0 1 2 1
1 2 3 2 Find DFT
2 3 4 3
1 2 3 2
4x4
 Soln: F = TfTT = TfT
 Frequency domain

T f

4 8 12 8
-2 -2 -2 -2
x =
0 0 0 0
-2 -2 -2 -2
Frequency domain

TT

32 -8 0 -8
-8 0 0 0
x =
0 0 0 0
-8 0 0 0
 Frequency domain

 Q: Find the DFT of f(x) = {0, 1, 2, 1}

 Soln: let we represent DFT as F(k) N=4
 F(K) = , k = 0, 1, …, N-1
k = 0, 1, 2, 3
 Frequency domain

 Q: Find the DFT of f(x) = {0, 1, 2, 1}

 Soln: let we represent DFT as F(k) N=4
 F(K) = , k = 0, 1, …, N-1
k = 0, 1, 2, 3
 Frequency domain

 Q: Find the DFT of f(x) = {0, 1, 2, 1}

 Soln: let we represent DFT as F(k) N=4
 F(K) = , k = 0, 1, …, N-1
k = 0, 1, 2, 3
 Frequency domain

 Q: Find the DFT of f(x) = {0, 1, 2, 1}

 Soln: let we represent DFT as F(k) N=4
 F(K) = , k = 0, 1, …, N-1
k = 0, 1, 2, 3

F(k) = {4, -2. 0, -2}

 Frequency domain

3. Hadamard Transform (HT):

 H ---> Symmetric
---> Non Sinusoidal function.
 1D F = H.f
 2D F = H.f.HT
 Frequency domain

 2x2 matrix of Hadamard H =

[ ]
1
1
1
-1
2x2

[ ]
 4x4 matrix of Hadamard H = 1 1 1 1
1 -1 1 -1
2 1 -1 -1
1 -1 -1 1
4x4
 Frequency domain

[ ]
1 1 1 1 1 1 1 1
 8x8 matrix of Hadamard
1 -1 1 -1 1 -1 1 -1
H = 2 1 -1 -1 2 1 -1 -1
1 -1 -1 1 1 -1 -1 1
3 1 1 1 -1 -1 -1 -1
1 -1 1 -1 -1 1 -1 1
4 1 -1 -1 -1 -1 1 1
8x8
1 -1 -1 1 -1 1 1 -1
 Frequency domain

 Q1: 1D: f(x) = {1,2,0,3}, apply HT?

 This is a 4x1 image but N=4 so, your transform matrix will be 4x4
 Soln: F = H.f

[ ] [ ]
1 6
2 = -4
0 0
3 2

 Matrix Multiplication
 We just convert an image from spatial domain f(x) to frequency domain F.
 Frequency domain

[ ]
 Q2: f = 2 1 2 1
3 2 3 2 => Apply HT ?
2 3 4 3
1 2 3 2
4x4
 Soln.
 2D: F = H.f. HT = H.f.H

[ ]
6 8 12 8
= 2 0 0 0
0 -2 -2 -2
0 -2 -2 -2
Frequency domain

[ ]
34 2 6 6
= 2 2 2 2
-6 2 2 2
-6 2 2 2

=
 Fourier Transform in Numpy

 First we will see how to find Fourier Transform using Numpy.

 Numpy has an FFT package to do this. np.fft.fft2() provides us the frequency transform
which will be a complex array.
 Its first argument is the input image, which is grayscale.
 Second argument is optional which decides the size of output array. If it is greater than size
of input image, input image is padded with zeros before calculation of FFT. If it is less than
input image, input image will be cropped. If no arguments passed, Output array size will be
same as input.
 Fourier Transform in Numpy

 Now once you got the result, zero frequency component (DC component) will be at top left
corner.
 If you want to bring it to center, you need to shift the result by N/2 in both the directions.
 This is simply done by the function, np.fft.fftshift(). (It is more easier to analyze). Once
you found the frequency transform, you can find the magnitude spectrum.
Fourier Transform in Numpy
 Implementation Assignment!!
OCR (Optical Character Recognition)

//End//
 Template Matching

Goals:
 To find objects in an image using Template Matching
 You will see these functions : cv2.matchTemplate(), cv2.minMaxLoc()
 Template Matching

 Template Matching is a method for searching and finding the location of a template
image in a larger image.
 OpenCV comes with a function cv2.matchTemplate() for this purpose.
 It simply slides the template image over the input image (as in 2D convolution) and
compares the template and patch of input image under the template image.
 Several comparison methods are implemented in OpenCV.
 Template Matching

 If input image is of size (WxH) and template image is of size (wxh), output image will
have a size of (W-w+1, H-h+1).
 Once you got the result, you can use cv2.minMaxLoc() function to find where is the
maximum / minimum value.
 Take it as the top-left corner of rectangle and take (w,h) as width and height of the
rectangle. That rectangle is your region of template.
 Template Matching

Template Image

Input Image
Template Matching
 Template Matching

cv2.TM_CCOEFF cv2.TM_CCOEFF_NORMED
 Template Matching

cv2.TM_CCORR cv2.TM_CCORR_NORMED
 Template Matching

cv2.TM_SQDIFF cv2.TM_SQDIFF_NORMED
 Template Matching

 Template Matching with Multiple Objects:

 If there is an object which has multiple occurrences, cv2.minMaxLoc() won’t give you
all the locations.
 In that case, we will use thresholding.
 Here, we try to search the coins exist in Mario games!
 Template Matching

Template Image

Input Image After Detection

Template Matching