BILATERAL FILTERING

COMP – 6771 IMAGE PROCESSING

Submitted By –

NISHA BHATIA- 40220958, VAISHAKHI PATEL - 40233201

Bilateral Filtering

PART I – Review of the paper

Motivation - With contrast, in gaussian or regular filters, edges are lost after transformation and we acquire a new blend of foreground and
background pixels, which can often result in the loss of information. Typically, gaussian filtering computes the weighted average of the nearby
pixel values by calculating the spatial distance from the central pixel, where the weight decreases as we move away from the centre pixel. But what
if there are two domains in the image, and after transformation, a pixel in one of them has the same weight as a pixel in the other? As a result, edge
information may be lost, and pixels are mixed. Therefore, bilateral filters are introduced to smooth the images while maintaining the edges. They
measure the spatial distance as well as the photometric distance to assign the weights to their neighbouring pixels to get around this limitation.

We have also referred another paper on the same topic but having different approach and have compared both the approaches. The motivation
behind both the papers remains same that is to smooth the images and removing the noise. Other paper we have referred is Fast-adaptive Bilateral
filtering

Contribution – Bilateral Filters is often used in computational photography processes including tone mapping, style transfer, relighting, and
denoising because of its capacity to divide a picture into several scales without producing phantom colors after alteration. The main contribution of
the other paper (Fast-adaptive Bilateral Filtering) which we have refereed is to build a fast and effective algorithm which can help smoothing the
images by using less computational resources. This algorithm used in fast adaptive bilateral filters is 20x times faster than the brute-force
implementation. Basically, this approach is used for removing noise and to sharpen the images, but this can also be used for texture filtering, JPEG
deblocking and artifact removal.

Approaches - The main approach used in the study of bilateral filtering is combining both spatial and range filtering, which measures spatial
distance by a geometric closeness function and photometric distance by a similarity function, and then multiplies both to produce the altered image
of the input. Below is the given equation for bilateral filtering where c is geometric closeness function and s is similarity function.

Figure 1- Bilateral Filtering Figure 2 – Flow of the approached method

Fast-adaptive Bilateral Filtering – In the above paper, they have used fixed gaussian range filter along with spatial filter but here they have used
adaptive bilateral filter where centre and width of the kernel is allowed to change from pixel to pixel. The algorithm proposed in this paper based
on local histogram approximation using polynomials whose complexity does not scale with spatial filter width and based on that filtering is
performed purely in range space using local histogram. The amount of sharpening and noise removal at a particular pixel is controlled by adapting
the centre and width of the kernel. This algorithm does not involve any form of kernel approximation rather than that approximation is purely based
on histogram.

The method proposed in the bilateral filtering paper is very time consuming as it is using nonlinear combinations of nearby pixel values and
moreover depends on the range and domain filtering constant. Further, it requires twice computing cost to convolve as much as a domain filter of
the same size, therefore, to tackle this issue, they precomputed all the similarity function's values and used them to alter the image. Another way to
keep the edges is to compute the median in the filter rather than the mean, while K-means is frequently used in place of the mean in many
applications.

1
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

PART II – RE- IMPLEMENTATION (Bilateral Filtering)

We have implemented the bilateral paper using OpenCV library of the python and implemented the concept as shown below.

import os

import cv2

import math

import numpy as np

import matplotlib.pyplot as plt

def apply_gaussian_filter(img: np.ndarray, variance: float) -> np.ndarray:

sigma = math.sqrt(variance)

return 1 / (sigma * math.sqrt(2 * math.pi)) * np.exp(-((img / sigma) ** 2) * 0.5)

def get_slice(img: np.ndarray, x: int, y: int, kernel_size: int) -> np.ndarray:

half = kernel_size // 2

return img[x - half : x + half + 1, y - half : y + half + 1]

def get_gauss_kernel(kernel_size: int, spatial_variance: float) -> np.ndarray:

matrix = np.zeros((kernel_size, kernel_size))

for i in range(0, kernel_size):

for j in range(0, kernel_size):

matrix[i, j] = math.sqrt(

abs(i - kernel_size // 2) ** 2 + abs(j - kernel_size // 2) ** 2

return apply_gaussian_filter(matrix, spatial_variance)

def bilateral_filter(

img: np.ndarray,

spatial_variance: float,

intensity_variance: float,

kernel_size: int,

) -> np.ndarray:

"""

Bilateral filtering method implementation

2
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

Input:

img: A 2d image with values in between 0 and 1

spatial_variance: variance in space dimension

intensity_variance: variance in Intensity level

kernel_size: kernel size (Must be an odd number)

Output:

img: A 3d zero padded image with values in between 0 and 1

"""

filtered_image = np.zeros(img.shape)

gaussKer = get_gauss_kernel(kernel_size, spatial_variance)

if len(img.shape) == 3:

sizeX , sizeY , _ = img.shape

else:

sizeX, sizeY = img.shape

for i in range(kernel_size // 2, sizeX - kernel_size // 2):

for j in range(kernel_size // 2, sizeY - kernel_size // 2):

img_spatial = get_slice(img, i, j, kernel_size)

img_intensity = np.linalg.norm(img_spatial-img_spatial[kernel_size // 2, kernel_size // 2], axis=2)

img_gaussian = apply_gaussian_filter(img_intensity, intensity_variance)

weights = np.multiply(gaussKer, img_gaussian)

vals = (np.sum(img_spatial*np.stack((weights,weights,weights), axis=2), axis=(0,1)) / np.sum(weights))

filtered_image[i, j] = vals

return filtered_image

def bilateral_filter_image(

img,

3
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

spatial_variance,

intensity_variance,

kernel_size

):

out = img / 255

out = out.astype("float32")

out = bilateral_filter(out, spatial_variance, intensity_variance, kernel_size)

out = out * 255

out = np.uint8(out)

return out

if __name__ == "__main__":

filename = os.path.abspath("lena_noise.jpg")

img = cv2.imread(filename, 0)

out = bilateral_filter_image(

img,

spatial_variance=15.0,

intensity_variance=75.0,

kernel_size=75

f = plt.figure()

f.add_subplot(1,2, 1)

plt.title("Before: Original Image")

plt.imshow(img)

plt.show()

f.add_subplot(1,2, 2)

plt.title("After: Bilateral Filtered Image")

plt.imshow(out)

plt.show()

4
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

This is the output of the above implemented code before and after applying bilateral filter and we have observed that image got blurred after
smoothing and the noise is also removed while preserving the edges as well. We have performed this algorithm on black and white as well as on
RGB bands as well.

Before: Original Image After: Bilateral Filtered Image

FOR RGB IMAGE -

if __name__ == "__main__":

filename = os.path.abspath("taj.jpg")

img = cv2.imread(filename)

img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)

f = plt.figure(figsize=(10, 7))

f.add_subplot(1,2,1)

plt.title("Before")

plt.imshow(img)

plt.axis('off')

plt.show()

out = bilateral_filter_image(img,spatial_variance=3.0, intensity_variance=15.0, kernel_size=5)

f.add_subplot(1,2,2)

plt.title("After Bilateral Filter")

plt.imshow(out, cmap='gray')

plt.axis('off')

plt.show()

5
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

RGB Image Output –

Multiple iterations of bilateral filtering – Doing multiple iterations result in flattening the colors in an image but without blurring the edges. All
shadows and edges are preserved, but most of the shading is gone, and no “new” colors are introduced by filtering

filename = os.path.abspath("bunny.jpg")

img = cv2.imread(filename)

img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)

f = plt.figure(figsize=(10, 7))

f.add_subplot(1,2,1)

plt.title("Original img")

plt.imshow(img)

plt.axis('off')

plt.show()

out = bilateral_filter_image(img,spatial_variance=3.0, intensity_variance=5.0, kernel_size=5)

for i in range(4):

out = bilateral_filter_image(out,spatial_variance=3.0, intensity_variance=5.0, kernel_size=5)

f.add_subplot(1,2,2)

plt.title("Iter = 4")

plt.imshow(out)

6
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

plt.axis('off')

Output –

For getting difference in images, we have change value of sigma such as spatial domain variance and intensity variance.

filename = os.path.abspath("lena_noise.jpg")

img = cv2.imread(filename)

results = []

sigma_ds = [1,3,10]

sigma_rs = [10,30,100,300]

for sigma_d in sigma_ds:

for sigma_r in sigma_rs:

result = bilateral_filter_image(img, spatial_variance=sigma_d, intensity_variance=sigma_r, kernel_size=15)

results.append(result)

rows,cols = 3,4

fig, ax = plt.subplots(rows, cols)

for a, sigma_r in zip(ax[0], sigma_rs):

a.set_title(f"sig_r = {sigma_r}")

for a, sigma_d in zip(ax[:,0], sigma_ds):

a.set_ylabel(f"sig_d = {sigma_d}")

for i in range(rows):

for j in range(cols):

ax[i][j].imshow(results[i*cols+j],cmap='gray')

ax[i][j].axis('off')

7
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

plt.show()

𝜎𝑑 = 1

𝜎𝑑 = 3

𝜎𝑑 = 10

Pros and Cons of Bilateral filtering

Bilateral filters perform operation on three bands at once and then measure the distance between pixels in combined space so this helps in preventing
the edges to be blurred and reduces phantom colors appear in original image. But as they perform nonlinear operation it is very time consuming
and very expensive in terms of computational cost.

Difficulties in Implementation

8
 BILATERAL FILTERING | NISHA BHATIA, VAISHAKHI PATEL |40220958, 40233201

References

1. R. G. Gavaskar and K. N. Chaudhury, "Fast Adaptive Bilateral Filtering," in IEEE Transactions on Image Processing, vol. 28, no. 2,
pp. 779-790, Feb. 2019, doi: 10.1109/TIP.2018.2871597.
2. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," Sixth International Conference on Computer Vision
(IEEE Cat. No.98CH36271), 1998, pp. 839-846, doi: 10.1109/ICCV.1998.710815.
3. Sylvain Paris, Pierre Kornprobst, Jack Tumblin and Frédo Durand (2009), "Bilateral Filtering: Theory and Applications", Foundations and
Trends® in Computer Graphics and Vision: Vol. 4: No. 1, pp 1-73. http://dx.doi.org/10.1561/0600000020