Image De-Noising With Machine Learning A Review

This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2021.3092425, IEEE Access
Image De-noising with Machine Learning: A

Review
Rini Smita Thakur1, Shubhojeet Chatterjee1, Ram Narayan Yadav1, and Lalita Gupta1
1
Maulana Azad National Institute of Technology, Bhopal, Madhya Pradesh, India
Corresponding author: Rini Smita Thakur (e-mail: rinithakur66@gmail.com).
ABSTRACT Images are susceptible to various kinds of noises, which corrupt the pictorial information
stored in the images. Image de-noising has become an integral part of the image processing workflow. It is
used to attenuate the noises and accentuate the specific image information stored within. Machine learning
is an important tool in the image-de-noising workflow in terms of its robustness, accuracy, and time
requirement. This paper explores the numerous state-of-the-art machine-learning-based image de-noisers
like dictionary learning models, convolutional neural networks and generative adversarial networks for a
range of noises like Gaussian, Impulse, Poisson, Mixed and Real-World noises. The motivation, algorithm
and framework of different machine learning de-noisers are analyzed. These de-noisers are compared using
PSNR as quality assessment metric on some benchmark datasets. The best de-noising results for different
noise type is discussed along with future prospects. Among various Gaussian noise de-noisers, GCBD,
BRDNet and PReLU network prove to be promising. CNN+LSTM, and MC2RNet are most suitable CNN-
based Poisson de-noisers. For impulse noise removal, Blind CNN, and CNN+PSO perform well. For mixed
noise removal, WDL, EM-CNN, CNN, SDL, and Mixed CNN are prominent. De-noisers like GRDN and
DDFN show accurate results in the domain of real-world de-noising.
INDEX TERMS Convolutional Neural Networks, Dictionary Learning, Generative Adversarial Networks,
Image De-noising, Machine Learning
I. INTRODUCTION different types of noise based on pdf are Gaussian,

Rayleigh, Uniform, Impulse, Poisson, etc. According to the
Image de-noising has played a pivotal role in recent years correlation, noise is classified into white and color noise.
with the advent of many latest computer vision The white noise has uniform power spectral density and
applications. The digital image is prone to noise corruption zero autocorrelation, unlike color noise. If an image is
due to camera sensors, illumination level, transmission corrupted with white noise, it implies that all the pixels are
error, timing error of A/D converters, storage sensor faulty uncorrelated with each other. It is additive or multiplicative
memory location, capturing medium, transmission channel (speckle) according to nature, i.e., noisy pixels are added or
interference, and compression artifacts. In biological multiplied with the reference image. It is termed as
imaging, low-light conditions and shorter exposure time quantization noise or photon noise as per source
degrade the image quality [1]. Image restoration is required classification. The description of commonly used noise
in various fields such as medical imaging, remote sensing, types is given as follows:
underwater de-noising, dehazing applications [2]. The Gaussian Noise: It is statistical and additive in nature
different medical imaging modalities like computed which follows normal distribution with zero mean and 𝜎
tomography (CT), magnetic resonance imaging (MRI), X- standard deviation and affects all the pixels in the image.
ray, PET, etc. use appropriate de-noising methods for The cause of its occurrence is sensor temperature
proper diagnosis. Moreover, image pre-processing also fluctuation and environmental illumination variations. It is
includes de-noising procedure prior to the medical image commonly found in magnetic resonance imaging, and
classification or segmentation problem to attain higher confocal laser scanning microscopy imaging [3]. The
accuracy. Remote sensing de-noising restores relevant data probability distribution function of Gaussian noise is given
from synthetic aperture radar images, satellite images, by the following equation.
hyperspectral images, and underwater images. 1
(𝑥−𝜇)2
−
𝑝(𝑥) = 𝑒 2𝜎2 (1)
𝜎√2п
A. TYPES OF NOISES where 𝑥 is image pixel value, 𝜇 is mean and σ is the
The noise classification is done based on its probability standard deviation.
distribution function, correlation, nature, and its source. The
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
Spatial Domain Transform Domain Fuzzy based Machine Learning
Linear and Sparsity based dictionary

Data Adaptive Transform Switching
Non-liners learning
(ICA, PCA)
Gradient
Multilayer Perceptron
Non-Data Adaptive
Non-local
Transform (Wavelet Domain,
Spatial Domain)
Convolutional Neural
Weighted Average Networks
Generative Adversarial
Non-local Based Transform Networks
Domain (BM3D, BM4D)
FIGURE 1. Block Diagram illustrating classification of image de-noising techniques
Impulse Noise: It is an additive noise which occurs due to Rayleigh Noise: The noise in synthetic aperture radar
faulty sensors and transmission error. It affects only certain (SAR) images is granular in nature, and it is modeled by
pixels in the entire image, unlike Gaussian noise. It is Rayleigh distribution [8]. Sometimes, ultrasound images
divided into two parts, i.e., salt and pepper impulse noise are also prone to Rayleigh noise corruption. The Rayleigh
(SPIN) and random valued impulse noise (RVIN). In salt distribution is given by the following probability density
and pepper noise corruption, some image pixels take either −(𝑥−𝑎)2
2
maximum or minimum value of image dynamic range. 𝑝(𝑥) = {𝑏 (𝑥 − 𝑎)𝑒 𝑏 𝑓𝑜𝑟 𝑥 ≥ 𝑎 (5)
Whereas RVIN corruption changes some image pixels with 0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
a random value, its detection is more difficult than salt and Cauchy Noise: The atmospheric and underwater acoustic
pepper noise detection. The salt and pepper impulse noise is signals of radar and sonar imaging are corrupted with
given by [4] additive heavy-tailed impulse like noise, known as Cauchy
𝑃𝑎 𝑓𝑜𝑟 𝑥 = 𝑎 noise [9]. The probability distribution function of Cauchy
𝑝(𝑥) = {𝑃𝑏 𝑓𝑜𝑟 𝑥 = 𝑏} (2) distribution is given by:
1 𝛾2
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 𝑝(𝑥; 𝛿, 𝛾) = (6)
𝜋𝛾 𝛾2 +(𝑥−𝛿)2
where 𝑎 and 𝑏 are minimum and maximum pixel values of
where 𝛾 > 0 denotes the scale parameter, and δ∈ ℝ denotes
an image dynamic range. 𝑃𝑎 and 𝑃𝑏 are probabilities are
the localization parameter.
equal for salt and pepper noise.
Mixed Noise: In many real-life applications, images are
Poisson or Photon Noise: The Poisson distribution is used
corrupted by more than one noise type. The mixture of
to model photon noise caused by the photon’s random
Gaussian and impulse noise is found in computed
arrival on the image sensor [5]. The applications of Poisson
tomography (CT) images and cDNA microarray imaging
noise removal include astronomy, medical imaging, and
[10], [11]. The mixed noise in cDNA microarray imaging
low-light photography. The conditional probability of
occurs due to photon and electronic noise interaction, dust
Poisson distributed image 𝑦 for clean image 𝑥 is given by
particles on surface of glass slides, and laser reflection. In
[6]
−𝑥𝑖,𝑗 𝑦 hyperspectral images, the combination of signal
𝑒 𝑥𝑖,𝑗 𝑖,𝑗
𝑝(𝑦 |𝑥) = ∏𝑁
𝑖,𝑗=1 (3) independent additive Gaussian noise and signal dependent
𝑦𝑖,𝑗 !
multiplicative Poisson noise is found [12].
where 𝑖, 𝑗 denotes pixel indices.
Gamma Noise: The speckle noise in ultrasound images
B. CLASSIFICATION OF IMAGE DE-NOISING
occurs due to coherent imaging mechanisms from the TECHNIQUES
scatters [7]. It reduces the image sharpness and creates The image de-noising methods can be grouped into spatial
difficulty for lesion diagnosis. It is modeled by Gamma domain techniques, transform domain techniques, fuzzy
distribution, whose probability distribution function is filtering-based techniques, and machine learning techniques
given by the following equation. [13], [14]. The block diagram illustrating the classification
𝑎𝑏 𝑥 𝑏−1
−𝑎𝑧 of image de-noising techniques is given in Fig.1.
𝑝(𝑥) = { (𝑏−1)! 𝑒 , 𝑥 ≥ 0 (4)
The spatial domain filtering is widely used for image
0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
where parameters 𝑎 and 𝑏 are positive integers. restoration in which filtering operation is directly applied to
the image pixels. They are further divided into linear and
2
non-linear filters. The most common linear filters are the directions. The fuzzy impulse noise detection and reduction
mean filter, Gaussian filter, and Weiner filter. The basic method calculate the gradient in eight directions for noisy
mean filter replaces the particular pixel of operation with pixel detection prior to the filtering [25]. In histogram fuzzy
the mean value from the pre-defined neighborhood. de-noising filters, the membership function is derived from
Similarly, Gaussian filters use a Gaussian kernel with a the input histogram [26]. It consists of the fuzzy detection
particular mean and deviation. They suffer from the phase and cancellation phase. A detailed explanation of
problem of over-smoothening and blurring of edges. To fuzzy-based techniques is given in [27], [14].
overcome this problem, Wiener filter was introduced but it The image de-noising models can be grouped into
is also unsuccessful while operating on sharp edges. Later, analytical models (stochastic and deterministic) and
non-linear filters were introduced in which output is a non- machine learning-based models. In analytical models,
linear function of input for edge, detail, and texture forward de-noising model is explicitly known to the user,
preservation. The primary examples of non-linear filters are and the solution approach is used based on certain chosen
total variation filters, anisotropic diffusion filters, bilateral criteria. The deterministic modeling of spatial filters is
filter, and fourth-order partial differentiation filter. The challenging for each image type. The edge deterioration and
bilateral filter replaces pixel value with neighborhood blurring are common artifacts in spatial and transform
weights which are function of both Euclidian distance and domain techniques. On the other hand, in the machine
range difference [15],[16]. The detailed comprehensive learning models, the inverse model is learned with the help
review of impulse and Gaussian de-noising filters is given of image datasets containing clean and noisy image pairs.
in [14]. The most important question arises: what is the relative
The transform domain techniques convert the image into advantage of machine (deep) learning approach over
the transform domain, and then mathematical operations are analytical methods? In deep learning models, computational
carried out on transform domain coefficients. It is followed burden exists in the learning phase, whereas the testing
by inverse transform to restore de-noised image. These phase consists of a feed-forward model. Whereas analytical
techniques are divided into data-adaptive and non-data methods rely on a computationally demanding optimization
adaptive techniques based on the transform basis function. process and heuristic selection of hyper-parameters is not
The independent component analysis (ICA) and principal the solution for getting good de-noising results. It has been
component analysis (PCA) are data-adaptive transform observed that machine learning models give superior
methods. The ICA is successfully utilized for non-Gaussian performance compared to analytical methods, as feature
de-noising. PCA is a de-correlation method that transforms learning makes a single model apt for considerable
the original image dataset into the PCA domain and selects variation in the noise level.
the most significant principal components (maximum Some de-noisers are based on an analytical optimization,
Eigen-vectors) for image restoration [17]. Wavelet-based which involves iterative process based on some stopping
image de-noising is a multi-resolution image analysis criteria. Although, analytical optimization is involved but it
technique that uses different mother wavelets such as cannot be directly categorized in the machine learning
Daubechies, Haar etc., to obtain wavelet coefficients. It has domain which is basically a numerical optimization
been used to de-noise Gaussian, salt and pepper, and problem. Some of the important analytical optimization
Poisson noise using the appropriate thresholding operator methods are total variation regularization [28] and weighted
[18], [19]. In recent years, the most promising non-local nuclear norm minimization (WNNM) [29]. Variational-
means, collaborative filtering method in the transform based methods find the appropriate priors such as low-rank
domain is block-matching and 3D filtering (BM3D) [20]. In priors, non-local self-similarity priors, sparse priors, low-
this approach, similar 2D image patches are compiled into rank priors, and gradient priors. WNNM assigns the weight
3D groups by the block matching process. The to the singular value of an image and analytical
collaborative Wiener filtering is done in the transform optimization is done based on some energy function.
domain on this 3D group. The improved versions of BM3D In recent years, there is a paradigm shift from analytical
are given in [21], [22]. Curvelet filter is based on theory of models to machine learning models owing to improved
multiscale geometry (i.e., position, scale, and orientation image quality assessment metrics. In the following section,
usage). It gives a better de-noising performance on edges machine learning-based image de-noisers are explained in
and borders than state-of-the-art wavelet de-noising detail. In this paper, the following convention is followed in
methods [23]. It uses ridgelet transform as a primary step, explaining methods: 𝒚 is the noisy input image, 𝒙 is the
and curvelet sub-bands are formed with a filter-bank clean image or ground-truth image, 𝒗 is the noise
structure formed by trous wavelet filters. The 2-D component added 𝒙 to generate 𝒚, and the final predicted
contourlet transform provides spatial and directional de-noised image from de-noiser is 𝒙 ̂.
resolution to keep contours and details intact [24].
The image restoration using fuzzy-based methods considers C. MACHINE LEARNING BASED IMAGE DE-NOISING
the image as a fuzzy set and its pixel values as its member. The machine learning image de-noising techniques have
Fuzzy-based filters use fuzzy rules to design membership made considerable progress with introducing benchmark
functions by calculating the gradient’s degree in various datasets for a particular application, deep learning
advancements, and increased computational power with intermediate hidden layers. The general equation of MLP
Graphical Processing Unit's (GPU's). They are further network with two hidden layers is given by
broadly classified into sparsity-based dictionary learning Algorithm 2: De-noising algorithm of multi-layer
models, multi-layer perceptron models, convolutional perceptron-based de-noiser [35]
neural network-based models, and generative adversarial 1. Multi-layer perceptron (MLP) (supervised) learns a
network-based models. function 𝑓(. ): ℝ𝑚 → ℝ𝑜 by training on a dataset,
1) SPARSITY-BASED DICTIONARY LEARNING MODELS where 𝑚 and 𝑜 are the input dimensions and the output
In the sparsity-based techniques, every image patch is dimensions, respectively
constituted as a linear combination of several patches from 2. Given 𝒀 = {𝑦1 , 𝑦2 , 𝑦3 , … , 𝑦𝑚 } and a target 𝑥, it learns a
an overcomplete dictionary 𝑫. The image encoding is done non-linear function approximator for either regression
with the coding vector 𝜶 over a complete dictionary and 𝑙1 - or classification
norm sparse regularizer on coding vector 𝜶, i.e., 3. Features 𝒀 are input to the MLP architecture, which
𝑚𝑖𝑛 ‖𝜶‖
𝜶 1 𝑠. 𝑡. 𝒙 = 𝑫𝜶, following a generalized model given has an input layer, one or more non-linear hidden
by [30], [31]: layers, and an output layer that outputs 𝑓(𝒀)
̂ = 𝑎𝑟𝑔 𝑚𝑖𝑛𝜶‖𝒚 − 𝑫𝜶‖22 + 𝜆‖𝜶‖1
𝜶 (7) 4. Each neuron in the hidden layer transforms the values
Here, 𝜆 is a sparseness-balancing regularization parameter, from the previous layer as 𝑔(𝑤1 𝑦1 + ⋯ + 𝑤𝑚 𝑦𝑚 ),
and ‖𝜶‖1 is 𝜶′𝑠 1-norm. Another design of the model uses where 𝑔(. ): ℝ → ℝ is a non-linear activation function
‖𝜶‖0 (𝜶′𝑠 0-norm) in place of ‖𝜶‖1 . K-Singular Value
like hyperbolic tan function
Decomposition (K-SVD) technique is the pioneering work
5. The output layer transforms the values received from
that uses dictionary learning to frame the sparse
the hidden layer into output values
representation model. The learning of this model can take
place from the benchmark datasets as well as from the input Sum Tanh
image by K-SVD [32]. The K-SVD is the iterative process
in which two consecutive steps take place sparse coding of Weights
Weights
the examples using the current dictionary and updating the
dictionary atoms for optimum data fitting. Some other Sum
works as in [33], [34] follow the same workflow like that
of K-SVD with variation in dictionary and optimization
problem. The clustering-based sparse representation
involves a cost function (double header 𝑙1 optimization
problem) in which both structural structuring and dictionary
learning is used as the regularizer. A typical sparsity-based
image de-noising algorithm is given in Algorithm 1. Output
Algorithm 1: De-noising algorithm of sparsity-based de- Input Layer
noiser [30] Layer
1. Input: 𝒚, where 𝒚 is the image observed in the noisy First Hidden Layer
environment FIGURE 2. Multi-layer Perceptron Network [35]
2. Find 𝒙 ̂ = 𝑫𝜶̂ , where 𝑫 is a sparse dictionary
̂ is the sparsity constraint, and
constructed to suit 𝒙, 𝜶 ̂ = 𝒃𝟑 + 𝒘𝟑 𝑡𝑎𝑛ℎ (𝒃𝟐 + 𝒘𝟐 tanh(𝒃𝟏 + 𝒘𝟏 𝒚))
𝒙 (8)
𝜶 is an unknown parameter, 𝜆 is a sparseness- where 𝒘 is the weight matrix, 𝒃 is vector-valued bias, and
balancing regularization parameter set according to: an activation function is 𝑡𝑎𝑛ℎ, which operates component-
1
𝐿 = ‖𝒚 − 𝑫𝜶‖22 + 𝜆‖𝜶‖0 (‖𝜶‖0 may be replaced wise. The stochastic gradient is used for training with noisy
2
by ‖𝜶‖1 ) and clean image pairs. The parameters of MLP are updated
Such that 𝐿 is as low as possible by back-propagation, minimizing the mean-square error. To
3. Find the estimate of 𝒙 according to the following: increase the training efficiency, data normalization, proper
̂ = 𝑎𝑟𝑔 min‖𝜶‖0 𝑠. 𝑡. ‖𝒚 − 𝑫𝜶‖2 < 𝛬, 𝒙 = 𝑫𝜶
𝜶 ̂, weight initialization, and learning rate division is done. The
𝜶
noisy image is broken into overlapping patches, and each
where 𝛬 is a small-value limiting parameter
patch is de-noised separately. MLP estimates the de-noised
4. Solve the above non-deterministic polynomial
version of the overlapping noisy patches, and then the
problem by using greedy pursuit or convex relaxation
average is calculated for overlapped de-noised patches [35].
5. ̂
Output: 𝒙
There is an improvement in de-noising performance when
2) MULTI-LAYER PERCEPTRON MODELS de-noised patches are weighted by the Gaussian window.
The multi-layer perceptron (MLP) network, as shown in The MLP with four hidden layers uses time-series images
Fig. 2, is the feed-forward model that maps the input image and has shown significant improvement in keeping details
vector (𝒚) with the output image vector (𝒙 ̂) with several and edges intact for SAR images [36]. The trainable non-
linear reaction-diffusion model [37] is a feed-forward
architecture that embeds a standard non-linear diffusion noised version is the fundamental cost function. Fig. 4
model in the neural network. The number of layers of MLP illustrates the basic architecture of CNN. Algorithm 3 gives
is less because of vanishing gradient compared to CNN de-noising process.
convolutional neural networks that limit their performance.
A multi-layer perceptron de-noising algorithm is given in Algorithm 3: De-noising algorithm of CNN based de-
Algorithm 2. This algorithm is in accordance with Fig. 3, noiser [38]
which depicts a single hidden layer MLP. 1. Input noisy image 𝒚, noise standard deviation 𝜎 and
clean image 𝒙
+𝑏 i.e. 𝒚 = 𝒙 + 𝒗
2. CNN Module: Input: input image or image plus noise
+𝑏 level maps
𝑦1 Intermediate units: Convolution + Batch Normalization
𝑂𝑢𝑡𝑝𝑢𝑡 + Activation function
𝐹𝑒𝑎𝑡𝑢𝑟𝑒𝑠 (𝒀) 𝑎1 Output unit: Convolution+ Residual learning
𝑓(𝒀) 3. Intermediate output is feature maps given by:
𝑦2 𝑭𝑴𝑗𝑖 = 𝐴(∑𝑖𝜖𝑀𝑗 𝑭𝑴𝑙−1 ∗ 𝒘𝑗𝑖𝑙 + 𝒃𝑗𝑙 ), where 𝑭𝑴𝑙−1
𝑖 𝑖
𝑎2 represents the feature map of 𝑙 − 1 layer, 𝒘𝑗𝑖𝑙 and 𝒃𝑗𝑙 are
𝑦3 weight and bias of 𝑙 layer, 𝐴 is activation function, 𝑀𝑗
is selection operator of feature maps.
4. Residual learning implies 𝒙 ̂ = 𝒚 − 𝑅(𝒚) , where 𝑅
FIGURE 3: One hidden-layer MLP
represents residual learning CNN operator.
5. 1 2
3) CNN-BASED DE-NOISING MODELS Loss function: 𝑙(Ɵ) = ∑𝑁 ‖𝑅(𝒚𝑖, Ɵ) − (𝒚𝑖 − 𝒙𝑖 )‖
2𝑁 𝑖=1
In recent years, the convolutional neural network (CNN) , where Ɵ denotes CNN parameters, 𝑁 are the number
based models have shown significant improvement in of images in the training dataset, 𝒚 and 𝒙 represent a
various image quality metrics compared to other state-of- noisy and clean image, 𝑅 is residual learning.
the-art methods [39]. The success of CNN models can be 6. If 𝑙(Ɵ) ≅ 0, model is trained, else retrain for the next
attributed to large modeling capacity and significant epoch.
advancement in network training and design. The CNN is
designed for grid or matrix kind of data as input taking
inspiration from the visual cortexes of animals. In CNN Input O/p Image
models, the convolutional kernel with learnable parameters Image 256 64 FM
256 x 256
64 FM
x 256 (RGB)
is shared across all image positions. The convolutional (RGB) 256 × 256 x
kernel can be visualized as a feature extractor for a 256 256
particular image restoration application. The convolutional
layers have a cascade connection, so extracted features
become more complex, hierarchically, and progressively. 3 Conv filters
64 Conv 64 Conv
CNN model consists of an input layer, series of filters3x3x3 filters 3x3x64
intermediate hidden layers, and the output layer. The Bias 3x3x64 Bias
convolutional kernel with learnable weights is applied on (1 x 1 x 64) Bias (1 x 1 x 3)
each layer, followed by some activation function. The (1 x 1 x 64)
output of each layer is fed as the input of the next one. The
output of intermediate layers is termed as feature maps. The
general equation of the intermediate feature maps (𝐹𝑀) of Desired Output
𝑙 𝑡ℎ layer of CNN is given by [40] Image
256 x 256 (RGB)
𝑭𝑴𝑗 = 𝐴(∑𝑖𝜖𝑆𝑗 𝑭𝑴𝑙−1 𝑖 ∗ 𝒘𝑗𝑖𝑙 + 𝒃𝑗𝑙 ) (9) Obj Func.
c(.) = || . ||22
where 𝑺𝑗 represents selection of the input feature
map, 𝑭𝑴𝑙−1 𝑖 is the previous feature map, 𝒘𝑗𝑖𝑙 is the weight FIGURE 4: CNN Architecture for image restoration [39]
of the convolution kernel of the 𝑙 𝑡ℎ layer, 𝐴 is the activation
function which can be a rectified linear unit, sigmoid
function etc. and 𝒃𝑗𝑙 is the bias in the 𝑙 𝑡ℎ layer. The training 3) GAN-BASED DE-NOISING MODELS
procedure involves optimizing parameters such as kernels The generative adversarial network (GAN) uses generative
by using clean and noisy image labels with stochastic modeling with two sub-models, termed generator and
gradient descent, Adam’s algorithm, etc. The Cost function discriminator [41]. This network is designed to overcome
optimization takes place during the training process. The deep generative model difficulty of learning complex
mean square error between the clean image and its de- probabilistic distributions. The generator model is used for
extracting new plausible images from the problem domain,
5
whereas the discriminator determines whether the generated 𝜕 1

image samples are real or fake. The discriminator model [𝑙𝑛(1 − 𝐷(𝐺(𝒚))]
𝜕𝜃𝐺 𝑚
acts as an adversarial network. The main motive of the
generator network is to obtain image samples that can For fixed 𝐷, 𝑚𝑖𝑛𝐺 𝑉 = 2𝐽𝑆(𝑝𝒙 ||𝑝𝒈 ) − 2𝑙𝑛2 is
disguise the discriminator network. Usually, the generator evaluated, where 𝐽𝑆(. ) is known as Jensen-Shannon
network maps the noisy image with the ground-truth, and divergence.
the discriminator network uses the loss function to find the 7. Training of GAN ends when 𝐽𝑆(𝑝𝒙 ||𝑝𝒈 ) becomes 0,
difference between the output image of the generator and i.e., 𝑝𝒙 = 𝑝𝒈 and 𝑚𝑖𝑛𝐺 𝑚𝑎𝑥𝐷 𝑉(𝐺, 𝐷) = −2𝑙𝑛2
the ground truth. The discriminator finds whether the
predicted image by generator output 𝒙 ̂ = 𝐺(𝒚) is real or
Noisy Image Generator Fake Images
fake. The other de-noising methodology involves the
extraction of noise blocks from the input noisy images with
the GAN. Thereafter, generated noise blocks along with
clean images form a training dataset for CNN to produce Real Images Discriminator
de-noised output [42]. The following equation represents
GAN objective function :
min 𝑚𝑎𝑥
𝑉(𝐷, 𝐺) = 𝐸𝒙~𝑝𝑑𝑎𝑡𝑎(𝒙) [𝑙𝑜𝑔𝐷(𝒙)] + FIGURE 5: Basic GAN Architecture for image restoration
𝐺 𝐷
𝐸𝒚~𝑝𝒚(𝒚) [log (1 − 𝐷(𝐺(𝒚)))] (10) II. MACHINE LEARNING BASED GAUSSIAN DE-
NOISERS
where 𝐷(𝒙) is the discriminator model, 𝐺(𝒚) is the
generator model, 𝑝𝑑𝑎𝑡𝑎(𝒙) is the real data distribution, The Gaussian noise de-noisers are used in many important
applications such as MRI de-noising, optical coherence
𝑝𝑦 (𝒚) is the generated data distribution (i.e., input noisy
tomography images, natural images, etc. The CNN-based
image 𝒚), and 𝐸 is the expected output. Fig. 5 shows the
machine learning model has shown tremendous
architecture of GAN for image restoration. Algorithm 4 improvement in image quality assessment metrics as
gives GAN de-noising pseudocode. TABLE I gives compared to other state-of-the-art networks. The CNN
advantages and disadvantages of different machine learning based de-noisers have excelled in Gaussian noise de-
image de-noisers. noising but real world image de-noising is still a
Algorithm 4: Image de-noising for GAN-based image de- challenging problem. The Gaussian de-noisers are being
noiser [41], [42] designed for adaptive white Gaussian, spatially variant
1. Input noisy image (𝒚) to Generator 𝐺; 𝑝𝒚 (𝒚) being the Gaussian noise, and blind Gaussian noise. The two
noise distribution. important benchmark datasets BSD-68 and Set-12 are used
2. Generator generates reconstructed image data 𝐺(𝒚) for comparative analysis for Gaussian de-noisers. The
with distribution 𝑝𝒈 (𝒚) CNN-based models follow discriminative learning while
3. Pass original image data (𝒙) with distribution 𝑝𝒙 (𝒙) GAN’s are generative learning models. CNN models
and reconstructed image data 𝐺(𝒚) to the require supervised learning, i.e., the availability of noisy
discriminator and clean image pairs in training datasets. In the recent
4. Discriminator 𝐷 outputs the probability of the input works, we are progressing towards unsupervised learning
belonging to the original data due to the lack of clean-noisy image pairs for real-world
5. 𝐺 and 𝐷 play a two-player minimax game in an applications. The training of deep learning models involves
adversarial setup, in which 𝐺 and 𝐷 try to minimize optimization of the loss function, which consists of data
and maximize a value function based on binary cross- fidelity term and regularizar. The different de-noiser
entropy function 𝑉(𝐺, 𝐷) = 𝐸𝒙~𝑝𝒙 [𝑙𝑛𝐷(𝒙)] + variants are designed by changing the loss function, the
𝐸𝒚~𝑝𝒚 [𝑙𝑛(1 − 𝐷(𝐺(𝒚))] respectively number of layers, training dataset size, activation functions,
6. Training of GAN: optimize the value function as and so on. The following section describes some
𝑚𝑖𝑛𝐺 𝑚𝑎𝑥𝐷 𝑉(𝐺, 𝐷) benchmark machine learning models which are used for
image de-noising:
a) Fix the learning of G: update generator
parameters (θD) by gradient ascent using 𝑚
A. METHODOLOGIES OF DICTIONARY LEARNING
data samples and 𝑚 fake samples as
MODELS (GAUSSIAN NOISE)
𝜕 1
[𝑙𝑛𝐷(𝒙) + 𝑙𝑛(1 − 𝐷(𝐺(𝒚))] The dictionary learning models achieve sparse
𝜕𝜃𝐷 𝑚 representation by updating the dictionary with the training
For fixed G, 𝑉(𝐺, 𝐷) will be maximum for images. The fixed dictionary is limited to a specific type of
𝑝𝒙 (𝒙)
𝐷(𝒙) = images, whereas atoms of the learned basis dictionary are
𝑝𝒙 (𝒙)+𝑝𝒈 (𝒚)
empirically learned for any family of images. The learned
b) Fix the learning of D: update discriminator dictionary provides more efficient image priors for
parameters (θG) by gradient descent using 𝑚 Bayesian estimation as compared to the fixed dictionary.
fake samples as The Gaussian dictionary learning techniques are K-SVD
6
Input Sparse Dictionary De-noised

Noisy Coding Step Update Step Output Algorithm 5: De-noising algorithm of sparsity-based
Image Image dictionary learning models [46]
1. Input image 𝒀 = 𝑨 = {𝐴1 , 𝐴2 , … , 𝐴𝑛 } (𝒀 is expressed
as a set of 𝑨 signals)
Fix Fix sparse 2. De-noise 𝒀 via an ill-posed problem of finding patches
dictionary representatio 𝒁, where 𝑨 = 𝒁 + 𝒗
𝑫, find n vector 𝜶, 3. De-noised image patches 𝒁 are obtained with the help
sparse update of an optimization problem, which aims at minimizing
representatio dictionary 𝑫
n vector 𝜶 the following cost function:
̂, 𝒁
𝑿 ̂ = arg 𝑚𝑖𝑛𝑿,𝒁 ‖𝒁 − 𝑨‖22 + 𝛽‖𝑫𝑿 − 𝒁‖2𝐹
FIGURE6: Flow Chart of K-SVD algorithm [32] + ∑ 𝜇𝑖 ‖𝒙𝑖 ‖0
𝑖
4. It is solved in terms of smaller optimization problems
[32], locally learned dictionary (KLLD) [43], non-local defined as:
hierarchical learning with wavelets [44], mean-corrected ̂ = arg 𝑚𝑖𝑛𝒙 ,𝒁 ‖𝒁 − 𝑨‖22 + 𝛽‖𝑫𝒙𝑖 − 𝑹𝑖 𝒁‖2𝐹 +
𝒙̂𝑖 , 𝒁 𝑖
atoms dictionary learning [45]. The K-SVD algorithm finds
∑𝑖 𝜇𝑖 ‖𝒙𝑖 ‖0 , where 𝑹𝑖 selects 𝑖 𝑡ℎ particle from 𝒁, i.e.,
the best dictionary for the 𝑁 image samples by solving the
𝒛𝑖 = 𝑹𝑖 𝒁, and 𝛽 is a parameter that depends on noise
following sparsity equation in which dictionary 𝑫 is
variance
initialized with 𝑙 2 normalized columns
𝑚𝑖𝑛 2 5. The cost function minimizes the error between the
𝑫,𝜶||𝒚 − 𝑫𝜶||𝐹 such that ||𝜶𝑖 ||0 ≤ 𝑇0 (11) restored image and the input noisy image, under the
In the above equation, 𝑇0 is the no. of non-zero entries in assumption that each patch in the input image can be
representation vector 𝜶. The two iterative steps are the represented as a sparse linear combination of patches in
sparse coding stage and the codebook or dictionary update the dictionary 𝑫
stage. In the sparse coding step, pursuit algorithm is used 6. The closed form solution of the above optimization
for the computation of the representation or sparse vector 𝜶 problem:
for each input image 𝒚, by solving the following equation 𝛽𝑨 + ∑𝑖 𝑹𝑖 𝑫𝒙𝑖
𝑚𝑖𝑛 2 ̂=
𝒁
𝛼𝑖 ||𝒚𝑖 − 𝑫𝜶𝑖 ||2 such that ||𝜶𝑖 ||0 ≤ 𝑇0 (12) 𝛽𝑰 + ∑𝑖 𝑹𝑻𝑖 𝑹𝑖
The next step is the dictionary update stage. Out of 𝐾
columns of the dictionary, i.e., 𝐾 atoms, each atom is B. METHODOLOGIES OF CNN-BASED MODELS
updated considering one at a time. The image input (GAUSSIAN NOISE)
examples that use a particular atom are retained, the rest of
the examples are discarded. The contribution of other atoms The de-noising CNN ( DnCNN) is the benchmark de-noiser
is also subtracted from the representation vector. Now, the which is being used for image restoration tasks like image
overall representation error is minimized by singular vector super-resolution and JPEG image de-blocking apart from
decomposition (SVD) to update the dictionary. The the Gaussian image de-noising [38]. DnCNN model
flowchart of K-SVD algorithm is given in Fig. 6. overcomes the disadvantage of trainable non-linear
KLLD algorithm involves clustering, dictionary selection, reaction-diffusion (TNRD) and a cascade of shrinkage
and coefficient calculation. In clustering local, features fields [47], which uses specific priors based on the analysis
capture the local structures of the image data. The next step model. So, the priors fail to capture image structures
that is dictionary selection is the optimization according to effectively. Moreover, many handcrafted parameters are
the clustering done in the first step. In the final step, used during stage-wise greedy training in combination with
coefficients are calculated for dictionary atoms, which are joint fine-tuning.
linearly combined subjected to the kernel weights. The non- The additive noise (𝒗) is combined with the clean image
local hierarchical dictionary learning is achieved by sparsity (𝒙) to form the noisy image (𝒚). The DnCNN model uses
and multiresolution analysis of wavelets in each residual learning with the batch normalization module. In
decomposition level. Recently, K-SVD algorithm is residual learning, the CNN learns the noise component
modified based on dictionary learning with mean corrected instead of a de-noised image. The residual learning model
atoms, which outperformed K-SVD in terms of PSNR. In is given by 𝑅(𝒚) ≈ 𝒗, and the desired output image is 𝒚 −
[46], the authors propose to break up the noisy image into 𝑅(𝒚). The batch normalization achieves faster training by
patches and treat the vectorized version of each patch as mini-batch stochastic gradient by reducing the internal co-
signals, thereby restricting the dimensionality of each atom variate shift. It is implemented by normalization and scale +
in the dictionary. The size of the patch is chosen to allow shift step before non-linearity in each layer. In the 𝑙 depth
for enough details of the underlying signal. Overlapping network, the first layer is a convolutional layer with ReLU
patches are chosen to reduce blocking artifacts that might activation, which uses sixty-four filters (3 x 3 x no. of
result at the boundaries. Dealing with patches as signals, the image channel) to give sixty-four feature maps as output.
K-SVD algorithm can be effectively scaled to de-noise The intermediate layers are repeated units of convolution
large images. The algorithm is given in Algorithm 5. (sixty-four filters of size 3 x 3 x 64) and
7
Input First layer Intermediate Last layer Output

layers
Downsampled sub- Conv+ ReLU
images and noise level (FFDNet, Conv+BN+ReLU
map M DnCNN, Conv Residual
(FFDNet, DnCNN)
(FFDNet) IDCNN) (FFDNet, Image
DnCNN, (DnCNN,IDC
IDCNN, NN, SCNN
SCNN)
Conv + Soft Conv+ ReLU
Noisy Image (DnCNN, Shrinkage (IDCNN) De-noised
IDCNN, SCNN) (SCNN) subimages
(FFDNet)
Conv + BN +Soft
Shrinkage (SCNN)
FIGURE 7: Block diagram depicting the architecture of DnCNN, IDCNN, SCNN and FFDNet
ReLU activation with batch normalization. The concluding the computational cost and enhances the extraction of more
layer is a convolutional layer which uses same number of context information. Batch Normalization Residual
filters as the number of image channels of size 3x3x64. Network (BRDNet) also uses residual learning, batch re-
The model gives same de-noising results with both normalization, and dilated convolution to address the
stochastic gradient descent and Adam’s algorithm by problem of internal co-variate shift for extraction of more
optimizing the following loss function: features [51]. Deep iterative down-up CNN (DIDN),
1 2 densely connected hierarchial denoising network (DHDN)
𝑙(Ɵ) = ∑𝑁 ‖𝑅(𝒚𝑖, Ɵ) − (𝒚𝑖 − 𝒙𝑖 )‖ (13)
2𝑁 𝑖=1 [52], and multi-level wavelet CNN (MWCNN) [53] are
where Ɵ denotes DnCNN parameters, 𝑁 are the number of
based on UNet [54] architecture which was designed for the
images in training dataset, 𝒚 and 𝒙 represent a noisy and
semantic segmentation. The deep iterative down-up CNN
clean image, 𝑅 is residual learning. There are other de- (DIDN) [55] is also based on receptive field size variation
noisers variants whose basic architecture resembles with for improving de-noising results. It consists of four stages:
DnCNN network. Wavelet de-noising CNN i.e., WDnCNN initial feature extraction, down-up block, reconstruction,
[48] uses residual learning in the novel feature space of the and enhancement. The initial feature maps are extracted by
wavelet domain. In this method, the network is trained convolution followed by iterative up and down sampling of
with four decomposed wavelet sub-bands, and the feature maps by down-up block. The outputs of all the
architecture is the same as that of DnCNN. SCNN [49] is down-up blocks are fed into the reconstruction block, which
residual learning-based model which uses soft shrinkage has convolutional and parametric rectified linear units. The
activation function for varying noise levels of the input concatenated output of the reconstruction block is fed into
image. an enhancement block with a convolution unit. DHDN
IDCNN [40] is another deep convolutional neural network network uses modified Unet architecture to learn large
that follows the same residual learning architecture as that number of parameters, solves vanishing gradient by
of DnCNN without incorporating batch normalization. This residual learning and dense connectivity to convolution
network fails to converge with stochastic gradient descent layers. In MWCNN, multiwavelet transform is integrated
because of the gradient explosion. So, this network clips the into UNet architecture to increase the receptive field size by
gradient in the specific pre-defined interval, i.e., gradient reducing the resolution of feature maps.
clipping procedure. It has been observed that network The fast and flexible de-noising convolutional neural
performance improves as the depth of the network increases network (FFDNet) is the fastest in terms of implementation
from four to ten. In this model, a non-fixed noise mask is time, and it can handle spatially variant Gaussian noise
used during the process so that a single model can be used [56]. The unique feature of this model is that unlike other
for different noise levels. The loss function of IDCNN is networks, the mapping function contains a noise level map
given by in the input. The noise level map plays a crucial role in
𝑙(Ɵ) = ‖𝒙 − 𝒙 ̂‖2 (14), where 𝒙 and 𝒙 ̂ denote clean and keeping the trade-off between noise reduction and detail
estimated images respectively. preservation. Conventionally, the mapping function learns
The ECNDNet [50] is a residual learning model which de-noised images from noisy images, CNN parameters, and
follows the loss function given in equation (13). The Gaussian noise standard deviation. In FFDNet, the CNN
architecture is the same as that of DnCNN. The main parameters are not affected with variation in Gaussian noise
feature of the ECNDNet network is the usage of dilated level. It works on downsampled sub-images, which tend to
convolution to increase the receptive field size. It reduces increase the receptive field. The architecture of FFDNet has
the same units as that of DnCNN, i.e., convolutional
Input Output
Noisy Image
SB
Image
FEB AB
Conv+BN+ReLU Conv Cat Tanh
FIGURE8: Architecture of ADNet [64]
operator in the first layer, repeated units of convolution, multiple de-noisers modules interleaved with back-
batch normalization, and ReLU activation, concluded with projection (BP) that ensure the observation consistencies.
the convolutional layer. The Adam’s algorithm [57] is used DRUNet [63] is the improved version of IRCNN and its
for training to minimize the following loss function. methodology involves usage of CNN as deep denoiser prior
1 2
𝑙(Ɵ) = ∑𝑁 ‖𝐹(𝒚𝑖, 𝑴𝑖, Ɵ) − (𝒙𝑖 )‖ (15) accompanied by half quadratic splitting based iterative
2𝑁 𝑖=1 algorithm for solving deblurring, super-resolution,
where 𝐹 denotes FFDNet learning function and 𝑴 is noise
denoising and color image demosacking.
level map.
Recently, the attention-guided de-noising convolutional
Some models like NN3D [58] and graph CNN [59] which
neural network (ADNet) [64] has outperformed all previous
exploits non-local and local similarities through non-local
CNN’s. They are specifically designed to overcome the
filter and graphical signal processing. The NN3D uses
disadvantage of increment in network length. As the length
standard pre-trained CNN in cascade connection with
of the network increases, the influence of shallow layers
standard non-local filter. The DnCNN, IDCNN, and
becomes weak in de-noising performance. It is divided into
FFDNet focus towards local features with biased receptive
four major modules; sparse block (SB), feature
field. The NN3D integrates non-local features in a single
enhancement block (FEB), attention block (AB) and
modular framework to further improve de-noising
reconstruction block (RB). The SB reduces the depth and
performance. Similarly, graph CNN also exploits the non-
improves the efficiency of the network with the usage of
local similarities by incorporation a graphical convolutional
convolution and dilated convolution operator. It is twelve
layer. The graph CNN layer works on feature maps to
layer block with dilated Conv + BN + ReLU (second, fifth,
aggregate similar spatially adjacent and spatially distant
ninth, and twelfth layer) and Conv + BN + ReLU in the rest
pixels. The averaging of local and non-local pixels is done
of the layers. The next (13th to 16th) layers form FEB to
to produce the desired feature map. Universal Denoising
create robust features by merging global and local features.
network (UNet and UNLNet) is another network that
The first three layer of FEB is Conv + BN + ReLU, and the
integrates convolution and non-local filtering layers for
fourth layer is Conv. The output of the Conv layer and
both gray and color image denoising [60]. Fig 7 depicts the
input noisy is concatenated to improve the representation
block diagram of the combined architecture of DnCNN,
capability further. It is followed by the usage of tanh
IDCNN, SCNN, and FFDNet.
activation for the non-linearity. The AB consists of just one
The models, namely PDNN[61], IRCNN [62] and DRUNet
Conv layer, which compresses the features into the weights
[63] integrate the observational model with deep CNN’s
to modify the previous layer output. RB is the final stage,
discriminative learning. The model-based methods require
which incorporates subtractor for the residual learning
several iterative steps to solve the optimization problem,
process. The architecture of ADNet is given in Fig. 8. The
but they are utilized to solve different image restoration
fully convolutional encoder-decoder structure with skip
tasks like de-blurring, super-resolution, and de-noising with
connections is also used for Gaussian and speckle noise
the single model with the help of an image degradation
removal [65].
matrix. They utilize the powerful de-noising capabilities of
The PReLU (parameteric rectified linear units) and edge
CNN and prior of the observational models in a single
aware based CNN de-noiser is one of the latest works that
modular framework. In [61], [62], model based
has produced good PSNR results for both BSD-68 and Set-
optimization is merged with robust image priors with
12 compared to other networks [66]. It is improvised
variable splitting technique. The variable splitting reduces
DnCNN network with PReLU as an activation function
the number of CNN parameters and enhances the CNN
which learns the slope in negative direction as well. The
training efficiency. The observation model is unfolded into
inclusion on principal component analysis on the feature
discriminating CNN learning, which is composed of
9
TABLE I
ADVANTAGES AND DISADVANTAGES OF DIFFERENT MACHINE-LEARNING MODELS
Machine- Advantages Disadvantages
learning model
Dictionary- • Dictionary-learning models are powerful tools for • Most dictionary-learning methods consider an
learning model many image restoration, de-noising and over-complete dictionary and formulate the
recognition tasks learning process as a minimization problem
• Most local image patches can be well • Minimization problems are very challenging and
approximated by a sparse linear combination of mostly non-convex
basis atoms • Minimization in such a case is usually greedy and
• Constructing dictionaries adaptive to the input computational demanding
image via some learning process helps achieve • If correlations among dictionary atoms are not well
better sparsity that fixed orthogonal dictionaries constrained, the redundancy of the dictionary does
like DCT, and wavelets not necessarily improve the performance of sparse
• Dictionary-based learning models help preserve coding
minute details and texture of images that undergo
noise addition
Multi-layer • Capability to learn non-linear models • MLP with hidden layers have a non-convex loss
perceptron • Capability to learn models in real-time (on-line function where there exists more than one local
model learning) minimum. Therefore different random weight
initializations can lead to different validation
accuracy
• MLP requires tuning a number of hyper-parameters
such as the number of hidden neurons, layers, and
iterations
• MLP is sensitive to feature scaling
CNN • Less number of parameters as compared to fully • Analytical approaches have advantage over CNN
connected neural networks in merging prior into the inverse problem solution
• Apt for both known and blind Gaussian de-noising • Non-avaibaility of image databases for medical
• Concept of transfer learning i.e. the weights image de-noising and classification
learned by CNN can be used by other network. • Difficulty in case of unsupervised learning in real
• Many methodologies are being designed owning world scenario.
to simple architecture and mathematical modelling
.
GAN • Unsupervised learning method; can be trained • Fail to model a multimodal probability distribution
using unlabelled image data as they learn internal of data; suffer from mode collapse. Sometimes,
representations of data suffer from complete collapse (generated samples
• Generate data similar to real image data; can are virtually identical)
generate images indistinguishable from the real • Suffer from the problem of vanishing gradients;
data training of the initial layers in the net is either
• Learn complex distributions of image data extremely slow or effectively stops
• Discriminator is a classifier that can classify • Internal covariance shift is induced by a change in
objects the input distribution; this slows down the training
• Training of GANs can be comparatively slow
owing to the above reasons
maps in sixteenth layer has led to the extraction of more normalization. It has eight convolutional layers and two
features. The final step is cascading the network with an fully connected layers, which assign a probability to
adaptive bilateral edge aware filter to further refine the edge generated images and ground-truth images. The value
and texture details. function of de-noising GAN network is given by
min 𝑚𝑎𝑥
𝑉(𝐷, 𝐺) = 𝐸𝒙~𝑝𝑑𝑎𝑡𝑎(𝒙) [𝐷(𝒙)] − 𝐸𝒚~𝑝𝒚(𝒚) [𝐷(𝒚)]
C. METHODOLOGIES OF GAN-BASED MODELS 𝐺 𝐷∈𝐷
(GAUSSIAN NOISE) (16)
The GAN network given [67] in uses DenseNet CNN as the where 𝐷 is the set of 1-Lipschitz functions. The objective is
generator network to ease up the vanishing-gradient to make an approximation of 𝐾. 𝑊(𝑝𝑑𝑎𝑡𝑎(𝑥), 𝑝𝑦 (𝑦)), in
problem and Wasserstein-GAN as the loss function. The which 𝐾 is a Lipschitz constant and 𝑊 is a Wasserstein
generator network outputs an estimated ground truth image distance. The gradient penalty term is added so that the
from the noisy image, whereas the discriminator eliminates gradient of the discriminator network does not exceed 𝐾,
the difference between the generator output and the ground- and is given by
truth image. The generator network follows the architecture 𝜆𝐸𝒚~𝑝𝒚(𝒚) [(‖𝛻𝒚 𝐷(𝒚) − 1)2 ]‖ (17)
2
of DenseNet with eight Dense Blocks, along with input, Loss function is the combination of content loss and
output, and bottleneck convolution block. The generator adversarial loss given as
extracts both low-level and high-level features efficiently. 𝑙 = 𝜆𝑙𝐺𝐴𝑁 + 𝑙𝑐𝑜𝑛𝑡𝑒𝑛𝑡 (18)
The discriminator network uses leaky ReLU as the where, content loss is given by 𝑙1 or 𝑙2 norm, and
activation function and layer normalization instead of batch
10
adversarial loss is given by Wasserstein-GAN critic selected from the RVIN-corrupted test image and feature
function. vectors that indicate whether the centre pixel is
The GAN-CNN based blind de-noiser (GCBD) model [42] contaminated or extracted by the predictor. These feature
extracts noise blocks from the clean images. The GAN vectors are composed of numerous statistics, viz., the
produces a noisy block instead of the de-noised image. The multiple rank-ordered absolute differences (ROADs), the
noisy blocks extracted from the GAN are used for the clean pixel median deviation (CPMD), and the edge pixel
creation of a training dataset for CNN. The GCBD model is difference (EPD). They are rapidly mapped to noisy/clean
a cascade connection of GAN followed by CNN. The (1 for noisy, 0 for clean) labels by the pre-trained noise
generated blocks by GAN along with extracted noise blocks detector. According to the ratio of the obtained noisy labels
are used for training the discriminative learning based to the total number of selected patches, the predictor
CNN. The GCBD can be used when there is an absence of provides the noise ratio of the whole image. From the
paired data for the supervised training of CNN. It gives output of the NRP, i.e., the predicted noise ratio, the most
promising results for Gaussian noise, mixed noise, and real- appropriate DnCNN specifically trained for this noise ratio
world noisy images. The limitation is that noise is taken is exploited for de-noising. Under the guidance of the NRP,
only as additive white noise with zero-mean. the proposed method has the ability to handle unknown
noise ratios. This method performs well in terms of
III. MACHINE LEARNING BASED IMPULSE DE-NOISERS execution efficiency and image restoration. Turkmen [72]
has proposed an artificial neural network for de-noising
A. METHODOLOGIES OF DICTIONARY LEARNING RVIN-incorporated images by detecting the noisy pixels.
MODELS (IMPULSE NOISE) The statistics used to detect the RVIN noisy centres are
Wang et al. have proposed an adaptive dictionary-learning- rank-ordered absolute differences (ROADs), and rank-
based method to preserve image structure in impulse- ordered logarithmic difference (ROLD) values. These are
contaminated images with the help of a robust 𝑙1 -norm the inputs to the ANN for the detection process. After the
data-fidelity term to help impulse noise cancellation [68]. In detection process is completed, the corrupted pixels are
this algorithm, the restoration problem is mathematically restored by the edge-preserving regularization (EPR)
formulated into an 𝑙1 − 𝑙1 minimization objective and method, allowing edges and noise-free pixels to be
solved under the augmented Lagrangian framework through preserved. This mechanism works well in the presence of
a two-level nested iterative procedure. The algorithm has high-density RVIN.
high image restoration power to produce restored images Li et al. [73] have improvised the usage of densely
with a high PSNR value. Guo et al. [69] have introduced a connected convolutional networks (DenseNet) to de-noise
novel algorithm to enhance image sparsity to help remove images corrupted by impulse noise with the help of CNN
salt and pepper noise removal with a fast multiclass to learn pixel-distribution features from noisy images. The
dictionary learning, and then both the sparsity proposed method, viz., a densely connected network for
regularization and robust data fidelity are formulated as impulse noise removal (DNINR), captures the pixel-level
minimizations of 𝑙0 − 𝑙0 norm for impulse noise removal. distribution information using wide and transformed
Additionally, a numerical algorithm of modified alternating network learning. This mechanism shows significantly
direction minimization is derived to solve the proposed de- better results in terms of edge preservation and noise
noising model. This algorithm excels in image detail suppression.
preservation. Deka et al. in [70] have proposed a novel two- Khaw et al. [74] have used an efficient CNN with particle
stage de-noising method for removing random-valued swarm optimization (PSO) for high-density impulse noise
impulse noise from an image. In the first stage, an impulse removal. This high-density impulse noise detection and
noise detection scheme is used to detect the pixels which removal model mainly consists of two parts: impulse noise
are likely to be corrupted by the impulse noise, viz., noise removal and impulse noisy pixel detection for restoration.
candidates. In the second stage, the noise candidates are The deep CNN architecture facilitates the de-noising
reconstructed by the image impainting method based on procedure to filter out noise from the noisy images. The
sparse representation in an iterative manner until PSO algorithm optimizes the threshold values for detecting
convergence is achieved. This algorithm works well in impulse noisy pixels. The method is robust and works well
terms of both visual and quantitative aspects. on both gray and colour images in terms of both qualitative
and quantitative aspects.
B. METHODOLOGIES OF CNN-BASED MODELS The RVIN can also be removed with the combination of
(IMPULSE NOISE) classifier and regression CNN [75]. Classifier network
Chen et al. have proposed a blind CNN architecture for separates noisy and noise-free pixels. Thereafter, the
random-value impulse noise (RVIN) removal [71]. This regression network uses noise-free pixels along with the
improvised de-noising mechanism for RVIN suppression original noisy input image to predict the output image.
works on the principle of flexible noise ratio prediction, Batch Normalization is embedded in both classifier and
which proved to be better than DnCNN-based RVIN regression network to accelerate the de-noising
suppression by eliminating unnecessary dependence on the performance.
exact perception of the noise ratio. Random patches are
11
PLs
Random Patch Noise Contamination Impulse Noise

Extractor Determiner Remover
1. DnCNN
FVs 2. EPR
Noise Remover 3. DenseNet
Classifier Statistics N/Ws (Post- 4. Regression
Network 1. ROADS detection) network
2. CPMD
3. EPD
4. ROLD
FIGURE 9: Block Diagram depicting the basic concept of impulse de-noisers
Fig.9 shows the overall architecture of the Impulse-noise Multi-directional Long-Short Term Memory Networks to
removal model. The first step is the extraction of a random de-noise images of Poisson noise. CNN layers are used to
patch from the noise-corrupted image, and then a classifier extract image features and to estimate noise bases in the
N/W is used to predict noisy labels. Thus, the noise- images, and the multi-directional LSTM layers are
contamination determiner determines the predicted labels efficiently used to memorise the statistics of residual noise
(PLs) in case of Jin et al., and extracts feature vectors (FVs) components, which possess long-range correlations and are
in case of Chen and Turkmen. These feature vectors are sparse in the spatial domain. The Blahut-Arimoto algorithm
composed of numerous statistics, viz., ROADs, CPMD, is used to numerically derive a distortion-mutual
EPD, ROLD, etc. Finally, a de-noiser N/W is used to de- information function for the image de-noising algorithm.
noise the contaminated image based of the identified noisy The algorithm shows state-of-the-art performance in terms
centres. of objective and subjective qualities. Su at al. [78] have
IV. MACHINE LEARNING BASED POISSON DE- proposed a novel method to tackle the problems caused due
NOISERS to Poisson noise in the low-light imaging field. This
Poisson noise is a special type of noise that is not additive proposal is that of a deep multi-scale cross-path
in nature. Unlike the Gaussian noise, the noise power is concatenation residual network (MC2RNet) which
measured by the peak value as its strength is dependent on incorporates cross-path concatenation modules for de-
the image intensity. It is natural to define the noise power in noising. MC2RNet learns the remnant residue between the
an image by the maximal value in the image, i.e., its peak noisy and the latent clean image to facilitate the model
value. Thus, Poisson de-noisers are described in terms of training procedure. This method opts for blind Poisson
the peak value as the strength of the noise power. training over discriminative de-noising algorithms to train a
single model for handling Poisson noise with different
A. METHODOLOGIES OF DICTIONARY LEARNING levels. The algorithm shows a better performance in terms
MODELS (POISSON NOISE) of peak signal-to-noise ratio and visual effects. Ramez et al.
Giryes et al. [76] have proposed a novel method to apply [79] have proposed a flexible and data-driven method to de-
the sparse-representation technique to image patches noise Poisson-corrupted images, which reduces the heavy
extracted, adopting the same exponential idea. The ad hoc engineering load occurring due to computational
proposed algorithm uses greedy pursuit with boot-strapping post-processing in the contemporary de-noising procedures.
based stopping condition and dictionary learning within the They have used a powerful framework of deep CNNs and a
de-noising process. The stopping criterion is novel in its training mechanism that trains the same network with
nature. The paper effectively migrates from the Gaussian images having a specific peak value. Thus, by using a
Mixture model (GMM) to a dictionary-learning based supervised approach and the representation capabilities of
model by resolving the difficulties involved in the deep CNNs, and using a specific class of images for
conversion process. The reconstruction performance of the training, the authors have presented a comparatively simple
proposed scheme is competitive with leading methods in method that shows state-of-the-art performance both
high SNR, and achieving state-of-the-art results in cases of qualitatively and quantitatively and is an order of
low SNR. magnitude faster than other methods. Ramez et al. [80]
have introduced a methodology that exploits the
B. METHODOLOGIES OF CNN-BASED MODELS architecture of a fully convolutional CNN that uses shallow
(POISSON NOISE) layers to handle local noise statistics and deeper layers to
Kumwilaisak et al. [77] have proposed a method recover edges and enhance textures. The de-noiser is made
(CNN+LSTM) based on Deep Convolutional Neural and
12
class-aware by exploiting semantic class information that A. METHODOLOGIES OF DICTIONARY LEARNING

boosts performance, and enhances textures, and reduces MODELS (MIXED NOISE)
artifacts. The residual learning based Gaussian de-noiser The dictionary learning models are designed for mixed
(DnCNN) [38], discussed in section 2.2 can also be trained Gaussian noise of different standard deviation and mixed
for Poisson noise removal with training with Poisson noise Gaussian-impulse noise [84]. The energy minimization
corrupted data patches and relevant hyper-parameter model with the weighted 𝑙 2 − 𝑙 0 norm is being used for
settings [77]. mixed noise removal such as Gaussian-Gaussian mixture,
V. MACHINE LEARNING BASED MIXED NOISE DE- impulse noise, and Gaussian-impulse noise. It integrates
NOISERS maximum likelihood estimation and sparsity over the
The extraction of the clean image from mixed noise learned dictionary. Modified-SVD is used for low rank
corrupted image is very complex problem because of the approximation. In the recent structured dictionary learning
high-level of non-linearity in the noise distribution. The model [87], two structured dictionary learning models are
combination of Gaussian and impulse noise is present in combined together. The data-fidelity term uses 𝑙𝑝 -norm
many practical applications. The CNN based transfer fidelity to fit image patches and 𝑙𝑞 -norm regularizar for the
learning models, dictionary learning model, and variational sparse coding. The authors in [32] propose a novel
based mixed noise model are the machine learning models algorithm to tackle mixed Gaussian noise, i.e., the K-SVD
developed for mixed-noise removal. The comparative algorithm, which generalizes the K-means clustering
analysis of various mixed noise models is difficult as mixed process for adapting dictionaries in order to achieve sparse
noise can be modeled in different ways. The expression of signal representations on a given set of training signals. A
noisy image pixel 𝑦 obtained by corruption with the dictionary is sought for that leads to the best representation
Gaussian and SPIN noise is given by for each member in the set, under strict sparsity constraints.
𝑑𝑚𝑖𝑛 with probablity 𝑝/2
𝑦 = { 𝑑𝑚𝑎𝑥 with probablity 𝑝/2 } (19) B. METHODOLOGIES OF CNN-BASED MODELS (MIXED
𝑥 + 𝑣 with probablity 1 − 𝑝 NOISE)
where 𝑑𝑚𝑖𝑛 and 𝑑𝑚𝑎𝑥 are minimum and maximum value in The CNN-based transfer learning, four-stage convolutional
the entire image dynamic range with probability of 𝑝/2. filtering model is mixed noise de-noiser designed for a
The AWGN noise 𝑣 is added with the probability 1 − 𝑝. mixture of Gaussian and impulse noise [82]. It uses a rank
Similarly, the expression of noisy image pixel 𝑦 obtained order filter in the preprocessing step, which is Cai’s filter in
by the corruption with the SPIN, RVIN and Gaussian noise case of Gaussian and SPIN, whereas the combination of
is given by: adaptive median filter and adaptive center weighted median
𝑑𝑚𝑖𝑛 with probablity 𝑝/2 filter is used in the case of Gaussian, SPIN, and RVIN. The
𝑑𝑚𝑎𝑥 with probablity 𝑝/2 bilinear interpolation is performed on rank order filter
𝑦={ } (20)
𝑑 with probablity 𝑟(1 − 𝑝) output to get a slightly smoother version of the noisy image.
𝑥 + 𝑣 with probablity (1 − 𝑟)(1 − 𝑝) The purpose of bilinear interpolation is to suppress high-
where 𝑑 is random pixel value with the probability 𝑟(1 − frequency components that occurred due to rank order
𝑝).
TABLE II
COMPARISION OF PSNR (IN dB) VALUE OF DIFFERENT MACHINE LEARNING MODELS ON SET-12 DATASET
Images C.Man Hous Pepper Starfis Monar Airpla Parrot Lena Barbara Boat Man Couple Average
e s h ch ne
Gaussian noise level σ=15
K-SVD [32] - 34.30 - - - - 30.97 - - - 30.46 - -
K-LLD [43] - 33.81 - - - - 30.91 - - - 30.65 - -
CSF [47] 31.95 34.39 32.85 31.55 32.33 31.33 31.37 34.06 31.92 32.01 32.08 31.98 32.32
TNRD [37] 32.19 34.53 33.04 31.75 32.56 31.46 31.63 34.24 32.13 32.14 32.23 32.11 32.50
DnCNN [38] 32.61 34.97 33.30 32.20 33.09 31.70 31.83 34.62 32.64 32.42 32.46 32.47 32.86
IDCNN1[40] 32.54 34.87 33.24 - 35.49 32.79 - 33.75 33.15 31.81 - - -
IDCNN2 [40] 32.24 34.83 33.11 - 35.38 32.68 - 33.70 32.98 31.73 - - -
FFDNet [56] 32.42 35.01 33.10 32.02 32.77 31.58 31.77 34.63 32.50 32.35 32.40 32.45 32.75
PDNN[61] 32.44 35.40 33.19 32.08 33.33 31.78 31.48 34.80 32.84 32.55 32.53 32.51 32.91
GraphCNN
[59]
32.58 35.13 33.27 32.42 33.25 31.84 31.89 34.57 32.84 32.41 32.42 32.40 32.917
ECNDNet[50] 32.56 34.97 33.25 32.17 33.11 31.70 31.82 34.52 32.41 32.37 32.39 32.39 32.81
IRCNN [62] 32.55 34.89 33.31 32.02 32.82 31.70 31.84 34.53 32.43 32.34 32.40 32.40 32.77
BRDNet[51] 32.80 35.27 33.47 32.24 33.35 31.85 32.00 34.75 32.93 32.55 32.50 32.62 33.03
ADNet [64] 32.81 35.22 33.49 32.17 33.17 31.86 31.96 34.71 32.80 32.57 32.47 32.58 32.98
ADNet-B [64] 31.98 35.12 33.34 32.01 33.01 31.63 31.74 34.62 32.55 32.48 32.34 32.43 32.77
PReLU [66] 33.18 35.59 33.54 33.17 34.20 32.65 32.73 35.21 32.25 32.90 32.76 32.78 33.41
MWCNN [53] - - - - - - - - - - -- - 33.20
K-SVD [32] - 32.12 - - - - 28.12 - - - 27.59 - -
13
K-LLD [43] - 31.77 - - - - 28.40 - - - 28.17 - -

MLP [35] 29.61 32.56 30.30 28.82 29.61 28.82 29.25 32.25 29.54 29.97 29.88 29.73 30.03
CSF[47] 29.48 32.39 30.32 28.80 29.62 28.72 28.90 31.79 29.03 29.76 29.71 29.53 29.83
TNRD[37] 29.72 32.53 30.57 29.02 29.85 28.88 29.18 32.00 29.41 29.91 29.87 29.71 30.06
DnCNN[38] 30.18 33.06 30.87 29.41 30.28 29.13 29.43 32.44 30.00 30.21 30.10 30.12 30.43
IDCNN1[40] 30.06 32.94 30.79 - 32.85 30.06 - 31.13 30.45 29.18 - - -
IDCNN2[40] 29.98 32.87 30.71 - 32.77 30.00 - 31.08 30.18 29.12 - - -
FFDNet [56] 30.06 33.27 30.79 29.33 30.14 29.05 29.43 32.59 29.98 30.23 30.10 30.18 30.43
PDNN[61] 30.12 33.54 30.90 29.43 30.31 29.14 29.28 32.69 30.30 30.34 30.15 30.24 30.54
GraphCNN[59] 30.12 33.22 30.87 29.76 30.51 29.30 29.48 32.42 30.28 30.17 30.10 29.99 30.516
ECNDNet[50] 30.11 33.08 30.85 29.43 30.30 29.07 29.38 32.38 29.84 30.14 30.03 30.03 30.39
IRCNN [62] 30.08 33.06 30.88 29.27 30.09 29.12 29.47 32.43 29.92 30.17 30.04 30.08 30.38
BRDNet[51] 31.39 33.41 31.04 29.46 30.50 29.20 29.55 32.65 30.34 30.33 30.14 30.28 30.61
ADNet[64] 30.34 33.41 31.14 29.41 30.39 29.17 29.49 32.61 30.25 30.37 30.08 30.24 30.58
ADNet-B[64] 29.24 33.38 30.99 29.22 30.38 29.16 29.41 32.59 30.05 30.28 30.01 30.15 30.46
PReLU [66] 31.20 34.37 31.86 31.06 32.02 30.60 30.73 33.03 30.68 30.82 30.73 30.74 31.49
MWCNN[53] - - - - - - - - - - - - 30.84
MLP [35] 26.37 29.64 26.68 25.43 26.26 25.56 26.12 29.32 25.24 27.03 27.06 26.67 26.78
TNRD [37] 26.62 29.48 27.10 25.42 26.31 25.59 26.16 28.93 25.70 26.94 26.98 26.50 26.81
DnCNN[38] 27.03 30.00 27.32 25.70 26.78 25.87 26.48 29.39 26.22 27.20 27.24 26.90 27.18
IDCNN1[40] 26.80 29.89 27.22 - 29.23 26.52 - 27.69 26.50 25.87 - - -
IDCNN2[40] 26.82 29.76 27.20 - 29.18 26.45 - 27.68 26.51 25.87 - - -
FFDNet [56] 27.03 30.43 27.43 25.77 26.88 25.90 26.58 29.68 26.48 27.32 27.30 27.07 27.32
PDNN [61] 27.12 31.04 27.44 25.95 27.00 25.97 26.42 29.85 27.21 27.42 27.32 27.23 27.50
GraphCNN
[59]
27.00 30.16 27.40 25.92 26.89 25.93 26.43 29.32 26.56 27.05 27.19 26.75 27.217
ECNDNet[50] 27.07 30.12 27.30 25.72
25.79 26.82
26.32 29.29 26.26 27.16 27.11 26.84 27.15
IRCNN [62] 26.88 29.96 27.33 25.57
25.89 26.61
26.55 29.40 26.24 27.17 27.17 26.88 27.14
BRDNet [51] 27.44 30.53 27.67 25.77
25.93 26.97
26.66 29.73 26.85 27.38 27.27 27.17 27.45
ADNet [64] 27.31 30.59 27.69 25.70
25.88 26.90
26.56 29.59 26.64 27.35 27.17 27.07 27.37
ADNet-B[64] 27.22 30.43 27.70 25.63
26.03 26.92
26.56 29.53 26.51 27.22 27.19 27.05 27.33
PReLU [66] 28.00 31.75 28.74 27.52
27.54 28.57
27.56 29.62 28.13 27.59 27.65 27.40 28.34
MWCNN[53] - - - - - - - - - - - -- 27.79
TABLE III
COMPARISION OF PSNR (IN dB) VALUE OF DIFFERENT MACHINE LEARNING MODELS ON BSD-68
BSD-68
σ 15 25 50
DnCNN [38] 31.72 29.23 26.23
IRCNN [62] 31.63 29.15 26.16
SCNN [49] 31.48 29.03 26.08
PDNN [61] 32.29 29.88 -
FFDNet [56] 31.62 29.19 26.30
UNLNet [60] 31.95 29.51 26.41
ECNDNet[50] 31.71 29.22 26.23
ADNet [64] 31.74 29.25 26.29
BRDNet [51] 31.79 29.29 26.36
DRUNet [63] 31.91 29.48 26.59
MWCNN[53] 31.91 29.46 26.58
PReLU [66] 32.60 30.48 27.20
σ 10 20 30 40 50 75
DnCNN-S [38] 33.88 30.27 28.36 27.11 26.23 24.64
IRCNN [62] 33.74 30.16 28.26 27.08 26.19 -
NN3D(WDnCNN)
- - 28.56 - 26.42 24.91
[58]
FFDNet [56] 33.76 30.23 28.38 27.18 26.29 24.79
DIDN [55] 33.98 30.44 28.58 27.37 26.47 -
NN3D(DnCNN)
- - 28.41 - 26.27 24.71
[58]
NN3D(FFDNet)
- - 28.37 - 26.29 24.80
[58]
DIDN+ [55] 34.01 30.47 28.61 27.40 26.50 -
WDnCNN [48] - - 28.56 - 26.39 24.85
DHDN [52] 34.04 - 28.58 - 26.43 -
Kodak-24
σ 10 20 30 40 50
DnCNN-S [38] 34.90 31.47 29.62 28.37 27.49
IRCNN [62] 34.76 31.38 29.52 28.37 27.45
FFDNet [56] 34.81 31.47 29.69 28.51 27.62
DIDN [55] 35.16 31.83 30.04 28.84 27.96
DIDN+[55] 35.20 31.87 30.08 28.88 28.01
14
DHDN [52] 35.24 - 30.11 - 28.01

σ 15 25 35 50 75
BRDNet [51] 34.88 32.41 30.80 29.22 27.49

ADNet [64] 34.76 32.26 30.68 29.10 27.40
TABLE IV
PSNR (in dB) VALUES OF COLOR DE-NOISING NETWORKS
CBSD-68
σ 10 20 30 40 50
CBM3D [81] 35.89 31.89 29.72 28.08 27.36
DnCNN-S [38] 36.12 32.37 30.32 28.95 27.92
IRCNN [62] 36.06 32.27 30.22 28.85 27.86
FFDNet [56] 36.14 32.34 30.31 28.96 27.96
DIDN [55] 36.48 32.73 30.71 29.36 28.35
DIDN+ [55] 36.52 32.77 30.75 29.40 28.40
DHDN [52] 36.48 - 30.54 - 28.01
σ 15 25 50
DRUNet [63] 34.30 31.69 28.51
Kodak-24
σ 10 20 30 40 50
CBM3D [81] 36.57 32.92 30.89 29.17 28.62
DnCNN-S [38] 36.58 33.20 31.28 29.95 28.94
IRCNN [62] 36.70 33.19 31.24 29.91 28.92
FFDNet [56] 36.80 33.32 31.39 30.08 29.10
DIDN [55] 37.32 33.88 31.97 30.68 29.72
DIDN+ [55] 37.37 33.94 32.03 30.75 29.80
DHDN [52] 37.37 - 32.01 - 29.74
σ 15 25 50
DRUNet [63] 35.31 32.89 29.86
TABLE V
PSNR(in dB) COMPARISION OF DIFFERENT METHODS ON LIVE1 DATASET ON POISSON NOISE
Peak Value DnCNN [38] DenoiseNet[79] CNN+LSTM[77]
0.1 18.90 19.15 19.61
1 22.68 22.59 22.87
10 27.77 27.49 27.77
40 30.92 30.72 31.01
80 32.65 32.31 32.71
TABLE VI
AVERAGE PSNR (IN dB) VALUES ON DIFFERENT DATASETS FOR POISSON NOISE
Baboo Averag
Method Peak Boats Airplane Barbara Flower Goldhill lenna Monarch Pens Pepper
n e
DnCNN [38] 0.1 19.63 18.60 17.34 18.84 19.50 20.89 19.30 18.39 19.92 18.66 19.11
DenoiseNet [79] 0.1 20.10 19.35 17.29 18.98 19.79 21.16 19.26 18.36 19.89 19.23 19.35
CNN+LSTM [77] 0.1 20.56 19.66 17.52 19.20 20.21 21.63 20.40 19.03 20.18 19.62 19.80
DnCNN [38] 1 24.37 22.98 19.30 22.03 24.52 24.46 24.33 23.70 24.30 23.60 23.36
DenoiseNet [79] 1 24.24 22.94 19.35 21.89 24.18 24.47 24.28 23.52 23.99 23.42 23.23
CNN+LSTM [77] 1 24.74 23.54 19.44 22.23 25.22 24.79 24.72 23.82 24.45 23.62 23.66
DnCNN [38] 10 29.76 28.40 22.95 2.20 30.74 28.62 29.27 29.96 29.64 28.32 28.48
DenoiseNet [79] 10 29.44 28.15 22.90 26.57 30.39 28.45 29.03 29.61 29.21 28.19 28.19
CNN+LSTM [77] 10 29.80 28.43 23.01 27.15 30.73 28.68 29.38 29.94 29.56 28.49 28.52
DnCNN [38] 40 32.78 31.23 25.38 30.43 33.66 31.01 31.27 33.23 32.48 30.35 31.18
DenoiseNet [79] 40 32.58 31.14 25.40 30.00 33.48 30.88 31.25 32.98 32.18 30.62 31.05
CNN+LSTM [77] 40 32.91 31.64 25.56 30.49 33.86 31.11 31.58 33.32 32.56 30.90 31.40
DnCNN [38] 80 34.29 32.57 26.78 31.95 35.07 32.34 32.33 34.85 34.10 31.47 32.56
DenoiseNet [79] 80 33.90 32.61 26.80 31.50 34.52 32.16 32.11 34.35 33.46 31.63 32.30
CNN+LSTM [77] 80 34.37 33.16 26.98 32.02 35.16 32.40 32.50 34.97 34.03 31.99 32.76
TABLE VII
AVERAGE PSNR (in dB) VALUE ON SET-10 DATASET FOR POISSON NOISE
Method Peak Flag House Camera Man Bridge Saturn Peppers Boat Couple Hill
DenoiseNet [79] 19.45 22.87 21.59 22.49 19.83 26.26 21.43 22.38 22.11 22.82
MC2RNet6−B
19.97 23.04 21.95 22.52 19.88 26.93 21.69 22.48 22.15 22.80
[78] 1
MC2RNet6−S
20.04 23.17 21.97 22.54 19.94 27.35 21.70 22.60 22.20 22.94
[78]
DenoiseNet [79] 21.38 24.77 23.25 23.64 20.80 28.37 23.19 23.66 23.30 23.95
MC2RNet6−B
21.87 24.99 23.42 23.66 20.86 29.01 23.20 23.69 23.40 23.99
[78] 2
MC2RNet6−S
21.92 25.23 23.66 23.76 20.87 29.22 23.35 23.80 23.57 24.07
[78]
DenoiseNet [79] 4 23.18 26.59 24.87 24.77 21.81 30.02 24.83 24.86 24.60 25.01
15
MC2RNet6−B
24.13 26.78 24.84 24.84 21.81 30.94 24.96 24.29 24.76 25.10
[78]
MC2RNet6−S
24.24 27.18 25.03 24.95 21.91 31.29 24.97 25.08 24.93 25.25
[78]
DenoiseNet [79] 25.73 28.42 26.35 26.10 22.91 32.28 26.45 26.23 26.11 26.26
MC2RNet6−B
26.61 28.56 26.39 26.06 22.95 33.28 26.66 26.24 26.26 26.34
[78] 8
MC2RNet6−S
26.71 28.75 26.45 26.24 23.01 33.71 26.68 26.46 26.41 26.43
[78]
DenoiseNet [79] 28.94 31.67 29.21 28.74 25.42 36.20 29.77 29.06 29.13 28.71
MC2RNet6−B
30.57 31.52 29.04 28.71 25.37 36.40 29.71 28.99 29.16 28.77
[78] 30
MC2RNet6−S
30.95 31.90 29.23 28.83 25.48 36.99 29.84 29.21 29.34 28.93
[78]
TABLE VIII
AVERAGE PSNR(in dB) VALUE ON BSD-68 DATASET FOR POISSON NOISE
Method Peak=1 Peak=2 Peak=4 Peak=8
IRCNN [62] [80] 21.66 22.86 24.00 25.27
DenoiseNet [79] 21.79 22.90 23.99 25.30
MC2RNet6−B [78] 21.92 23.00 24.13 25.39
MC2RNet6−S [78] 22.00 23.08 24.25 25.51
TABLE IX
AVERAGE PSNR (in dB) COMPARISION OF MIXED NOISE (RVIN+AWGN) ON DIFFERENT NATURAL IMAGES
Noise
Image Lena Barbara Bridge Boat Airplane Pepper Hill
level
Mixed CNN [82] σ=15, 32.28 25.67 28.86 29.12 32.52 34.07 31.61
[83] ρ=0.15 32.56 29.43 29.16 30.30 32.87 33.49 31.73
Mixed CNN [82] σ=15, 29.10 24.17 26.54 27.02 28.81 29.90 29.04
[83] ρ=0.30 31.71 28.32 28.09 29.19 31.88 33.66 30.98
Mixed CNN [82] σ=15, 24.87 21.67 22.86 23.62 23.32 24.54 24.69
[83] ρ=0.45 30.36 26.25 26.53 27.60 30.44 32.09 29.86
σ=25,
Mixed CNN [82] 29.87 24.52 26.61 27.61 30.34 31.65 29.48
ρ=0.15
[83] 30.46 27.28 26.72 28.31 30.73 31.76 29.55
σ=25,
Mixed CNN [82] 27.93 23.36 25.10 26.08 28.00 28.70 27.80
ρ=0.30
[83] 29.79 26.35 25.97 27.39 29.90 31.30 28.99
σ=25,
Mixed CNN [82] 24.88 21.65 22.62 23.45 23.97 24.87 24.59
ρ=0.45
[83] 28.54 24.75 24.81 26.10 28.40 30.04 28.01
TABLE X
DE-NOISING RESULTS FOR MIXED GAUSSIAN NOISE ON BARBARA IMAGE (PSNR=19.02 dB)
Method PSNR (dB) Method PSNR (dB)
K-SVD (known parameters)
WDL (case 1) [84] 32.39 27.66
[32]
W-KSVD (unknown TYPE-II (𝑟1 ∶ 𝑟2 = 0.7 ∶
WDL (case 2) [84] TYPE-I 30.07 29.35
parameters) [84] 0.3, 𝜎1 = 10, 𝜎2 = 50)
K-SVD [85] 26.95 IRCNN [62] 28.95
EM-CNN [86] 30.68
TABLE XI
RESULTS FOR BLIND DE-NOISING
Noise Type Gaussian noise
Mode Non-blind Blind
Method BM3D [20] DnCNN-B [38] GCBD[42]
𝜎 = 15 31.07 31.61 31.59
𝜎 = 25 28.57 29.16 29.15
Noise Type Mixed noise
Mode Non-blind Blind
Method BM3D [20] DnCNN-B[38] GCBD[42]
𝑠 = 15 41.08 40.75 42.00
𝑠 = 25 37.85 37.54 39.87
filtering on the Gaussian noise. It is followed by the four- propagation algorithm. The other CNN model for mixed
stage convolutional filtering. The first stage consists of the Gaussian and impulse noise involves two parts: the first
the conv layer and ReLU activation function followed by half for impulse noise removal and the second half for the
the max-pooling layer. The second and third stages consist Gaussian noise removal [83]. It consists of the input layer,
of the conv layer and ReLU activation function. The fourth intermediate layers of convolution, batch normalization,
stage is the conv layer. The squared Frobenius norm is used and leaky ReLU followed by the convolutional output
as the loss function, and training is done by the back layer. The second part of Gaussian noise removal has a skip
16
connection for residual learning. CNN model given in ,has more attention to noise regions, which contributes in
conv+ReLu+BN as basic building block and shows best balancing between noise removal and texture preservation.
structural metrics results for both known and unknown Extensive experiments show that this method performs well
noise level of mixed Gaussian-Impulse noise [88]. The both qualitatively and quantitatively. Chen et al. have
CNN is used as a regularizar in traditional variational based proposed a Deep Boosting Framework (DBF) [93] for real-
methods for mixed noise removal [86]. The mixed noise world image denoising by combining the deep learning into
parameters are iteratively estimated by variational method the boosting algorithm. The DBF replaces conventional
followed by noise classification according to the statistical boosting units by elaborate convolutional neural networks.
parameters. The methodology is implemented by The outcome is a lightweight Dense Dilated Fusion
optimization of sub-problem involving four steps which are Network (DDFN) as the boosting unit, which addresses the
regularization, synthesis, parameters estimation and noise vanishing gradient problem during training due to the
classification. cascading of networks while promoting the efficiency of
VI. REAL WORLD-DENOISERS limited parameters. This method reduces the domain-shift
Xu et al. have constructed a benchmark dataset to de-noise issue with the one-shot domain transfer scheme. This is a
real-world images [89]. The authors have used different strong technique in terms of real-world de-noising. Real-
cameras with different camera settings. They have world de-noising has been tested and evaluated on different
evaluated different de-noising methods on the new datasets like DND and NIGHT. DND is a novel benchmark
proposed dataset as well as previous datasets for a proper dataset which consists of realistic photos from 50 scenes
comparison and subsequent analysis. Extensive taken by 4 consumer cameras. The NIGHT dataset is
experimental results demonstrate that the methods designed divided into 20 images (denoted as NIGHT-A) and the
specifically for realistic noise removal based on sparse or other 5 images (denoted as NIGHT-B). Another dataset
low rank theories, achieve good de-noising performance used is RID. It has 20 representative scenes, which are
and are robust. Another observation made by the authors captured under different shooting conditions. The problems
suggests that the proposed dataset is more challenging for faced in real-world de-noising are as follows: (1) The noise
the state-of-the-art methods. In Kim et al. [90], a grouped in real-world noisy images is very complex, which cannot
residual dense network (GRDN) is proposed, which is an be described by simpler distributions like Gaussian or
extended and generalized architecture of the state-of-the-art Poisson. (2) The inherent practicality of real-world noisy
residual dense network (RDN) [91]. The core part of RDN images makes the de-noising more difficult than the
is the grouped residual dense block (GRDB) and used as a synthetic case. (3) the noise distribution may change along
building module of GRDN. Cascading GRDNs aids the de- with the in-camera imaging pipeline [94]. It thus makes the
noising performance significantly. Inspired by the GAN noise distribution in a captured RGB image different from
modeling technique, the authors have made their own its Gaussian assumption in the RAW space. (4) The
generator and discriminator for real-world noise modeling. problem of domain shift cannot be neglected in the practical
Lin et al. [92] have constructed a new dataset to solve the scenario. It can neither be neglected between the synthetic
problem of low availability of proper datasets and obtained and the real-world noise, but the characteristics of real-
the corresponding ground truth by averaging, and then they world noise can also exhibit differences pertaining to
extended them through noise domain adaptation. different camera settings (viz., sensor or aperture size),
Furthermore, they went on to propose an attentive shooting conditions (viz., light, environment, and
generative network by injecting visual attention into the temperature), and imaging pipelines (viz., smartphone and
generative network. During the training, the visual attention
map learns noise regions. The generative network pays
(a) Original Image (b) Noisy Image (𝜎 = 25) (c) DnCNN (PSNR 30.28 dB) [38]
17
(d) FFDNet (PSNR 30.08 dB) [56] (e)IRCNN (PSNR 30.09 dB) [62] (f) ECNDNet(PSNR 30.30 dB) [50]
(g) BRDNet (PSNR 30.50 dB) [51]

FIGURE 10: Butterfly image (Set-12) de-noising results at 𝝈 = 𝟓𝟎
Random-Valued Impulse Noise
Plane Image
(d) Blind CNN De-noised

(b) Noisy Image (c) ANN De-noised Image
(a) Original Image Image (PSNR=37.37 dB)
(20% RVIN) (PSNR=28.96 dB) [72]
[71]
Boat Image
(h) Blind CNN De-noised

(f) Noisy Image (g) ANN De-noised Image
(e) Original Image Image (PSNR=27.32 dB)
(60% RVIN) (PSNR=25.89 dB) [72]
[71]
FIGURE 11: RVIN de-noising results for Plane and Boat image
Real-World De-noising
Noisy Image DnCNN De-noised Image

Original Image
(PSNR=37 MLP De-noised (PSNR=39 (PSNR=37.26
18
dB/SSIM=0.9345) dB/SSIM=0.9695) Image dB/SSIM=0.9389) [38][89]

[35], [89]
Noisy Image RDN De-noised Image [91], GRDN De-noised Image

Original Image (PSNR=26.32 [90] [90]
dB/SSIM=0.7576) (PSNR=39.11 (PSNR=39.59
dB/SSIM=0.9899) dB/SSIM=0.9902)
FIGURE 12: De-noising results for real-world noisy images
Gaussian + RVIN (σ=10, ri=10%)

Barbara Image
Noisy Image SDL De-noised Image (7th iteration)

Original Image
(PSNR=18.6041 dB) [87]
(PSNR=31.7117 dB)
Gaussian + SPIN (σ=10, 𝑝=0.30)
Boat Image
Noisy Image CNN De-noised Image [83]

Original Image
(PSNR=10.65 dB/SSIM=0.0731) (PSNR=31.92 dB/SSIM=0.8596)
High-Density Impulse Noise
Flower Image
Noisy Image (90%)

Original Image CNN+PSO De-noised Image [74]
Gaussian Mixture Noise

Barbara Image
19
WDL (Case 1) WDL (Case 2) K-SVD (PSNR=26.95 dB)

Noisy (PSNR=19.02 dB)
(PSNR=32.39 dB) [84] (PSNR=30.07 dB) [84] [85]
Gaussian Mixture Noise (𝑟1 ∶ 𝑟2 = 0.7 ∶ 0.3, 𝜎1 = 10, 𝜎2 = 50)
W-KSVD (unknown
IRCNN [62] K-SVD (known parameters)
Noisy (PSNR=19.02 dB) parameters)
(PSNR=28.95 dB) [62] (PSNR=27.66 dB) [32]
(PSNR=29.35 dB) [84]
EM-CNN [86]
(PSNR=30.68 dB)
FIGURE 13: De-noising results for mixed noise
Blind De-noising
Uniform noise (10%, [−𝑠, 𝑠]) + Gaussian noise (20%, 𝑁(0,1)) + Gaussian noise (70%, 𝑁(0,0.01))
Noisy BM3D (non-blind) DnCNN-B (blind) GCBD (blind)

Ground Truth
(PSNR=34.86 dB) (PSNR=38.36 dB) (PSNR=38.12 dB) (PSNR=40.30 dB)
FIGURE 14: De-noising results for mixed noise on BSD68
professional camera) even under the same ISO values [93]. image, which does not produce accurate de-noised images
These problems make real-world image de-noising difficult due to inaccurate noise approximation. BM3D is the non-
and still a challenging task. learning blind denoiser based on leveraging self-similarity
VII. BLIND IMAGE DE-NOISERS by joint filter application on self-similarity image patches.
The noise models are defined for particular noise type with The DnCNN [38] model based on residual learning and
known probability distribution function. For example, in batch normalization is the blind Gaussian denoiser in which
case of Gaussian noise the standard deviation of the noisy single model is trained with varying noise levels from zero
image is known and the corresponding de-noised images to fifty-five, down-sampled images with different upscaling
are calculated. However, in real life scenario, the noise can factors and images (JPEG) with multiple quality factors.
be due to combined effect of various sources and noise So, the same network can be used as blind Gaussian
modelling input parameters may not be well-defined. So, denoiser, JPEG image de-blocker and single-image super-
de-noisers which produce de-noised image even when noise resolution. Although FFDNet [56] improves DnCNN
level of input image is not defined are termed as blind de- performance at noise level fifty and seventy-five on BSD-
noisers. The models are trained in such a way that it can 68 dataset, it is a non-blind de-noiser as it requires noise
incorporate wide range of unknown noise levels. The level map at the input. Similarly [60] involves two sub-
another approach is to estimate noise levels of the input networks which are trained separately based on noise-level
20
choice at inference time making it inappropriate for blind Segmentation Dataset comprises of sixty eight natural
level denoising. Blind Universal Image Fusion De- images is commonly used. Kodak-24, LIVE and McMaster
noiser[95] is the network that extracts features to learn an are also being used for synthetic de-noising. RENOIR,
image prior and intermediate noise level values, which is NAM, DND, SIDD and Xu are datasets for real world de-
fed into the fusion part of model for final de-noising. The noising [89]. Some of the benchmark datasets are given in
latest de-noisers such as ADNet[64], BRDNet[51] , SCNN [96].
[49], PReLU [66], [78], [42] are designed for blind TABLE XIII
MODELS USED IN MULTI-DOMAIN IMAGE DE-NOISING
denoising. The blind de-noiser for mixed Gaussian impulse
NOISE SUPPRESSION IN
noise is also being designed [83].The recent research trend MODEL SPECIALITY
VARIOUS MODELS
in the field of computer vision is progressing towards the 1. Single DnCNN
development of universal and blind de-noiser for real-world Intuitively removes
model for blind
de-noising. Gaussian
DnCNN latent clean image in
VII. DESCRIPTION OF DATASET AND SOFTWARE denoising
the hidden layers
TOOLS [38] utilizing the residual
2. Single image
super-resolution
Software: The tremendous success of machine learning learning strategy
3. JPEG image
particularly deep learning is because of the parallel deblocking
computing of GPU. TABLE XII describes popular Uses an integration
of CNN and EM-
machine-learning libraries used for various computer vision based mixed noise
tasks. 1. Removal of
removal to give a
TABLE XII Gaussian mixture
variational method
POPULAR SOFTWARE PACKAGES EM-CNN noise
that can estimate the
[86] 2. Removal of
Software noise parameters
Developer Description Gaussian-Impulse
Packages iteratively to
noise
C++, Python and Matlab categorize noise
Berkeley AI interfaces types and levels in
Caffe
Research Widely used for object each pixel
detection tasks. Uses maximum
Matlab interface, C++ likelihood estimation
Oxford Visual compiler framework and
Matconvnet
Geometry Group Pre-trained models for sparse 1. Gaussian-Gaussian
computer vision tasks. representations over mixture
Facebook AI Python interface, Open a trained dictionary 2. Impulse noise
Pytorch
Research Lab source software W-KSVD and uses a self- 3. Gaussian-impulse
Python library for fast [84] determined noise
Montreal Institute numerical computation weighting data 4. Modified K-SVD
Theano of Learning Used for image de- fidelity function that for weighted rank-
Algorithms noising, classification, detects noise in one approximation
super-resolution terms of different
Faster than Theano estimated noise
Google Brain compiler parameters
Tensorflow
Team C++ and Python Uses variable
interfaces splitting technique to
Python interface bring strong image 1. Gaussian noise
Keras ONEIROS project Acts as interface for prior into model- (grey and color)
Tensorflow library based optimization 2. Mixed Gaussian
IRCNN [62]
Scalable, Flexible, methods and learned noise
Apache Software
MXNet Multiple programming CNN de-noisers are 3. Various low-level
Foundation
languages used as modules in vision applications
model-based
optimization
Datasets: The machine leaning based methods have shown Constructs paired
significant progress due to availability of open access training data from
benchmark datasets. The datasets are available for gray the given noisy
scale de-noising, color image de-noising, medical image de- images, and then 1. Blind Gaussian
GCBD [42] trains a deep noise
noising and real-world de-noising. The training dataset is denoising network 2. Blind Mixed noise
used for training the model, whereas testing dataset images for removing the
are used to assess the de-noising results. The peak signal to noise; GAN is used
noise ratio (PSNR) and structural similarity index (SSIM) to build the dataset
Uses enhanced
are most commonly used image quality assessment metrics. sparse representation
However, there are many image quality assessment metrics in transform domain 1. Gaussian noise
which are given in [39]. The performance comparison of BM3D [20]
where enhancement 2. Color image de-
de-noisers can be done if they use common testing dataset. of sparsity is noising
achieved by 3. Mixed noise
In case of Gaussian de-noisers, Set-12 dataset comprises of grouping similar 2-D
twelve scenes and BSD-68 dataset, i.e. Berkeley image fragments into
21
3-D data arrays PSNR values attain saturation. It implies that further
VIII. RESULT AND DISCUSSION increment in network length does not improve the de-
Out of all machine learning methods, dictionary learning noising performance. Apart from ADNet, BRDNet is the
models performance is inferior in terms of PSNR. The other network that integrates residual learning with batch
disadvantages of dictionary learning are heuristic selection renormalization and dilated convolutions to enhance de-
of the hyperparameters like sparsity level, number of atoms noising performance. The de-noising performance of
and iterations [97]. It fails to learn invariant features such as BRDNet can be attributed to an increase in receptive field
translational, rotation and scale invariance and it is apt for size by dilated convolutions and an increase in network
low dimensional signal only. The machine learning models width by concatenation of two networks. Therefore it
have evolved from fully connected neural networks to CNN overcomes the disadvantages of the previous networks,
based de-noisers. CNN’s have various advantages over such as (a) training difficulty and stagnation of results due
fully connected neural networks such as multi-layer to an increase in network length (b) mini-batch and internal
perceptron. The spatial information is intact in the case of co-variate shift problems. Further, PReLU [66] based edge
CNN whose input is multi-dimensional image data. The aware filter has attained best PSNR results both on Set-12
parameters of CNN are reduced due to weight sharing as a and BSD-68 dataset at different sigma levels. It has used
fixed weight kernel is used. Therefore, reduction in number parameteric rectified linear units as activation, which
of learning parameters, translational invariance and locality overcomes the disadvantage of ReLU by learning in the
due to convolutional operation has given edge to CNN over negative direction. The success can be attributed to the fact
other fully-connected models [98].Most of the CNN de- that this is a hybrid methodology which has the inclusion of
noisers require application oriented large datasets for principal component analysis and edge aware bilateral
supervised learning. The availability of medical image filter. Moreover, CNN uses supervised learning which is
datasets is still challenging as it requires manual becoming computationally demanding with an increase in
intervention for its annotation. Moreover, the de-noising dataset size. Therefore, the generative learning model of
results are almost stagnant after the network attains certain Generative Adversarial Network is being used. The GCBD
depth and there is no significant change by increasing the model gives promising result even in the absence of
number of training images. The CNN methodologies supervised learning data. Its PSNR value is same as that of
involve change in activation function, network depth, loss the DnCNN network on the BSD-68 dataset for noise
function, training dataset, etc. In order to solve this problem level=15, and 25. TABLE III gives a comparative analysis
there is a gradual shift from discriminative learning CNN of machine learning methods on the BSD-68 and Kodak-24
model to generative learning model GAN model. It uses datasets. It has been observed that there is no significant
two neural networks generator and the discriminator, where difference in the PSNR values of different networks. The
generator model creates plausible images and discriminator DIDN network designed with receptive field variation and
model constantly evaluates the generator images as real or modification of U-net architecture designed for semantic
fake. Therefore, both networks work in synchronization and segmentation perform better for color images too as shown
act as an adversarial for each other. The fundamental design in TABLE IV. DRUNet [63] network which is based on
of GAN is based on indirect training of the generator by the deep learning CNN based image prior plugged into the half
discriminator. This falls under the category of semi- quadratic splitting-based de-noising iterative algorithm
supervised learning. The training efficiency of GAN is shows good results on both gray and color images.
more than that of CNN as more features are learned in The impulse de-noisers predict pixels affected by noise in
GAN in the same number of epochs as compared to CNN the first step. It is followed by a noise contamination
[99]. The GAN’s achieve better results with less training determiner and post noise detection processing. The
images as compared to CNN. dictionary learning and CNN-based models are designed for
TABLE II shows de-noising results in terms of PSNR for impulse noise removal. However, the noise ratios are varied
dictionary learning and CNN based networks. The models in a very large range. To overcome the problem of less
progressed from K-SVD, KLLD i.e., from dictionary flexibility due to the unknown severity of contamination,
learning models to CNN based models. The DnCNN model [70] uses a noise ratio predictor that can measure the
is the benchmark residual learning-based Gaussian de- severity of corruption, i.e., the noise ratio of the image
noiser, which has led to the further development of many rapidly and efficiently. Fig. 11 (d) shows that blind CNN
de-noisers. The methodologies involve a change in loss achieves a higher value of PSNR, compared to ANN [72].
function, increase in receptive field size, the change in Blind CNN removes the noise and retains image details, the
number of layers, integration of transform, spatial domain reason being the NRP. It converts the noise mask into a
methods with CNN and inclusion of graph theory in CNN. noise ratio, and, according to this ratio, the most
It can be inferred from the TABLE II that PSNR values appropriate CNN model is selected for de-noising, rather
obtained by different CNN based methods are very close to than restoring image by removing the detected RVIN noise
each other. However, ADNet [64] network with four pixel-by-pixel.
modules suppresses the effect of network length on shallow The Poisson noise is modeled by its peak value and it is
layers and gives good PSNR results on Set-12 dataset. In also categorized into dictionary learning models and CNN
the CNN network, after an optimum number of layers,
22
based models. There is just one dictionary learning model learning approach, graph theory inclusion in neural
which perform de-noising by greedy pursuit algorithm and network, prior design, and receptive field enhancement are
boot strapping based stopping criterion. Gaussian de- some of the areas for future research.
noisers such as DnCNN and IRCNN are also being used for References
Poisson de-noising with different parameter settings. [1] W. Meiniel, J. C. Olivo-Marin, and E. D. Angelini, “Denoising
of microscopy images: A review of the state-of-the-art, and a
Although Gaussian de-noisers are being used for the
new sparsity-based method,” IEEE Trans. Image Process., vol.
poisson noise, the heuristic setting of network parameters is 27, no. 8, pp. 3842–3856, Aug. 2018, doi:
again a big challenge. TABLE V gives poisson de-noising 10.1109/TIP.2018.2819821.
performance on Live1 dataset. DenoiseNet is the first [2] B. Goyal, A. Dogra, S. Agrawal, B. S. Sohi, and A. Sharma,
“Image denoising review: From classical to state-of-the-art
residual learning based CNN de-noiser designed for poisson
approaches,” Inf. Fusion, vol. 55, pp. 220–244, Mar. 2020, doi:
noise. Later, CNN+LSTM and MC2RNet models have 10.1016/j.inffus.2019.09.003.
outperformed DenoiseNet. The CNN + LSTM Poisson de- [3] Y. E. Gökdağ, F. Şansal, and Y. D. Gökdel, “Image denoising
noiser, which uses CNN for feature extraction and LSTM using 2-D wavelet algorithm for Gaussian-corrupted confocal
microscopy images,” Biomed. Signal Process. Control, vol. 54,
layers to store noise components, outperforms DenoiseNet
p. 101594, Sep. 2019, doi: 10.1016/j.bspc.2019.101594.
as given in TABLE V and VI. The inclusion of Blahut- [4] R. Gonzalez and R. Wood, Digital Image Processing, 3rd ed.,
Arimoto algorithm to determine number of CNN layers and London, U.K., Pearson Education, 2008.
learning of residual noise statistics by LSTM improves the [5] S. W. Hasinoff, “Photon, Poisson Noise,” in Computer Vision,
Springer US, pp. 608–610, 2014, doi:
de-noising results of CNN+LSTM. Deep multi-scale cross-
https://doi.org/10.1007/978-0-387-31439-6_482
path concatenation residual network (MC2RNet) which [6] I. Rodrigues, J. Sanches, and J. Bioucas-Dias, “Denoising of
incorporates cross-path concatenation modules for de- medical images corrupted by poisson noise,” in Proceedings -
noising also outperforms CNN based DenoiseNet as given International Conference on Image Processing, ICIP, 2008, pp.
1756–1759, doi: 10.1109/ICIP.2008.4712115.
in TABLE VII and TABLE VIII on Set 10 and BSD-68
[7] H. Yu, M. Ding, X. Zhang, and J. Wu, “PCANet based nonlocal
dataset respectively. Therefore, CNN+LSTM, DenoiseNet means method for speckle noise removal in ultrasound images,”
and MC2RNet are the available CNN based Poisson de- PLoS One,vol.13, no. 10, pp. e0205390, 2018, doi:
noisers which are less in number as compared to Gaussian 10.1371/journal.pone.0205390.
[8] E. E. Kuruoǧlu and J. Zerubia, “Modeling SAR images with a
de-noisers.
generalization of the Rayleigh distribution,” IEEE Trans. Image
The mixed noise can be modeled mathematically in Process., vol. 13, no. 4, pp. 527–533, Apr. 2004, doi:
different ways. There are models designed for mixture of 10.1109/TIP.2003.818017.
impulse and Gaussian noise. The four-stage residual [9] G. Kim, J. Cho, and M. Kang, “Cauchy Noise Removal by
Weighted Nuclear Norm Minimization,” J. Sci. Comput., vol.
learning-based mixed network [82] and de-noiser with two-
83, no. 1, pp. 1–21, Apr. 2020, doi: 10.1007/s10915-020-01203-
stage cascade connection of impulse and Gaussian de- 2.
noiser are used for mixed Gaussian Impulse noise given in [10] R. Lukac, K. N. Plataniotis, B. Smolka, and A. N.
TABLE IX. TABLE X discusses mixed Gaussian noise. Venetsanopoulos, “A multichannel order-statistic technique for
cDNA microarray image processing,” IEEE Trans.
And, TABLE XI discusses blind de-noising. Figs. 10 to 14
Nanobioscience, vol. 3, no. 4, pp. 272–285, Dec. 2004, doi:
depict qualitative results of different images with different 10.1109/TNB.2004.837907.
noise types, i.e., Gaussian noise, impulse noise, mixed [11] Y. Norose, K. Mizutani, N. Wakatsuki, and T. Ebihara, “Noise
noise, real-world, and blind noise. TABLE XIII discusses reduction in ultrasonic computerized tomography by
preprocessing for projection data,” Jpn. J. Appl. Phys., vol. 54,
the methods that extend into multiple domains of image de-
no. 7, p. 07HC12, Jul. 2015, doi: 10.7567/JJAP.54.07HC12.
noising. [12] M. Ye and Y. Qian, “Mixed Poisson-Gaussian noise model
IX. CONCLUSION AND FUTURE SCOPE based sparse denoising for hyperspectral imagery,” in Workshop
In this paper, comprehensive study and analysis of machine on Hyperspectral Image and Signal Processing, Evolution in
learning models for removal of different noises is provided. Remote Sensing, 2012, pp. 1–4, doi:
10.1109/WHISPERS.2012.6874280.
The categorization of different de-noisers is done into [13] L. Fan, F. Zhang, H. Fan, and C. Zhang, “Brief review of image
dictionary learning models such as CNN based models and denoising techniques,” Vis. Comput. Ind. Biomed. Art, vol. 2, no.
GAN based models. The comparative analysis PSNR 7, 2019, doi: 10.1186/s42492-019-0016-7.
results of different de-noisers on some benchmark datasets [14] M. Mafi, H. Martin, M. Cabrerizo, J. Andrian, A. Barreto, and
M. Adjouadi, “A comprehensive survey on impulse and
are provided for better understanding of reader. It has been Gaussian denoising filters for digital images,” Signal
observed that integration of analytical methods in machine Processing, vol. 157. Elsevier B.V., pp. 236–260, 01-Apr-2019,
learning model can further improve the results. Although doi: 10.1016/j.sigpro.2018.12.006.
there are numerous networks designed for synthetic [15] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and
color images,” in Proceedings of the IEEE International
datasets, but real-world image de-noising is still a Conference on Computer Vision, 1998, pp. 839–846, doi:
challenging problem. The GAN based de-noisers are still in 10.1109/iccv.1998.710815.
primitive stage. However, the generative learning based [16] R. G. Gavaskar and K. N. Chaudhury, “Fast Adaptive Bilateral
GAN and deep belief networks can perform unsupervised Filtering,” IEEE Trans. Image Process., vol. 28, no. 2, pp. 779–
790, Feb. 2019, doi: 10.1109/TIP.2018.2871597.
learning to certain extent unlike CNN. The future prospects [17] L. Zhang, W. Dong, D. Zhang, and G. Shi, “Two-stage image
lie in design of real-world de-noisers with unsupervised denoising by principal component analysis with local pixel
learning framework for practical applications. The transfer grouping,” Pattern Recognit., vol. 43, pp. 1531–1549, 2010, doi:
10.1016/j.patcog.2009.09.023.
23
[18] R. Guhathakurta, “Denoising of image: A wavelet based 2399, doi: 10.1109/CVPR.2012.6247952.

approach,” in 2017 8th Industrial Automation and [36] X. Tang, L. Zhang, and X. Ding, “SAR image despeckling with
Electromechanical Engineering Conference, IEMECON 2017, a multilayer perceptron neural network,” International Journal
2017, pp. 194–197, doi: 10.1109/IEMECON.2017.8079587. of Digital Earth, vol. 12, no. 3. Taylor and Francis Ltd., pp. 354–
[19] S. D. Ruikar and D. D. Doye, “Wavelet Based Image Denoising 374, 04-Mar-2019, doi: 10.1080/17538947.2018.1447032.
Technique,” International Journal of Advanced Computer [37] Y. Chen and T. Pock, “Trainable Nonlinear Reaction Diffusion:
Science and Application, vol. 2, no.3,pp. 49-53, 2011,doi: A Flexible Framework for Fast and Effective Image
10.14569/IJACSA.2011.020309 Restoration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39,
[20] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image no. 6, pp. 1256–1272, Jun. 2017, doi:
denoising by sparse 3-D transform-domain collaborative 10.1109/TPAMI.2016.2596743.
filtering,” IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080– [38] K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, "Beyond a
2095, Aug. 2007, doi: 10.1109/TIP.2007.901238. Gaussian Denoiser: Residual Learning of Deep CNN for Image
[21] K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, and K. E. Bm3d, Denoising," in IEEE Transactions on Image Processing, vol. 26,
“BM3D Image Denoising with Shape-Adaptive Principal no. 7, pp. 3142-3155, July 2017, doi:
Component Analysis,” in Signal Processing with Adaptive 10.1109/TIP.2017.2662206.
Sparse Structured Representations, 2009, p. 00369582. [39] R. S. Thakur, R. N. Yadav, and L. Gupta, “State-of-art analysis
[22] M. Maggioni, V. Katkovnik, K. Egiazarian, and A. Foi, of image denoising methods using convolutional neural
“Nonlocal transform-domain filter for volumetric data denoising networks,” IET Image Process., vol. 13, no. 13, pp. 2367–2380,
and reconstruction,” IEEE Trans. Image Process., vol. 22, no. 1, 2019, doi: 10.1049/iet-ipr.2019.0157.
pp. 119–133, 2013, doi: 10.1109/TIP.2012.2210725. [40] F. Zhang, N. Cai, J. Wu, G. Cen, H. Wang, and X. Chen, “Image
[23] Jean-Luc Starck, E. J. Candes and D. L. Donoho, "The curvelet denoising method based on a deep convolution neural network,”
transform for image denoising," in IEEE Transactions on Image IET Image Process., vol. 12, no. 4, pp. 485–493, 2018, doi:
Processing, vol. 11, no. 6, pp. 670-684, June 2002, doi: 10.1049/iet-ipr.2017.0389.
10.1109/TIP.2002.1014998 [41] I. J. Goodfellow, J.P Abadie, M. Mirza et al., “Generative
[24] M. N. Do and M. Vetterli, “The contourlet transform: An Adversarial Nets.”in Proceedings of the 27th International
efficient directional multiresolution image representation,” IEEE Conference on Neural Information Processing
Trans. Image Process., vol. 14, no. 12, pp. 2091–2106, Dec. System,2014,pp.2672-2680, doi: 10.5555/2969033.2969125
2005, doi: 10.1109/TIP.2005.859376. [42] J. Chen, J. Chen, H. Chao, and M. Yang, “Image Blind
[25] S. Schulte, M. Nachtegael, V. DeWitte, D. Van der Weken, and Denoising with Generative Adversarial Network Based Noise
E. E. Kerre, “A fuzzy impulse noise detection and reduction Modeling,” in Proceedings of the IEEE Computer Society
method,” IEEE Trans. Image Process., vol. 15, no. 5, pp. 1153– Conference on Computer Vision and Pattern Recognition, 2018,
1162, May 2006, doi: 10.1109/TIP.2005.864179. pp. 3155–3164, doi: 10.1109/CVPR.2018.00333.
[26] J. H. Wang, W. J. Liu, and L. Da Lin, “Histogram-based fuzzy [43] P. Chatterjee and P. Milanfar, “Clustering-based denoising with
filter for image restoration,” IEEE Transactions on Systems, locally learned dictionaries,” IEEE Trans. Image Process., vol.
Man, and Cybernetics, Part B: Cybernetics, vol. 32, no. 2. pp. 18, no. 7, pp. 1438–1451, 2009, doi: 10.1109/TIP.2009.2018575.
230–238, Apr-2002, doi: 10.1109/3477.990880. [44] R. Yan, L. Shao, and Y. Liu, “Nonlocal hierarchical dictionary
[27] M. Pathak and G. R. Sinha, “A Survey of Fuzzy Based Image learning using wavelets for image denoising,” IEEE Trans.
Denoising Techniques,” IOSR J. Electron. Commun. Eng., vol. Image Process., vol. 22, no. 12, pp. 4689–4698, 2013, doi:
9, no. 4, pp. 27–36, 2014. 10.1109/TIP.2013.2277813.
[28] P. Rodríguez, “Total variation regularization algorithms for [45] T. Xia, H. Dong, X. Wang, X. Yu, Y. Liu, and X. Li, “Image
images corrupted with different noise models: A review,” denoising based on dictionary learning of mean corrected
Journal of Electrical and Computer Engineering. 2013, doi: atoms,” in Proceedings - 18th IEEE/ACIS International
10.1155/2013/217021. Conference on Computer and Information Science, ICIS 2019,
[29] S. Gu, L. Zhang, W. Zuo, and X. Feng, “Weighted nuclear norm 2019, pp. 202–207, doi: 10.1109/ICIS46139.2019.8940197.
minimization with application to image denoising,” in [46] A. Kumar and S. Kataria, “Dictionary Learning Based
Proceedings of the IEEE Computer Society Conference on Applications in Image Processing using Convex Optimisation,”
Computer Vision and Pattern Recognition, 2014, pp. 2862– in International Conference on Signal Processing and Integrated
2869, doi: 10.1109/CVPR.2014.366. Networks (SPIN), 2017.
[30] D. L. Donoho, M. Elad, and V. N. Temlyakov, “Stable recovery [47] U. Schmidt and S. Roth, “Shrinkage fields for effective image
of sparse overcomplete representations in the presence of noise,” restoration,” in Proceedings of the IEEE Computer Society
IEEE Trans. Inf. Theory, vol. 52, no. 1, pp. 6–18, Jan. 2006, doi: Conference on Computer Vision and Pattern Recognition, 2014,
10.1109/TIT.2005.860430. pp. 2774–2781, doi: 10.1109/CVPR.2014.349.
[31] S. Chatterjee, R. S. Thakur, R. N. Yadav, L. Gupta, and D. K. [48] W. Bae, J. Yoo, and J. C. Ye, “Beyond Deep Residual Learning
Raghuvanshi, “Review of noise removal techniques in ECG for Image Restoration: Persistent Homology-Guided Manifold
signals,” IET Signal Process., vol. 14, no. 9, pp. 569–590, Dec. Simplification,” in IEEE Computer Society Conference on
2020, doi: 10.1049/iet-spr.2020.0104. Computer Vision and Pattern Recognition Workshops, 2017, vol.
[32] M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm 2017-July, pp. 1141–1149, doi: 10.1109/CVPRW.2017.152.
for designing overcomplete dictionaries for sparse [49] K. Isogawa, T. Ida, T. Shiodera, and T. Takeguchi, “Deep
representation,” IEEE Trans. Signal Process., vol. 54, no. 11, pp. shrinkage convolutional neural network for adaptive noise
4311–4322, Nov. 2006, doi: 10.1109/TSP.2006.881199. reduction,” IEEE Signal Process. Lett., vol. 25, no. 2, pp. 224–
[33] R. Rubinstein, M. Zibulevsky, and M. Elad, “Efficient 228, Feb. 2018, doi: 10.1109/LSP.2017.2782270.
Implementation of the K-SVD Algorithm using Batch [50] C. Tian, Y. Xu, L. Fei, J. Wang, J. Wen, and N. Luo, “Enhanced
Orthogonal Matching Pursuit,” Cs Tech., vol. 40, no. 8, pp. 1– CNN for image denoising,” CAAI Trans. Intell. Technol., vol. 4,
15, 2008. no. 1, pp. 17–23, Mar. 2019, doi: 10.1049/trit.2018.1054.
[34] L. Ma, L. Moisan, J. Yu, and T. Zeng, “A dictionary learning [51] C. Tian, Y. Xu, and W. Zuo, “Image denoising using deep CNN
approach for Poisson image Deblurring,” IEEE Trans. Med. with batch renormalization,” Neural Networks, vol. 121, pp.
Imaging, vol. 32, no. 7, pp. 1277–1289, 2013, doi: 461–473, Jan. 2020, doi: 10.1016/j.neunet.2019.08.022.
10.1109/TMI.2013.2255883. [52] B. Park, S. Yu, and J. Jeong, “Densely connected hierarchical
[35] H. C. Burger, C. J. Schuler, and S. Harmeling, “Image network for image denoising,” in IEEE Computer Society
denoising: Can plain neural networks compete with BM3D?,” in Conference on Computer Vision and Pattern Recognition
Proceedings of the IEEE Computer Society Conference on Workshops, 2019, vol. 2019-June, pp. 2104–2113, doi:
Computer Vision and Pattern Recognition, 2012, pp. 2392– 10.1109/CVPRW.2019.00263.
24
[53] P. Liu, H. Zhang, W. Lian, and W. Zuo, “Multi-Level Wavelet Dec. 2018, doi: 10.3390/a12010007.
Convolutional Neural Networks,” IEEE Access, vol. 7, pp. [70] B. Deka and P. K. Bora, “Removal of random-valued impulse
74973–74985, 2019, doi: 10.1109/ACCESS.2019.2921451. noise using sparse representation,” in 2011 National Conference
[54] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional on Communications, NCC 2011, 2011, doi:
networks for biomedical image segmentation,” in Lecture Notes 10.1109/NCC.2011.5734762.
in Computer Science (including subseries Lecture Notes in [71] J. Chen, G. Zhang, S. Xu, and H. Yu, “A blind CNN denoising
Artificial Intelligence and Lecture Notes in Bioinformatics), model for random-valued impulse noise,” IEEE Access, vol. 7,
2015, vol. 9351, pp. 234–241, doi: 10.1007/978-3-319-24574- pp. 124647–124661, 2019, doi:
4_28. 10.1109/ACCESS.2019.2938799.
[55] S. Yu, B. Park, and J. Jeong, “Deep iterative down-up CNN for [72] I. Turkmen, “The ANN based detector to remove random-valued
image denoising,” in IEEE Computer Society Conference on impulse noise in images,” J. Vis. Commun. Image Represent.,
Computer Vision and Pattern Recognition Workshops, 2019, vol. vol. 34, no. October, pp. 28–36, 2016, doi:
2019-June, pp. 2095–2103, doi: 10.1109/CVPRW.2019.00262. 10.1016/j.jvcir.2015.10.011.
[56] K. Zhang, W. Zuo, and L. Zhang, “FFDNet: Toward a fast and [73] G. Li, X. Xu, M. Zhang, and Q. Liu, “Densely connected
flexible solution for CNN-Based image denoising,” IEEE Trans. network for impulse noise removal,” Pattern Anal. Appl., vol.
Image Process., vol. 27, no. 9, pp. 4608–4622, 2018, doi: 23, no. 3, pp. 1263–1275, Aug. 2020, doi: 10.1007/s10044-020-
10.1109/TIP.2018.2839891. 00871-y.
[57] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic [74] H. Y. Khaw, F. C. Soon, J. H. Chuah, and C. O. Chow, “High-
optimization,” in 3rd International Conference on Learning density impulse noise detection and removal using deep
Representations, ICLR 2015 - Conference Track Proceedings, convolutional neural network with particle swarm optimisation,”
2015. IET Image Process., vol. 13, no. 2, pp. 365–374, Feb. 2019, doi:
[58] C. Cruz, A. Foi, V. Katkovnik, and K. Egiazarian, “Nonlocality- 10.1049/iet-ipr.2018.5776.
Reinforced Convolutional Neural Networks for Image [75] L. Jin, W. Zhang, G. Ma, and E. Song, “Learning deep CNNs for
Denoising,” IEEE Signal Process. Lett., vol. 25, no. 8, pp. 1216– impulse noise removal in images,” J. Vis. Commun. Image
1220, 2018, doi: 10.1109/LSP.2018.2850222. Represent., vol. 62, pp. 193–205, 2019, doi:
[59] D. Valsesia, G. Fracastoro, and E. Magli, “Image Denoising with 10.1016/j.jvcir.2019.05.005.
Graph-Convolutional Neural Networks,” in Proceedings - [76] R. Giryes and M. Elad, “Sparsity-based poisson denoising with
International Conference on Image Processing, ICIP, 2019, vol. dictionary learning,” IEEE Trans. Image Process., vol. 23, no.
2019-September, pp. 2399–2403, doi: 12, pp. 5057–5069, Dec. 2014, doi: 10.1109/TIP.2014.2362057.
10.1109/ICIP.2019.8803367. [77] W. Kumwilaisak, T. Piriyatharawet, P. Lasang, and N.
[60] S. Lefkimmiatis, “Universal Denoising Networks : A Novel Thatphithakkul, “Image Denoising with Deep Convolutional
CNN Architecture for Image Denoising,” in Proceedings of the Neural and Multi-Directional Long Short-Term Memory
IEEE Computer Society Conference on Computer Vision and Networks under Poisson Noise Environments,” IEEE Access,
Pattern Recognition, 2018, pp. 3204–3213, doi: vol. 8, pp. 86998–87010, 2020, doi:
10.1109/CVPR.2018.00338. 10.1109/ACCESS.2020.2991988.
[61] W. Dong, P. Wang, W. Yin, G. Shi, F. Wu, and X. Lu, [78] Y. Su, Q. Lian, X. Zhang, B. Shi, and X. Fan, “Multi-scale
“Denoising Prior Driven Deep Neural Network for Image Cross-path Concatenation Residual Network for Poisson
Restoration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, denoising,” IET Image Process., vol. 13, no. 8, pp. 1295–1303,
no. 10, pp. 2305–2318, 2019, doi: Jun. 2019, doi: 10.1049/iet-ipr.2018.5941.
10.1109/TPAMI.2018.2873610. [79] R. Tal, O. Litany, R. Giryes, and A. M. Bronstein,
[62] K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep CNN “[1701.01687] Deep Convolutional Denoising of Low-Light
denoiser prior for image restoration,” in Proceedings - 30th Images,” https://arxiv.org/abs/1701.01687, 2017. [Online].
IEEE Conference on Computer Vision and Pattern Recognition, Available: https://arxiv.org/abs/1701.01687.
CVPR 2017, 2017, vol. 2017-January, pp. 2808–2817, doi: [80] T. Remez, O. Litany, R. Giryes, and A. M. Bronstein, “Class-
10.1109/CVPR.2017.300. Aware Fully Convolutional Gaussian and Poisson Denoising,”
[63] K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. IEEE Trans. Image Process., vol. 27, no. 11, pp. 5707–5722,
Timofte, “Plug-and-Play Image Restoration with Deep Denoiser Nov. 2018, doi: 10.1109/TIP.2018.2859044.
Prior,” arXiv, 31-Aug-2020. [81] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Color
[Online].Available:http://arxiv.org/abs/2008.13751. image denoising via sparse 3D collaborative filtering with
[64] C. Tian, Y. Xu, Z. Li, W. Zuo, L. Fei, and H. Liu, “Attention- grouping constraint in luminance-chrominance space,” in
guided CNN for image denoising,” Neural Networks, vol. 124, Proceedings - International Conference on Image Processing,
pp. 117–129, Apr. 2020, doi: 10.1016/j.neunet.2019.12.024. ICIP, 2006, vol. 1, doi: 10.1109/ICIP.2007.4378954.
[65] R. Couturier, G. Perrot, and M. Salomon, “Image denoising [82] M. T. Islam, S. M. Mahbubur Rahman, M. Omair Ahmad, and
using a deep encoder-decoder network with skip connections,” M. N. S. Swamy, “Mixed Gaussian-impulse noise reduction
in Lecture Notes in Computer Science (including subseries from images using convolutional neural network,” Signal
Lecture Notes in Artificial Intelligence and Lecture Notes in Process. Image Commun., vol. 68, no. June, pp. 26–41, 2018,
Bioinformatics), 2018, vol. 11306 LNCS, pp. 554–565, doi: doi: 10.1016/j.image.2018.06.016.
10.1007/978-3-030-04224-0_48. [83] R. Abiko and M. Ikehara, “Blind Denoising of Mixed Gaussian-
[66] R. S. Thakur, R. N. Yadav, and L. Gupta, “PReLU and edge- impulse Noise by Single CNN,” in ICASSP, IEEE International
aware filter-based image denoiser using convolutional neural Conference on Acoustics, Speech and Signal Processing -
network,” IET Image Process., vol. 14, no. 15, pp. 3869–3879, Proceedings, 2019, vol. 2019-May, pp. 1717–1721, doi:
Dec. 2020, doi: 10.1049/iet-ipr.2020.0717. 10.1109/ICASSP.2019.8683878.
[67] Y. Zhong, L. Liu, D. Zhao, and H. Li, “A generative adversarial [84] J. Liu, X. C. Tai, H. Huang, and Z. Huan, “A weighted
network for image denoising,” Multimed. Tools Appl., vol. 79, dictionary learning model for denoising images corrupted by
no. 23–24, pp. 16517–16529, Jun. 2020, doi: 10.1007/s11042- mixed noise,” IEEE Trans. Image Process., vol. 22, no. 3, pp.
019-7556-x. 1108–1120, 2013, doi: 10.1109/TIP.2012.2227766.
[68] S. Wang et al., “Dictionary learning based impulse noise [85] M. Elad and M. Aharon, “Image denoising via sparse and
removal via L1-L1 minimization,” Signal Processing, vol. 93, redundant representations over learned dictionaries,” IEEE
no. 9, pp. 2696–2708, 2013, doi: 10.1016/j.sigpro.2013.03.005. Trans. Image Process., vol. 15, no. 12, pp. 3736–3745, Dec.
[69] D. Guo, Z. Tu, J. Wang, M. Xiao, X. Du, and X. Qu, “Salt and 2006, doi: 10.1109/TIP.2006.881969.
Pepper Noise Removal with Multi-Class Dictionary Learning [86] F. Wang, H. Huang, and J. Liu, “Variational-Based Mixed Noise
and L0 Norm Regularizations,” Algorithms, vol. 12, no. 1, p. 7, Removal with CNN Deep Learning Regularization,” IEEE
25
Trans. Image Process., vol. 29, pp. 1246–1258, 2020, doi:

10.1109/TIP.2019.2940496.
[87] H. Zhu and M. K. Ng, “Structured Dictionary Learning for
Image Denoising under Mixed Gaussian and Impulse Noise,”
IEEE Trans. Image Process., vol. 29, pp. 6680–6693, 2020, doi:
10.1109/TIP.2020.2992895.
[88] M. Mafi, W. Izquierdo, H. Martin, M. Cabrerizo, and M.
Adjouadi, “Deep convolutional neural network for mixed
random impulse and gaussian noise reduction in digital images,”
IET Image Process., vol. 14, no. 15, pp. 3791–3801, Dec. 2020,
doi: 10.1049/iet-ipr.2019.0931.
[89] J. Xu, H. Li, Z. Liang, D. Zhang, and L. Zhang, “[1804.02603]
Real-world Noisy Image Denoising: A New Benchmark.”
[Online]. Available: https://arxiv.org/abs/1804.02603.
[Accessed: 30-Dec-2020].
[90] D. W. Kim, J. R. Chung, and S. W. Jung, “GRDN:Grouped
residual dense network for real image denoising and GAN-based
real-world noise modeling,” in IEEE Computer Society
Conference on Computer Vision and Pattern Recognition
Workshops, 2019, vol. 2019-June, pp. 2086–2094, doi:
10.1109/CVPRW.2019.00261.
[91] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual
dense network for image restoration,” IEEE Trans. Pattern
Anal., Dec. 2018, doi: 10.1109/tpami.2020.2968521.
[92] K. Lin, T. H. Li, S. Liu, and G. Li, “Real photographs denoising
with noise domain adaptation and attentive generative
adversarial network,” IEEE Comput. Soc. Conf. Comput. Vis.
Pattern Recognit. Work., vol. 2019-June, pp. 1717–1721, 2019,
doi: 10.1109/CVPRW.2019.00221.
[93] C. Chen, Z. Xiong, X. Tian, Z. J. Zha, and F. Wu, “Real-World
Image Denoising with Deep Boosting,” IEEE Trans. Pattern
Anal. Mach. Intell., vol. 42, no. 12, pp. 3071–3087, Dec. 2020,
doi: 10.1109/TPAMI.2019.2921548.
[94] S. Nam, Y. Hwang, Y. Matsushita, and S. J. Kim, “A Holistic
Approach to Cross-Channel Image Noise Modeling and Its
Application to Image Denoising,” in Proceedings of the IEEE
Computer Society Conference on Computer Vision and Pattern
Recognition, 2016, vol. 2016-December, pp. 1683–1691, doi:
10.1109/CVPR.2016.186.
[95] M. El Helou and S. Susstrunk, “Blind Universal Bayesian Image
Denoising with Gaussian Noise Level Learning,” IEEE Trans.
Image Process., vol. 29, pp. 4885–4897, 2020, doi:
10.1109/TIP.2020.2976814.
[96] C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo, and C. W. Lin,
“Deep learning on image denoising: An overview,” Neural
Networks, vol. 131, pp. 251–275, 2020, doi:
10.1016/j.neunet.2020.07.025.
[97] T. Tong, J. Caballero, K. Bhatia, and D. Rueckert, “Dictionary
learning for medical image denoising, reconstruction, and
segmentation,” in Machine Learning and Medical Imaging,
Elsevier Inc., 2016, pp. 153–181.
[98] A. Lucas, M. Iliadis, R. Molina, and A. K. Katsaggelos, “Using
Deep Neural Networks for Inverse Problems in Imaging: Beyond
Analytical Methods,” IEEE Signal Process. Mag., vol. 35, no. 1,
pp. 20–36, Jan. 2018, doi: 10.1109/MSP.2017.2760358.
[99] S. Barua, S. M. Erfani, and J. Bailey, “FCC-GAN: A Fully
Connected and Convolutional Net Architecture for GANs,”
arXiv, May 2019.
26

Image De-Noising With Machine Learning A Review

Uploaded by

Copyright:

Available Formats

Image De-Noising With Machine Learning A Review

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Image De-Noising With Machine Learning A Review

Uploaded by

Copyright:

Available Formats

This article has been accepted for publication in a future issue of this journal, but has not been

Image De-noising with Machine Learning: A

Corresponding author: Rini Smita Thakur (e-mail: rinithakur66@gmail.com).

I. INTRODUCTION different types of noise based on pdf are Gaussian,

Spatial Domain Transform Domain Fuzzy based Machine Learning

Linear and Sparsity based dictionary

FIGURE 1. Block Diagram illustrating classification of image de-noising techniques

whereas the discriminator determines whether the generated 𝜕 1

Input Sparse Dictionary De-noised

Input First layer Intermediate Last layer Output

Conv+BN+ReLU Conv Cat Tanh

FIGURE8: Architecture of ADNet [64]

Random Patch Noise Contamination Impulse Noise

FIGURE 9: Block Diagram depicting the basic concept of impulse de-noisers

class-aware by exploiting semantic class information that A. METHODOLOGIES OF DICTIONARY LEARNING

K-LLD [43] - 31.77 - - - - 28.40 - - - 28.17 - -

DHDN [52] 35.24 - 30.11 - 28.01

BRDNet [51] 34.88 32.41 30.80 29.22 27.49

(g) BRDNet (PSNR 30.50 dB) [51]

(d) Blind CNN De-noised

(h) Blind CNN De-noised

Noisy Image DnCNN De-noised Image

dB/SSIM=0.9345) dB/SSIM=0.9695) Image dB/SSIM=0.9389) [38][89]

Noisy Image RDN De-noised Image [91], GRDN De-noised Image

Gaussian + RVIN (σ=10, ri=10%)

Noisy Image SDL De-noised Image (7th iteration)

Noisy Image CNN De-noised Image [83]

Noisy Image (90%)

Gaussian Mixture Noise

WDL (Case 1) WDL (Case 2) K-SVD (PSNR=26.95 dB)

Noisy BM3D (non-blind) DnCNN-B (blind) GCBD (blind)

[18] R. Guhathakurta, “Denoising of image: A wavelet based 2399, doi: 10.1109/CVPR.2012.6247952.

Trans. Image Process., vol. 29, pp. 1246–1258, 2020, doi:

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.