imagesmoothing
imagesmoothing
imagesmoothing
NOTE
A conceptually simple but effective noise smoothing algorithm is described. This filter is
motivated by the sigma probability of the Gaussian distribution, and it smooths the image
noise by averaging only those neighborhood pixels which have the intensities within a fixed
sigma range of the center pixel. Consequently, image edges are preserved, and subtle details
and thin tines such as roads are retained. The characteristics of this smoothing algorithm are
analyzed and compared with several other known filtering algorithms by their ability to retain
subtle details, preserving edge shapes, sharpening ramp edges, etc. The comparison also
indicates that the sigma filter is the most computationally efficient filter among those evaluated.
The filter can be easily extended into several forms which can be used in contrast enhancement,
image segmentation, and smoothing signal-dependent noisy images. Several test images 128 X
128 and 256 X 256 pixels in size are used to substantiate its characteristics. The algorithm can
be easily extended to 3-D image smoothing.
1. INTRODUCTION
Generally, digital image smoothing techniques fall into two categories. In the first
category, the noisy image is processed globally in the sense that the whole or a large
section of a noisy image is correlated to obtain a smoothed image. Techniques in the
transform domain using Wiener or least squares filtering [l, 21 and techniques
applying one-dimensional or two-dimensional Kalman filter are in this category.
Statistical models for the signal (noise free image) and the noise are required for the
implementation of these techniques. Unfortunately, the statistical model for most
images is either unknown or impossible to describe adequately with a simple random
process. The smoothed images display blurred edges and conceal subtle details. In
addition, the techniques are computationally costly. In the second category local
operators are applied to noisy images. Only those pixels in a small neighborhood of
the concerned pixel are involved in the computation. The immediate advantage of
these techniques is their efficiency. They have great potential for real-time or near
real-time implementation, because several pixels can be processed in parallel without
waiting for their neighboring pixels to be processed. Recent research in image
smoothing and segmentation favors the local techniques.
There are many local smoothing methods. The well-known median filter [4] in one
or two dimensions has attracted much attention. The edge preserving smoothing of
Nagao and Matsuyama [5], the gradient inverse weighting scheme of Wang et al. [6],
the box filtering algorithm [7], and the local statistics method of Lee [8, 91 are just a
few other algorithms in this category. Obviously, it is nearly impossible to rank
them, because an algorithm may be effective for a class of images, but ineffective for
others. In this paper a new class of local smoothing schemes is introduced. It is
motivated by the sigma probability of the Gaussian distribution. The basic idea is to
replace the pixel to be processed by the average of only those neighboring pixels
having their intensity within a fixed sigma range of the center pixel. Replacing the
255
0734-189X/83 $3.00
Copyright 0 1983 by Academic Press, Inc.
All rights of reproduction in any form reserved
256 JONG-SEN LEE
center pixel by the average of selected neighboring pixels has been explored by many
algorithms. Nagao’s filter [5] replaces the center pixel by the average of a subregion
which has minimum variance. Lee [9] in his refined local statistics method selected
the region by using gradient information. Graham [lo] and Prewitt [ll] replace a
pixel by the average of the surrounding area if the absolute value of their difference
is smaller than some threshold. Rosenfeld [l] in his region growing and tracking
algorithm excludes high contrast edges, lines, and points from the average by judging
the gray level difference between the average of the region and the new pixel. The
extended box-filtering algorithm [7] restricts the average to only neighboring pixels
within a fixed intensity range. The main difference between the box filter and the
sigma filter of this paper is that the former has the intensity range fixed throughout
the whole image, while the latter lets the intensity range float with the intensity of
the center pixel. The advantages are numerous: (1) noise near edge areas will be
smoothed without blurring the edge because only pixels on one side of the edge are
included in the average; (2) subtle details of several pixel clusters and linear features
of one to three pixels in width will be preserved since only those pixels and not the
background are included in the average; (3) it will not create artifacts and will retain
shapes, because no directional masks are used, unlike the algorithms of Nagao [S]
and Lee [9]; (4) it is computationally efficient, since only simple compare and fixed
point add instructions are involved.
A comparison of the sigma filter, the gradient inverse filter, Nagao’s filter, and the
median filter are conducted in this paper. Comparisons are based on the following
criteria: (1) effectiveness in smoothing noise; (2) preservation of subtle details and
linear features; (3) immunity from shape distortion; (4) retention of step edges and
sharpening of ramp edges; (5) removal of high-contrast spot noise; (6) computa-
tional efficiency. For the smoothing algorithms to be effective they are applied
iteratively three times to test images of dimension 128 X 128. In many respects the
sigma filter performs better than other algorithms except as regards the ability to
remove sharp spot noise. Some methods of reducing this deficiency are presented.
The sigma filter and its extended forms are discussed in the next section. ,A
comparison study of the algorithms is given in Section 3. Section 4 is devoted to
presenting the characteristics of the sigma filter and its extensions to images with
multiplicative noise.
2. THE SIGMA FILTER
the range of two standard deviations. In image smoothing, any pixel outside the
two-sigma range most likely comes from a different population and, therefore,
should be excluded from the average. If we assume that the a priori mean is the gray
level of the pixel to be smoothed, we can establish a two-sigma range from the gray
level and include in the average only those pixels within the two-sigma intensity. Let
xi, j be the intensity or gray level of pixel (i, j), and &, j be the smoothed pixel (i, j).
Also we assume that the noise is additive with zero mean and standard deviation u.
The sigma filter procedure is then described as follows:
(1) Establish an intensity range (xi, j + A, xi, - A), where A = 2~.
(2) Sum all pixels which lie within the intensity range in a (2n + 1,2m + 1)
window.
(3) Compute the average by dividing the sum by the number of pixels in the
sum.
(4) Then R;, j = the average. (To reduce sharp spot noise, step (4) will be
modified later in this section.)
Or, mathematically, let
The two-sigma range is generally large enough to include 95.5% of the pixels from
the same distribution in the window, yet in most cases it is small enough to exclude
pixels representing high-contrast edges and subtle details. Linear features such as
roads one or two pixels wide are retained, because only those pixels with intensity
near that of the feature are included in the average. The main drawback is that sharp
spot noise represented by clusters of one or two pixels will not be smoothed. This
could be very annoying especially for a fairly noisy image. To remedy this, we shall
replace the two-sigma average with the center pixel’s immediate neighbor average, if
M, the number of pixels within the intensity range, is less than a prespecified value
K. In other words, step (4) is replaced by
FIG. 1. This figure shows the results of the 7 x 7 sigma filter when applied once, twice, and three
times to a medical image. The biased sigma filter when applied to (D) is shown in (E), and the result of
the 3 x 3 median filter is shown in (F) for comparison.
SIGMA FILTER IMAGE SMOOTHING 259
For images with unknown noise characteristics, the intensity range A can be
determined either from a rough estimation of the noise standard deviation in a flat
area, or from the desirability of retaining the gray level difference between the
desirable features and its background. The sigma filter can be applied repeatedly
with reduced u after each iteration. Two or three iterations are generally sufficient to
reduce the noise level significantly.
As an illustration, Fig. l(A) shows a medical image of cell structure. The results of
applying the 7 x 7 sigma filter once, twice and three times are shown in Fig. l(B),
(C), and (D), respectively. The result of applying the median filter twice is shown in
(F). It should be noted that (E) is the result of applying a derivative version of the
sigma filter, to be discussed in Section 4.
3. A COMPARISON OF LOCALLY SMOOTHING ALGORITHMS
wherek,l= {-l,O,-tl}.
(2) Compute weights for the eight neighbors:
1 gkl 1
and w~,~=-
Wk,l=-‘d
2 Q 2
Nagao and Matsuyama [5] proposed an algorithm which selects the most homoge-
neous neighborhood and replaces the pixel by its neighborhood average. They
260 JONG-SEN LEE
FIG. 2. Nagao’s filter with its nine subregions in a 5 X 5 window. Two of the eight directional
subregions are shown on the left, and the ninth mask is on the right. The center pixel is replaced by the
mean of the subregion having the minimum variance.
FIG. 3. The noise corrupt images ((A) and (F)) of bars with increasing width (one pixel, three
pixels,. , 15 pixels). Several noise smoothing algorithms are applied, and the results are shown in
(B)-(E) and (G)-(J), respectively. (B) and (G) are the results of applying the sigma filter, while (C) and
(H), (D) and (I), and (E) and (J) are the results of the gradient inverse filter, Nagao’s filter and the 3 x 3
median filter, respectively.
262 JONG-SEN LEE
FIG 4. (A) An aerial image (courtesy of Image Processing Institute, USC); (B) Gaussian noise of
standa id deviation 20 is artificially added to (A); (c), (D), (E), and (F) are the results applied by Ithe
sigma filter, gradient inverse filter, Nagao’s filter, and the 3 X 3 median filter, respectively.
SIGMA FILTER IMAGE SMOOTHING 263
TABLE 1
Comparison of Reduction in Noise Standard Deviation
PIXEL DISTANCE
FIG. 5. The effect of Nagao’s filter as applied in the one-dimensional case, to illustrate tbat it will blur
and eventually devour any feature with dimension of three pixels or less. (A) shows the original
one-dimensional three-pixel-wide pulse; the center pixel drops in value after one application as shown in
(B); (C) shows that the value of the pulse decreases even more after another application.
in Fig. 4(E), does create significant distortion because of the directional subregion
average. It will round off comers of less than 90”. Median filter is known to create
artifacts. The 3 x 3 median filter will round off comers and produce patterns of
patches, the same as Nagao’s filter. As shown in Fig. 4(C), the sigma filter is
practically free of shape distortion.
(4) Retention of Step Edges and Sharpening Ramp Edges
The intensity variations in the direction perpendicular to a sharp edge in the
image plane form a step edge. Retaining the sharpness of a step edge is highly
desirable in both image smoothing and segmentation. The gradient inverse filter will
blur the step edge, as it computes the average on all pixels. The median filter will
maintain a noise free step edge, but it will smear a noise step edge. Figure 6 shows a
3 x 3 mask moving through an edge. Assuming the edge is contaminated by noise,
the 3 x 3 median filter replaces the center pixel with the fifth least bright pixel of the
six pixels on the left side of the edge, while as the window moves right by one pixel,
the center pixel is replaced by the fifth brightest among the six pixels on the dark
side of the edge. Consequently the sharpness of the edge is degraded. The s&&a
filter, however, retains its sharpness by replacing the center pixel by the average of
the six pixels.
Sharpening a ramp edge is generally of interest in studies of image segmentation
by gray level difi’erence. In this application Nagao’s filter is excellent due to its
SIGMA FILTER IMAGE SMOOTHING 265
EDGE BOUNDARY
FIG. 6. The 3 X 3 median filter will maintain a noise free step edge, but it will smear a noisy step
edge. Assuming the edge is contaminated by noise, the median is the fifth least bright pixel of the six
pixels on the left side of the edge as shown in the upper portion of the figure. As the window moves to the
right by one pixel (refer to the lower portion of the figure), the median is the fifth brightest pixel of the six
in the shaded area.
directional subregion average. The other three algorithms will not sharpen a ramp
edge but all will maintain a ramp edge fairly well. A derivative of the sigma filter
which will sharpen a ramp edge will be discussed in the next section.
(5) Removing Spot Noise
The median filter is well known for its effectiveness in removing sparsely posi-
tioned sharp spot noise, since the spot noise has intensity at either end of the
intensity scale. Nagao’s filter is also effective, but requires a few iterations. The
gradient inverse filter weights the spot noise much higher than its surrounding pixels.
Consequently, it is not effective. The sigma filter with large window size is highly
suseptible to spot noise, since no other pixel but the spot noise itself is within the
two sigma range. The modified version with threshold K (as shown in Fig. 3(G))
discussed in the last section will remove most isolated spot noise. However, spot
noise near edges remains because the 7 X 7 mask contains several edge pixels which
will fall into the two-sigma range. Increasing the value of K will further reduce the
spot noise, but at the expense of blurring edges and subtle details. The spot noise
can be further reduced by applying a 3 X 3 sigma filter with K = 1, or 2. Figure 7
shows the effect of spot noise reduction by applying it to Figs. 3(G) and 4(C) for
K = 1 and K = 2. Figures 3(G) and 4(C) are repeated in Fig. 7 for comparison. As
shown in Fig. 7(C), the spot noise is almost completely removed; however, the
one-pixel-wide bar is badly broken up. Figures 6(E) and (F) show the effect on the
aerial image.
(6) Computational Eficiency
In our comparison, the algorithms were coded in FORTRAN and no special
efforts were devoted to accelerate their executions. The computations were carried
266 JONG-SEN LEE
FIG. 7. The sigma filter is susceptible to spot noise as indicated in (A) and (D). The problem is
overcome by applying the 3 X 3 sigma filter with K = 1 and 2 as shown in (B) and (C) for the bar pattern,
and (E) and (F) for the aerial scene. (A) and (D) repeat Figs. 2(G) and 3(C), respectively.
SIGMA FILTER IMAGE SMOOTHING 267
out on a Data General NOVA 800 with a Comtal 8000 image display. The ratio of
computational time required for images of size 128 X 128 for each filter (per
iteration) is listed in increasing order as follows:
(1) The sigma filter (7 X 7), 1 unit of time;
(2) The median filter (3 X 3) 1.5 units of time;
(3) The gradient inverse filter, 4.0 units of time;
(4) Nagao’s filter, 11.0 units of time.
The sigma filter is the fastest algorithm in this group even with a 7 X 7 window. In
our simulation, it took no more time than computing a straight 7 X 7 average.
Nagao’s filter is extremely slow, since it requires the computation of variance for
nine subregions.
4. THE EXTENDED SIGMA FILTER
The sigma filter can be easily extended to perform image enhancement, segmenta-
tion, smoothing of signal-dependent noise, and even 3-D images. Here, only a few of
these possibilities will be mentioned.
(I) The Biased Sigma Filter
This extended sigma filter will sharpen a ramp edge and also enhance the contrast
of subtle detail. The bias is introduced by separately averaging pixels in the upper
intensity range of (xi, i, xi, j + A) and in the lower intensity range of (xi, i, xi, j - A).
The absolute difference between the upper average and xi, j, and also the absolute
difference between the lower average and xij, are computed. The center pixel is
replaced by the average which has the smaller absolute difference. The function of
the biased sigma filter can be easily explained in a one-dimensional case. Figure 8(A)
shows the effect of a seven-pixel-wide biased sigma filter with A = 3. With one
application, the ramp edge becomes much sharper, and it will approach a step edge
as the number of applications increase. It should be cautioned that the intensity
range A should be chosen to be relatively large. As shown in Fig. 8(B) a ramp edge
may become a two step edge with A = 1. This algorithm is useful in sharpening
edges in preprocessing for image segmentation by gray level difference and also in
bringing out subtle details in a smoothed image. Figure l(E) shows the image of Fig.
l(D) processed by the biased sigma filter.
(2) Signal-Dependent Noise
Signal-dependent noise or speckles occur in coherent optical images as well as in
synthetic aperture radar images [13]. To deal with this noise, a reasonably effective
method based on local statistics was recently proposed by Lee [14]. In our experi-
ment the sigma filter modified for signal-dependent noise performs better in many
cases and requires much less computational time. The intensity range will not only
float up and down with xi, j but also shrink or grow with xi, j, since u is a function of
x,, j. A more detailed discussion will be given in a separate study [15].
(3) Extension to 3-D Images
It is straightforward to extend the sigma filter and its derivatives to 3-D image
smoothing. The two-dimensional window will be replaced by a three-dimensional
268 JONG-SEN LEE
011’ ’ ’ ” ’ ’ ’ ’ ’ ’ ’ ’ ’ ’
PIXEL POSITION
(A) ?PlXEL WINDOW,INTENSITY RANGE,A=3
PIXEL POSITION
(8) 7PlXEL WINOOW,INTENSITY RANGE,A=I
FIG. 8. A ramp edge as shown in (A) is sharpened by applying the biased sigma filter. The intensity
range in the biased filter should be chosen to be relatively large. As shown in (B), the ramp edge becomes
a two step edge with the intensity range reduced from 3 to 1.
cube. Pixels within the cube are processed by the same procedures established for the
two-dimensional case.
5. REMARKS
(a) Most local smoothing algorithms do not require prespecified parameters.
Clearly, this is a distinct advantage if the algorithm is to be effective for all image
categories. The sigma filter does require specification of the intensity range and the
size of the window. However, these parameters permit us to fine-tune the filter to a
specific image or class of images. Once the characteristics of the sigma filter with
respect to the parameters are understood, it is fairly easy to determine the ap-
propriate values. In addition, the computational efficiency of this algorithm permits
us to adjust the parameters interactively.
(b) The basic principle of the sigma filter can be incorporated into other algo-
rithms to modify the characteristics of these filters. For example, it could be included
in Nagao’s filter or Lee’s local statistics algorithm [9] to obtain the two-sigma
average in the directional subregion after it has been chosen by the procedures of
these algorithms.
6. CONCLUSION
A simple, effective, and computationally efficient noise smoothing algorithm has
been developed. Detailed comparisons with a few local smoothing algorithms are
made to substantiate the basic characteristics of this filter. The procedure and
strategy of utilizing this filter has been explored. Applications of this filter to image
segmentation and other problems are currently under investigation. It is hoped that
SIGMA FILTER IMAGE SMOOTHING 269
the sigma filter will be accepted as a basic digital image processing technique because
of its simplicity and effectiveness.
ACKNOWLEDGMENTS
The author thanks Dr. I. Jurkevich and Dr. A. F. Petty for many helpful
discussions.
REFERENCES
1. A. Rosenfeld and A. C. Kak, Digital Picture Processing, Academic Press, New York, 1976.
2. H. C. Andrews and B. R. Hunt, Digital Image Restoration, Prentice-Hall, Englewood Cliffs, N.J.,
1977.
3. A. K. Jam, A semicausal model for recursive filtering of two dimensional images, IEEE Trans.
Comput. C-26, 1977.
4. W. K. Pratt, Digital Image Processing, Wiley, New York, 1978.
5. M. Nagao and T. Matsuyama, Edge preserving smoothing, Computer Graphics Image Processing 9,
394-407, 1979.
6. D. Wang, A. Vagnucci, and C. Li, Image enhancement by gradient inverse weighted smoothing
scheme, Computer Graphics Image Processing 15, 167-181, 1981.
7. M. J. McDommll, Box-filtering techniques, Computer Graphics Image Processing 17, 65-70, 1981.
8. J. S. Lee, Digital image enhancement and noise filtering by use of local statistics, IEEE Trans. Pattern
Anal. Mach. Intell. PAMI-2, 1980.
9. J. S. Lee, Refined filtering of image noise using local statistics, Computer Graphics Image Processing
15, 380-389, 1981.
10. R. E. Graham, Snow removal-A noise-stripping process for picture signal, IRE Trans. Inf. Theor. 8,
129-1441966.
11. J. M. S. Prewitt, Object enhancement and extraction, in Picture Procession and Psychopictorics (B. S.
Lipkin and A. Rosenfeld, Eds.), Academic Press, New York, 1970.
12. A. Lev, S. Zucker, and A. Rosenfeld, Interactive enhancement of noisy images, IEEE Trans. Syst.
Man Cybern. SMC-7,435-442, 1977.
13. J. W. Goodman, Some fundamental properties of speckles, J. Opt. Sot. Amer. 66, 1976.
14. J. S. Lee, Speckle analysis and smoothing of synthetic aperture radar image, Computer Graphics Image
Processing 17, 24-32, 1981.
15. J. S. Lee, A simple speckle smoothing algorithm for synthetic aperture radar images, IEEE Trans.
Sysf. Man Cybern., vol. SMC-13, No. 1, 85-89, 1983.