ISSN 2322-0929

Vol.03, Issue.05,
July-2015,
Pages:0683-0688

www.ijvdcs.org

A Novel Approach for Patch-Based Image Denoising Based on Optimized

Pixel-Wise Weighting
M. VENU1, P. NAGARAJU2, B. NAGARJUN SINGH3
1

Assoc Prof, Dept of ECE, Sarada Institute Technology and Science, TS, India, E-mail: Venumangapudi1986@yahoo.com.
PG Scholar, Dept of ECE, Sarada Institute Technology and Science, TS, India, E-mail: Nagaraju.parikapalli5@gmail.com.
3
Assoc Prof & HOD, Dept of ECE, Sarada Institute Technology and Science, TS, India, E-mail: nagarjun.singh24@gmail.com.
2

Abstract: In this paper, we propose a de-noising method motivated by our previous analysis of the performance bounds for
image de-noising. Insights from that study are used here to derive a high-performance practical de-noising algorithm. We
propose a patch-based Wiener filter that exploits patch redundancy for image de-noising. Our framework uses both
geometrically and photo metrically similar patches to estimate the different filter parameters. We describe how these parameters
can be accurately estimated directly from the input noisy image. Our de-noising approach, designed for near-optimal
performance (in the mean-squared error sense), has a sound statistical foundation that is analyzed in detail. The performance of
our approach is experimentally verified on a variety of images and noise levels. The results presented here demonstrate that our
proposed method is on par or exceeding the current state of the art, both visually and quantitatively.
Keywords: EPLL, Image Denoising, K-SVD.
I. INTRODUCTION
In the recent years, images and videos have become
integral parts of our lives. Applications now range from
casual documentation of events and visual communication, to
the more serious surveillance and medical fields. This has led
to an ever-increasing demand for accurate and visually
pleasing images. However, images captured by modern
cameras are invariably corrupted by noise. With increasing
pixel resolution, but more or less the same aperture size,
noise suppression has become more relevant. While advances
in optics and hardware try to mitigate such undesirable
effects, software-based de-noising approaches are more
popular as they are usually device independent and widely
applicable. In the last decade, many such methods have been
proposed leading to considerable improvement in de-noising
performance. In, we studied the problem from an estimation
theory perspective to quantify the fundamental limits of denoising. The insights gained from that study are applied to
develop a theoretically sound de-noising method in this
paper. The challenge of any image de-noising algorithm is to
suppress noise while producing sharp images without loss of
finer details. The first modern adaptive method to
successfully address these contradictory goals can be
attributed to where the authors proposed a generalization of
the SUSAN filter, which itself was an extension of the
Yaroslavky filter.
The authors there proposed de-noising by weighted
averaging using pixels similar in intensity within a local
neighborhood. Under strong noise, identifying such similar
pixels can be challenging. In, Takeda et al. proposed a signal

dependent steering kernel regression (SKR) framework for

de-noising. This method proved to be much more robust
under strong noise. A patch-based generalization of the
bilateral filter was proposed in where the concept of locality
was extended to the entire image. Although the results there
were encouraging, the true potential for this Non-Local
means (NLM) method was only realized. Another patch
redundancy-based framework, BM3D, adopts a hybrid
approach of grouping similar patches and performing
collaborative filtering in some transform (say, DCT) domain.
It ranks among the best performing methods that define the
current state-of-the-art. A significantly different approach to
de-noising was introduced in K-SVD. Building on the notion
of image patches being sparse-represent able, Elad et al.
proposed a greedy approach for dictionary learning tuned for
de-noising. In, we proposed a hybrid approach (K-LLD) that
bridged such dictionary-based approaches with the
regression-based frameworks. The motivation there was that
similar patches shared similar sub-dictionaries, and such subdictionaries could be used for better image modeling. A
similar observation was exploited in the form of a non-local
sparse model (NLSM)] to improve performance of the KSVD framework.
The dictionary-based methods provide implicit modeling
for natural images. More explicit models have also been used
for de-noising. In, Joshi et al. address the problem of denoising color images by explicitly modeling each pixel as a
combination of two colors, the basis colors themselves being
estimated from a local neighborhood. Denoising is achieved
by enforcing such a model the use of local principal

Copyright @ 2015 IJVDCS. All rights reserved.

M. VENU, P. NAGARAJU, B. NAGARJUN SINGH

components as bases for de-noising was proposed by later
filter is an approximation of the optimal filter (in the MSE
refined this idea with a local pixel grouping mechanism such
sense) obtained if one ignores the geometric structure of
that the principal component bases are estimated from only
image patches. Although extensively used for de-noising, the
similar patches within a neighborhood. Another model-based
Wiener filter is usually used in conjunction with some
approach using Markov random fields (MRF) as a Field-oftransform basis. For example, the collaborative Wiener filter
Experts (FoE) was applied to de-noising natural images in,
used in BM3D works in the DCT domain where an estimate
where the parameters for the model are learned from example
of the ground-truth (signal-to-noise ratio) is obtained through
images proposed a method where, in addition to a locally
an initial filtering of the image. Our spatial domain method is
affine signal model, the noise level is also estimated from the
motivated by our analysis of the image de-noising bounds.
input image, leading to a practical de-noising method.
In our framework, graphically illustrated in Fig. 1, we
develop a locally optimal Wiener filter where the parameters
are learned from both geometrically and photometrically
similar patches. For this, the noisy image is first segmented
into regions of similar geometric structure, as shown in Fig.
2. The mean and covariance of the patches within each
cluster are then estimated. Next, for each patch, we identify
photometrically similar patches and compute weights based
on their similarity to the reference patch. These parameters
are then used to perform de-noising patch-wise. To reduce
artifacts, image patches are selected to have some degree of
overlap (shared pixels) with their neighbors. A final
aggregation step is then used to optimally fuse the multiple
estimates for pixels lying on the patch overlaps to form the
de-noised image.
Fig.1.Outline of our proposed patch-based locally optimal
Wiener (PLOW) filtering method.
While most of the above-mentioned approaches work in
the spatial domain, a vast section of image de-noising
literature is devoted to transform domain methods. The main
motivation in such methods is that in the transform (e.g.,
DCT, wavelets, etc.) domain, it is possible to separate image
and noise components, and de-noising can be performed
through shrinkage of the transform coefficients. In, Chang et
al. showed that using a spatially adaptive threshold parameter
along with the over-complete wavelet basis, de-noising
performance can be considerably improved. Another
wavelet-domain method was considered the state-of-the-art,
until recently. There the authors perform de-noising by
modeling the wavelet coefficients of images as mixtures of
Gaussians. Enforcing such a model for noisy images leads to
considerable de-noising. In, an additional global model for
natural images in the form of homogeneous Gaussian
Markov random fields was used to improve performance
considerably. In the proposed a de-noising method aimed at
reducing the estimated mean squared error (MSE) through
wavelet thresholding.
In this paper we propose a new de-noising filter
motivated by our statistical analysis of the performance
bounds for patch-based methods. The contributions of our
work are as follows: We design a patch-based statisticallymotivated redundancy exploiting Wiener filter, where the
parameters of the method are learned from both
geometrically and photometrically similar patches. As will be
clear from our discussions in the next section, our method is
formulated to approach the performance bounds for patchbased de-noising. As a side note, we also show that the NLM

Fig.2. Clustering of a simple image based on geometric

similarity Note how pixels in any particular cluster can
have quite different intensities but similar geometric
structure (edge, corner, flat regions, etc.)
II. PATCH-BASED LOCALLY OPTIMAL WIENER
FILTER (PLOW)
A. Motivation
In we analyzed the performance bounds for the problem
of image de-noising. This was done from an estimation
theory point of view, where we seek to estimate the pixel
intensity zi at each location i from its noisy observation
(1)
Here, i is assumed to be independent and identically
distributed (iid), and M is the total number of pixels in the
image. In our study, we specifically considered patch-based
methods, where the observation model can be posed as
(2)
with yi Rn representing the vectorized
patch
centered at i. Using a Bayesian Cramer-Rao bound analysis,
we showed that the MSE of de-noising (estimating) any
given patch in the image is bounded from below by

International Journal of VLSI System Design and Communication Systems

Volume.03, IssueNo.05, July-2015, Pages: 0683-0688

A Novel Approach for Patch-Based Image Denoising Based on Optimized Pixel-Wise Weighting
observed noisy image. The procedure is algorithmically
(3)
represented in Algorithm. 1. We first identify geometrically
where
Rn is the estimate of zi, Ji Rnn is the Fisher
similar patches within the noisy image. Once such patches
information matrix (FIM), Cz Rnn is the patch covariance
are identified, we can use these patches to estimate the
matrix, and ||.|| denotes the l2 norm. This covariance matrix
moments (sz and Cz) of the cluster, taking care to account for
captures the complexity of the patches, and is estimated from
noise (steps 8 & 9 of Algorithm. 1). Next, we identify the
all the geometrically similar patches present in the given
photometrically similar patches and calculate the weights wij
image. Fig. 2 illustrates what we mean by geometric
that control the amount of influence that any given patch
similarity, where it can be seen that each cluster groups
exerts on de-noising patches similar to it. These parameters
together patches containing flat regions, edges in the
are then used to de-noise each patch. Since we use
horizontal or vertical directions, and corners of the simulated
overlapping patches, multiple estimates are obtained for
box image. Note that such grouping is done irrespective of
pixels lying in the overlapping regions. These multiple
the actual patch intensities. This is justified for intensityestimates are then optimally aggregated to obtain the final deindependent noise when de-noising performance is dictated
noised image. Below, we describe each step in greater detail.
by complexity of patches, rather than their actual intensities.
Algorithm 1:
The FIM, on the other hand, is influenced by the noise
characteristics. When additive white Gaussian noise (WGN)
is considered, the FIM takes the form
(4)
where I is the nn identity matrix, is the noise standard
deviation, and Ni is the patch redundancy measured as the
number of patches zj within the latent image that are
photometrically similar to a given patch zi. We define such
similarity as
(5)
In, is chosen as a small threshold dependent on the
number of pixels (n) in each patch. The relationship between
similar patches shown in Eq. 5 is based on the underlying
noise-free patches that are not known in practice. In this
expression was extended to define photometric similarity
between the corresponding noisy patches yj as
(6)
The Ni values can then be estimated directly from the
noisy image as the number of yj patches (including yi) that
satisfy the above criterion. Note that the condition for
photometric similarity, as defined here, is stricter than that
for geometric similarity. As such, photometric similarity can
be expected to imply geometrically similar as well. The
bounds expression (3) thus takes into account the complexity
of the image patches present in the image as well as the
redundancy level and the noise variance corrupting the
image. In, the bound was shown to characterize the
performance of the optimal affine-biased de-noising method.
In particular, for WGN, the right-hand side of (3) is the
performance achieved by the optimal linear minimum mean
squared-error (LMMSE) estimator, with Ji and Cz being the
parameters of the estimator. The Wiener filter is, in fact, the
LMMSE estimator that achieves this lower bound. Thus, a
patch-based Wiener filter, where the parameters are
estimated accurately, can lead to near-optimal de-noising.
This forms the basis of our approach. We outline the theory
behind the proposed approach next.
III. PARAMETER ESTIMATION FOR DENOISING
Our proposed de-noising framework, graphically outlined
in Fig. 1, requires us to infer various parameters from the

A. Geometric Clustering
In Sec. II, our proposed filter was derived assuming
geometrically similar patches to be sampled from some
unknown pdf. So far we have assumed such clustering to be
available to us from an oracle. To perform practical
clustering, we need to identify features that capture the
underlying geometric structure of each patch from its noisy
observations. Such features need to be robust to the presence
of noise, as well as to differences in contrast and intensity
among patches exhibiting similar structural characteristics.
An example of such variations among geometrically similar
patches can be seen in Fig.2. Possible choices of features
include contrast adjusted image patches or principal

International Journal of VLSI System Design and Communication Systems

Volume.03, IssueNo.05, July-2015, Pages: 0683-0688

M. VENU, P. NAGARAJU, B. NAGARJUN SINGH

components in conjunction with pre-determined clustering
We point out that in deriving the bounds (3), and our PLOW
guides. For our purposes, where the image patches can be
filter, we assume that the underlying noise-free patches zi are
considerably noisy, we make use of the Locally Adaptive
independent samples of a random variable z. In practice,
Regression Kernels (LARK) introduced for de-noising and
when working with overlapping patches, this assumption is
subsequently adapted as features for geometric clustering and
not strictly accurate. As with all other patch-based methods,
object detection. We refer the interested reader to where the
since we estimate each patch independently without
designs of the kernels are covered in detail.
explicitly taking into account information from estimates of
other overlapping patches, the estimation framework is in
line with the assumption of independence of underlying
(noise-free) patches. However, in estimating the covariance
matrix (in PLOW and also for the bounds we do not enforce
independence on the patches and the covariance matrix
estimated from overlapping patches is not necessarily
diagonal. Therefore, both in our bounds and our current
work, the correlation among the underlying noise-free
patches are implicitly taken into account.

Fig.3. Clustering based on geometric similarity of patches

illustrated on noise-free and noisy Barbara image of
standard deviation 15. Note how clustering in the noisy
image largely corresponds to that from the noise-free
image.
Using the LARK features, we run K-Means to cluster the
noisy image into regions containing geometrically similar
patches. In Fig 3, we illustrate the robustness of clustering
using LARK features, even in the presence of noise of
standard deviation 15. Note that the 5 clusters from the noisy
Barbara image are largely in keeping with those obtained
from the noise-free image. As has been noted in, the number
of clusters chosen affects the de-noising result. In general,
too few clusters can lead to structurally dissimilar patches
being clustered together resulting in incorrect estimation of
the moments. On the other hand, too many clusters lead to
too few patches within each cluster, making the moment
estimation process less robust. Fortunately, the denoised
output is not too sensitive to the choice of the number of
clusters (K). In our experiments we found that using a fixed
value of K=15 yields good results for any given image, with
MSE fairly close to that obtained by tuning the number of
clusters for that particular image.
B. Estimating Cluster Moments
Once the image is segmented into structurally similar
regions, we estimate the moments, namely mean and
covariance, from the noisy member patches of each cluster.
Since the i noise patches are assumed to be zero mean iid,
the mean of the underlying noise-free image can be
approximated by the expectation of the noisy patches within
each cluster as
(7)
where k denotes the k-th cluster with cardinality Mk. Note
that the stability of this estimate is dependent on M k. If too
few patches are present, the mean vector will remain noisy.

The issue is more subtle when considering the overlapping

noise patches. In that case, our assumption of i patches
being independent is a simplification used mainly for ease of
mathematical derivation. By now this is a standard practice
employed in all patch-based methods (NLM, BM3D, NLSM,
etc.) where information shared among overlapping noisy
patches are not exploited. We hazard a guess that one can
expect some modest improvement in performance if such
information are accounted for in the de-noising process.
Doing so is, however, non-trivial, and may be a good
direction for future research. The parameters estimated from
each cluster of the image can be directly used for de-noising
using however, as mentioned earlier; performance can be
improved by exploiting the self-similarities within an image.
Using photometrically similar patches can then contribute to
better de-noising of the reference patch. We describe this
process next.
IV. NUMERICAL EXPERIMENTS
In this section, we test the proposed weighting approach
under three representative de-noising algorithms: K-SVD,
EPLL, and BM3D. Due to space limitation, we list the
experimental results done on frequently used standard images
in image de-noising domain here, and list the other
supportive results on our webpage. For the K-SVD
algorithm, we find the de-noising performance can be
improved most significantly under high noise levels using the
practical profile with negative *s. For each noise level, *
is pre-learned from a training set with three standard images.
Taking = 50 as an example, as shown in Fig. 4, we compute
the PSNR gain of using w()for R, and find that * = 0.25 can lead to almost the maximum PSNR gain for any of
the training images. Therefore, we set * =-0.25 under =50
to de-noise all the images. After the training process, we
apply the practical profile to the other 8 standard images
under from 30 to 50, and compare the PSNR with the original
K-SVD algorithm. As shown in Table I, the averaged PSNR
gain increases from about 0.1 dB to 0.2 dB as increases.
For the EPLL algorithm, we apply the proposed weighting
approach for multiple times since it is an iterative algorithm.

International Journal of VLSI System Design and Communication Systems

Volume.03, IssueNo.05, July-2015, Pages: 0683-0688

A Novel Approach for Patch-Based Image Denoising Based on Optimized Pixel-Wise Weighting
We find both of the two profiles are effective for moderate
noise levels, while the approximation profile performs even
better. Therefore, we choose the approximation profile to
improve the EPLL algorithm. As shown in Table II, for noise
level from = 5 to = 20, the averaged PSNR gain can reach
around 0.1dB. The proposed weighting approach is not
effective for high noise levels probably because: It is
designed to minimize the MSE under only one F&W process
when it is used for multiple times, minimizing the MSE
within each iteration may not be the optimal. As the noise
level increases, the number of iteration also increases, which
enlarges the impact of the misleading objective. For the
BM3D algorithm, we find the PSNR improvement by using
the proposed weighting approach is insignificant, no matter
which profile is used. This is probably because BM3D has
much more estimates for the same pixel compare to K-SVD
and EPLL, and their correlation is also more complicated,
Fig.4. The PSNR gain by using w () under = 50 for Kwhich makes approximating the hidden covariance matrix
SVD. Each curve represents one training image and the
Covi accurately very hard. Therefore, we need to design
circle indicates the position of the optimal that leads to
more sophisticated profiles for BM3D in the future.
the maximum gain for that image.
TABLE I: PSNR Comparison Under K-SVD Under Each
V. CONCLUSION
Noise Level, The Left Column Uses The Original Weight,
In this paper, we have proposed a method of de-noising
The Right Column Uses Weight Of The Practical Profile
motivated from our previous work in analyzing the
performance bounds of patch-based de-noising methods. We
have developed a locally optimal Wiener-filter-based method
and have extended it to take advantage of patch redundancy
to improve the de-noising performance. Our de-noising
approach does not require parameter tuning and is practical,
with the added benefit of a clean statistical motivation and
analytical formulation. We analyzed the framework in depth
to show its relation to nonlocal means and residual filtering
methods. Through experimental validation, we have shown
that our method produces results quite comparable with the
state of the art. While mainly developed for grayscale
images, with trivial modification, our method achieves near
state-of-the-art performance in de-noising color images as
well. The de-noising performance can be expected to
TABLE II: PSNR Comparison under Epll Under Each
improve further by taking into account the correlation across
Noise Level, The Left Column Uses The Original Weight,
color components. Since the method works by learning the
The Right Column Uses Weight Of The Approximation
moments in geometrically similar patches, the inter channel
Profile
color dependences can be implicitly captured in this
framework. In a more practical setting where signaldependent noise is observed, the clustering step needs to take
into account color (or intensity) information as well. The
noise in each cluster can be then assumed to be
homogeneous, and the proposed filter can be independently
applied in each cluster.
VI. REFERENCES
[1] Jianzhou Feng, Li Song, Xiaoming Huo, Xiaokang Yang,
and Wenjun Zhang, An Optimized Pixel-Wise Weighting
Approach for Patch-Based Image Denoising, IEEE Signal
Processing Letters, Vol. 22, No. 1, January 2015.
[2] M. Elad and M. Aharon, Image de-noising via sparse
and redundant representations over learned dictionaries,

International Journal of VLSI System Design and Communication Systems

Volume.03, IssueNo.05, July-2015, Pages: 0683-0688

M. VENU, P. NAGARAJU, B. NAGARJUN SINGH

IEEE Trans. Image Process., vol. 15, no. 12, pp. 37363745,
Dec. 2006.
[3] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian,
Image de-noising
by sparse 3d transform-domain
collaborative filtering, IEEE Trans. Image Process., vol. 16,
no. 8, pp. 20802095, Aug. 2007.
[4] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman,
Non-local sparse models for image restoration, in Proc.
IEEE Int. Conf. Computer Vision, 2009, pp. 22722279.
[5] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian,
Bm3d image de-noising with shape-adaptive principal
component analysis, in Proc. Workshop on Signal
Processing with Adaptive Sparse Structured Representations
(SPARS09, 2009), 2009.
[6] G. Yu, G. Sapiro, and S. Mallat, Image modeling and
enhancement via structured sparse model selection, in Proc.
IEEE Int. Conf. Image Process, Sep. 2010, pp. 16411644.
[7] W. Dong, X. Li, L.Zhang, and G. Shi, Sparsity-based
image de-noising via dictionary learning and structural
clustering, in Proc. IEEE Int. Conf. Computer Vision and
Pattern Recognition, Jun. 2011.
[8] D. Zoran and Y. Weiss, From learning models of natural
image patches to whole image restoration, in Proc. IEEE Int.
Conf. Computer Vision, 2011, pp. 479486.
[9] J. Feng, L. Song, X. Huo, X. Yang, and W. Zhang,
Image restoration via efficient Gaussian mixture model
learning, in Proc. IEEE Int. Conf. Image Process, Sep. 2013.
[10] O. G. Guleryuz, Weighted averaging for de-noising
with over complete dictionaries, IEEE Trans. Image
Process., vol. 16, no. 12, pp. 30203034, 2007.
[11] A. Buades, B. Coll, and J.Morel, A non-local algorithm
for image de-noising, in Proc. IEEE Int. Conf. Computer
Vision and Pattern Recognition, Jun. 2005.
Authors Profile:
M. VENU, Assoc Prof, Dept of ECE,
Sarada Institute Technology and Science,
TS, India, E-mail:
Venumangapudi1986@yahoo.com.

P. NAGARAJU, PG Scholar, Dept of ECE,

Sarada Institute Technology and Science,
TS, India, E-mail:
Nagaraju.parikapalli5@gmail.com.

B.NAGARJUN SINGH, Assoc Prof & HOD,

Dept of ECE, Sarada Institute Technology
and Science, TS, India, E-mail:
nagarjun.singh24@gmail.com.

International Journal of VLSI System Design and Communication Systems

Volume.03, IssueNo.05, July-2015, Pages: 0683-0688