第1页 / 共18页
第2页 / 共18页
第3页 / 共18页
第4页 / 共18页
第5页 / 共18页
第6页 / 共18页
第7页 / 共18页
第8页 / 共18页
Digital Image Restoration.pdf
gital Image Restoration
MARK R. BANHAM AND AGGELOS I<. KATSAGGELOS
he field of image restoration began primarily
with the efforts of scientists involved in the
space programs of both the United States
and the former Soviet Union in the 1950s and early
1960s. These programs were responsible for pro-
ducing many incredible images of the Earth and our
solar system that, at that time, were unimaginable.
Such images held untold scientific benefits which
only became clear in the ensuing years as the race
for the moon began to consume more and more of
our scientific efforts and budgets. However, the
images obtained from the various planetary mis-
sions of the time, such as the Ranger, Lunar Orbiter,
and Mariner missions, were subject to many photo-
graphic degradations. These were a result of sub-
standard imaging environments, the vibration in
machinery and the spinning and tumbling of the
spacecraft. Pictures from the later manned space
missions were also blurred due to the inability of the
astronaut to steady himself in a gravitationless en-
vironment while taking photographs. The degrada-
tion of images was no small problem, considering
the enormous expense required to obtain such pic-
tures in the first place. The loss of information due
to image degradation could be devastating. For
example, the 22 pictures produced during the Mari-
ner IV flight to Mars in 1964 were later estimated
to cost almost $10 million just in terms of the
number of bits transmitted alone [83]. Any degra-
dations reduced the scientific value of these images
considerably and clearly cost the space agencies money.
This was probably the first instance in the engineering
community where the extreme need for the ability to retrieve
meaningful information from degraded images was encoun-
tered. As a result, it was not long before some of the most
common algorithms from one-dimensional signal processing
and estimation theory found their way into the realm of what
is today known as “digital image restoration.”
The goal of this article is to introduce digital image resto-
ration to the reader who is just beginning in this field, and to
provide a review and analysis for the reader who may already
be well-versed in image restoration. The perspective on the
topic offered here is one that comes primarily from work done
in the field of signal processing. Thus, many of the techniques
and works cited here relate to classical signal processing
approaches to estimation theory, filtering, and numerical
analysis. In particular, the emphasis here is placed primarily
on digital image restoration algorithms that grow out of an
area known as “regularized least squares” methods. It should
be noted, however, that digital image restoration is a very
broad field, as we will discuss, and thus contains many other
successful approaches that have been developed from differ-
ent perspectives, such as optics, astronomy, and medical
imaging, just to name a few.
In the process of reviewing this topic, we hope to address
a number of very important issues in this field that are not
typically discussed in the technical literature. The nature of
these issues may be accurately summed up in these open
questions to the image restoration research community:
“Where have we been?”, “Where are we now?”, and “Where
are we going?” Although these may seem questions too large
to tackle in this forum, they are ones that warrant discussion
now because of the relative maturity of the image restoration
field. One indicator of this maturity is that reported improve-
ments over tried-and-true algorithms in recent years might be
24
IEEE SIGNAL PROCESSING MAGAZINE
1053-588S/97/$10.0001997IEEE
MARCH 1997
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
considered quite small. Because of this, we would be well
served now to take a step back and try to understand the
contributions of the past and the needs of the future in order
to best take advantage of the wealth of experience and knowl-
edge in the area of digital image restoration.
Applications of Digital Image Restoration
The first encounters with digital image restoration in the
engineering community were in the area of astronomical
imaging, as previously mentioned. Ground-based imaging
systems were subject to blurring due to the rapidly changing
index of refraction of the atmosphere. Extraterrestrial obser-
vaiions of the Earth and the planets were degraded by motion
blur as a result of slow camera shutter speeds relative to rapid
spacecraft motion. Images obtained were often subject to
noise of one form or another. For example, the astronomical
imaging degradation problem is often characterized by Pois-
son noise, which is signal-dependent and has its roots in the
photon-counting statistics involved with low light sources.
Another type of noise found in other digital imaging applica-
tions is Gaussian noise, which often arises from the electronic
components in the imaging system and broadcast transmis-
sion effects.
Not surprisingly, astronomical imaging is still one of the
primary applications of digital image restoration today. Not
only is it still necessary to restore various pictures obtained
from spacecraft such as the space shuttle, but the well-publi-
cized problems with the initial Hubble Space Telescope
(HST) main mirror imperfections [87, 1251 have provided an
inordinate amount of material for the restoration community
over the last few years. For example, Fig. 1 shows an HST
Wide Field Planetary Camera
image of Saturn.
urn, using the algorithm in [42].
poisson distributed film-
grain noise in chest X-rays,
mammograms, and digital
angiographic images [12,32,
1131, and for the removal of
additive noise in Magnetic
Resonance Imaging (MRI)
[13, 88, 1141. Another
emerging application of im-
age restoration in medicine
is in the area of quantitative
autoradiography (QAR). In
this field, images are ob-
tained by exposing X-ray-
sensitive film to a radioactive specimen. QAR is performed
on post-mortem studies, and provides a higher resolution than
techniques such as positron emission tomography (PET),
X-ray computed tomography (CAT), and MRI, but still needs
to be improved in resolution in order to study drug diffusion
and cellular uptake in the brain. This can be accomplished
through digital image restoration techniques [30]. Figure 3
shows a medical example of digital image restoration applied
to an autoradiographic image of Cr-5 1 microspheres that are
10 microns in diameter. Figure 3(a) is the original image, and
Fig. 3(b) is the restored image. The plot in Fig. 3(c) shows a
line profile through the images demonstrating the improve-
ment obtained through restoration. Here, an iterative restora-
MARCH 1997
IEEE SIGNAL PROCESSING MAGAZINE
25
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
tion algorithm that was formulated to consider the signal-de-
pendent nature of film grain noise was used [30]. The full-
width half maximum (FWHM) resolution of the
microspheres was improved by 60% from about 259 microns
to 103 microns.
Image restoration has also received some notoriety in the
media, and particularly in the movies of the last decade. Ten
years ago, the climax of the 1987 fi1m“No Way Out,” starring
Kevin Costner, was based on the digital restoration of a blurry
Polaroid negative image. The 199 1 movie “JFK’ made sub-
stantial use of a version of the famous Zapruder 8mm film of
the assassination of President Kennedy, which has been
enhanced and restored many times over the years. Similar
restoration ideas showed up in the Michael Crichton book and
subsequent 1993 film “Rising Sun,” where researchers were
needed to help restore the shadowy picture of a murderer from
a surveillance videotape. Although some of these fictional
uses of restoration were far-fetched, it is no surprise that
digital image restoration has been used in law enforcement
and forensic science for a number of years. For example, one
of the most frequent needs for image restoration arises when
viewing poor-quality security videotapes. In addition, the
restoration of blurry photographs of license plates and crime
scenes are often needed when such photographs can provide
the only link for solving a crime. Such use of restoration is
becoming more and more prevalent is our society. In fact,
images restored in our laboratory were recently presented and
accepted into evidence in a court of law for the first time by
Dr. W. R. Oliver of the Office of the Armed Forces Medical
Examiner 1891. Clearly, law enforcement agencies all over
the world have made, and continue to make use of digital
image restoration ideas in many forms.
Another application of this field which is especially im-
portant to our popular culture is the use of digital techniques
to restore aging and deteriorated films. The idea of motion
picture restoration is probably most often associated with the
digital techniques used not only to eliminate scratches and
dust from old movies, but also to colorize black-and-white
films. For the purposes of this article, only a small subset of
the vast amount of work being done in this area can be
classified under the category of image restoration. Much of
this work belongs to the field of computer graphics and
enhancement. Nonetheless, some very important work has
been done recently in the area of digital restoration of films.
Some of the most interesting has been accomplished on
animated films, such as the recent digital restoration of the
film “Snow White and the Seven Dwarfs” by Walt Disney,
which originally premiered in 1937 [22]. Though not restor-
ing for blur degradation, the process used to correct for the
cell dust, scratch and color fading problems with this original
film could be classified as a form of spatially adaptive image
restoration. There has been significant work in the area of
restoration of image sequences in general as well, as dis-
cussed in [9, lo].
Perhaps the most exciting and expanding area of applica-
tion for digital image restoration is that in the field of image
and video coding. As techniques are developed to improve
(4
(bl
4.(a) JPEG encoded image from sequence “Cauphone ” (28:lj;
(b) Restored image, using the algorithm in [91].
coding efficiency, and reduce the bit rates of coded images,
artifacts such as blocking become quite a problem. Blocking
artifacts are a result of the coarse quantization of transform
coefficients used in typical image and video compression
techniques. Usually, a discrete cosine transform (DCT) will
be applied to prediction errors on blocks of 8 x 8 pixels.
Intensity transitions between these blocks become more and
more apparent when the high-frequency data is eliminated
due to heavy quantization. Already, much has been accom-
plished to model these types of artifacts, and develop ways
of restoring coded images as a post-processing step to be
performed after decompression [70, 91, 102, 90, 129, 1301.
In particular, very low bit rate coding applications such as
mobile video communications impose bandwidth restrictions
that require high compression. An example showing a still
JPEG compressed image from a mobile video sequence at a
compression ratio of 28:l is shown in Fig. 4(a). Using a
process based on mean field annealing and Markov Random
Fields [91], a post-processed (restored) image is seen in Fig.
4(b). This image has most of the blocking artifacts removed,
while still maintaining the important edges around the face
in the picture. This idea of trading off smoothness and sharp-
ness of an image in a spatially adaptive way forms the basis
of regularization theory which is applied to the solution of
the ill-posed restoration problem [118].
Digital image restoration is being used in many other
applications as well. Just to name a few, restoration has been
used to restore blurry X-ray images of aircraft wings to
improve federal aviation inspection procedures [61]. It is
used for restoring the motion induced effects present in still
composite frames (produced by the superposition of two
temporally spaced fields of a video image [77]), and, more
generally, for restoring uniformly blurred tel
[7 11. Printing applications often require the U
to ensure that halftone reproductions of CO
are of high quality. In addition, restoration can improve the
quality of continuous images generated from halft
[34]. Digital restoration is also used to restore
electronic piece parts taken in assembly-line manufacturing
environments. Many defense-oriented applications require
restoration, such as that of guided missiles, which
distorted images due to the effects of pressure differences
around a camera mounted on the missile. All in all, it is clear
’ ‘
26
IEEE SIGNAL PROCESSING MAGAZINE
MARCH 1997
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
that there is a very real and important place for image resto-
ration technology today. Our task at hand now is to evaluate
what types of applications may arise in the future and demand
further innovation in this field. As a means of achieving this
task, however, it is best to first understand the accomplish-
ments of the past.
Where Have We Been?
technique, using the major categories: Direct, Iterative, and
Recursive.
Sources of Image Degradation
In digital image processing, the general, discrete model for a
linear degradation caused by blurring and additive noise can
be given by the following superposition summation,
A useful place to start is with a comprehensive definition of
what digital image restoration is, and what it is not. Given
such a definition, it will be easier to address the development
of the various signal processing algorithms used for restora-
tion, and to study how they affect the current trends in
research.
Digital image restoration is a field of engineering that
studies methods used to recover an original scene from de-
graded observations. It is an area that has been explored
extensively in the signal processing, astronomical, and optics
communities for some time. Many of the algorithms used in
this area have their roots in well-developed areas of mathe-
matics, such as estimation theory, the solution of ill-posed
inverse problems, linear algebra and numerical analysis.
Techniques used for image restoration are oriented toward
modeling the degradations, usually blur and noise, and apply-
ing an inverse procedure to obtain an approximation of the
original scene.
Image restoration is distinct from image enhancement
techniques, which are designed to manipulate an image in
order to produce results more pleasing to an observer, without
making use of any particular degradation models. Image
reconstruction techniques are also generally treated sepa-
rately from restoration techniques, since they operate on a set
of image projections and not on a full image. Restoration and
reconstruction techniques do share the same objective, how-
ever, which is that of recovering the original image, and they
end up solving the same mathematical problem, which is that
of finding a solution to a set of linear or nonlinear equations.
Some excellent treatment and review of different restoration
and recovery techniques from a signal processing perspective
can be found in these books and articles: [2, 8, 46, 48, 67,
1091. Much of the review material discussed here can be
found with further detail in these references.
Developing techniques to perform the image restoration
task requires the use of models not only for the degradations,
but also for the images themselves. It will be valuable to study
how some such models were used in the early applications
and solutions in this field. Here, we will concern ourselves
only with approaches based on digital techniques, although
there have been significant efforts to restore degraded images
through strictly optical and photographic means. There are a
number of different ways in which to classify the many
approaches to digital image restoration. One useful classifi-
cation based on Deterministic and Stochastic approaches was
given in Chapter 1 of [48]. In the second subsection below,
we classify the well-known approaches to regularized least-
squares restoration from the viewpoint of implementation
k=l 1=1
(1)
whereflij) represents an original M x N image, and y ( i j ) is
the degraded image which is acquired by the imaging system.
In this formulation, n(ij) represents an additive noise intro-
duced by the system, and is usually taken to be a zero mean
Gaussian distributed white noise term. In this article, we deal
only with additive Gaussian noise, as it effectively models
the noise in many different imaging scenarios. Many methods
not detailed in this article utilize signal-dependent noise and
lead to non-linear approaches to image restoration (see, for
example, [62]).
In Equation (I), h(ij;m,n) represents the two-dimensional
point spread function (PSF) of the imaging system, which, in
general, can be spatially varying. The difficulty in solving the
restoration problem with a spatially varying blur commonly
motivates the use of a stationary model for the blur. This leads
to the following expression for the degradation system,
k=l
[=I
= h(i, j ) * * f ( i , j ) + n(i, j )
where * * indicates two-dimensional convolution. The use of
linear techniques for solving the restoration problem is facili-
tated by using this shift-invariant model. Models that utilize
space-variant degradations are also common, but lead to
more complex solutions.
An important aspect of image processing that deserves
some mention here is that of the treatment of borders. The
blurring process described by Equation (2) is linear. How-
ever, we often approximate this linear convolution by circular
convolution, for mathematical reasons discussed later. This
involves treating the image as one period from a two-dimen-
sional periodic signal. The borders of an image are also often
treated as symmetric extensions of the image, or as repeated
instances of the edge pixel values. Such approaches seek to
minimize the distortion at the borders caused by filtering
algorithms which must perform deconvolution over the entire
image. When implementing image restoration algorithms, it
is very important to consider how the borders of the image
are treated, as different approaches can result in very different
restored images [2].
The following analytical models are frequently used in
Equation (2) to represent the shift-invariant image degrada-
MARCH 1997
IEEE SIGNAL PROCESSING MAGAZINE
27
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
tion operator [44, 671. The first two are encountered in the
application of astronomical imaging mentioned before.
Motion Blur: Represents the 1-D uniform local averaging
of neighboring pixels, a common result of camera panning
or fast object motion, shown here for horizontal motion,
0, otherwise.
2
(3)
0 Atmospheric Turbulence Blur: Common in remote sensing
and aerial imaging, the blur due to long-term exposure
though the atmosphere can be modeled by a Gaussian PSF,
(4)
where K is a normalizing constant ensuring that the blur is
of unit volume, and o2 is the variance that determines the
severity of the blur.
Photographic defocusing is a also problem in many differ-
ent imaging situations. This type of blurring is primarily due
to effects at the camera aperture that result in the spreading
of a point of incoming light across a circle of confusion. A
complete model of the camera’s focusing system depends on
many parameters. These parameters include the focal length,
the camera aperture size and shape, the distance between
object ahd camera, the wavelength of the incoming light, and
the effects due to diffraction [7,29]. Accurate knowledge of
all of these parameters is not frequently available after a
picture has been taken. When the blur due to poor focusing
is large, however, the following uniform models have been
used as approximations of the PSF.
Uniform Out-of-Focus Blur: This models the simple de-
focusing found in a variety of imaging systems as a uniform
intensity distribution within a circular disk,
(5)
Uniform 2-D Blur: This is a more severe form of degrada-
tion that approximates an out-of-focus blur, and is used in
many research simulations. This is the model for the blur
used in the examples throughout this article,
where L is assumed to be an odd integer.
Usually, all blur-degraded images exhibit similar charac-
teristics, namely a lowpass smoothing of the original image,
attenuating the edge information which is very important for
human visual perception [86]. In the process of trying to
invert Equation ( 1 ) to obtain an estimate of f(ij), different
artifacts may be introduced as a result of the characteristics
of each blur operator. This issue will be discussed later. First,
(4
(b)
5.(a) Original “Cameraman” image (256 x 256); (b) Degraded
by a 7x7 Uniform 2-0 Blur, 40 dB BSNR.
we will review some of the early or “classical” ways to
perform the required inversion. As a tool for demonstrating
these techniques, we can utilize an example of a synthetically
blurred image which is often used for comparing results in
the research literature. This image is referred to as the “Cam-
eraman” image, and is seen in Fig. 5(a). Figure 5(b) shows
the effects of a 7x7 uniform 2-D blur, at 40dB BSNR (Blurred
Signal-to-Noise Ratio).
Some Classical Image Restoration Techniques
In this section, we review a some of the many common
approaches to image restoration that utilize minimum mean
square error as an optimization criterion. The image degra-
dation process is often represented in terms of a matrix-vector
formulation of Equation (1). This is given by
y = H f + n ,
(9)
where y,f, and n are the observed, original, and noise images,
ordered lexicographically by stacking either the rows or the
columns of each image into a vector. Assuming that the original
image is of support N x N, then these vectors have support N2 x
1, and H represents the Nz x N2 superposition blur operator.
When utiiizing the stationary model of Equation (Z), H becomes
a block-Toeplitz matrix representing the linear convolution
operator h(ij). Toeplitz, and block-Toeplitz matrices have spe-
cial “banded” properties which make their use desirable for
representing linear shift-invariant operators (see [Z] for further
explanation of these matrices). By padding y andfappropriately
with zeros so that the results of linear and circular convolution
are the same, H becomes a block circulant matrix. This special
matrix smcture has the form
r H(0) H(N-1)
... H(1)1
1) H ( N - 2 ) ”
where each sub-matrix H(i) is itself a circulant matrix.
28
IEEE SIGNAL PROCESSING MAGAZINE
MARCH 1997
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
BSNR
In most image restoration studies, the degradation mod-
eled by blurring and additive noise is referred to in terms
of a metric called the Blurred Signal-to-Noise Ratio
(BSNR). This figure is defined in terms of the additive
noise variance, IS,, , according to
2
lead to efficient computation of inverse matrices, as discussed
in the next section.
Inverse Filtering. Classical direct approaches to solving
Equation (9) have dealt with finding an estimate 3 which
minimizes the norm
I
1
(7)
for an M x Nimage, where g(i,j) = y ( i j ) - n(ij) in Equation
(l), and g(m, a) = E{g}, which represents the expected
value, or the mean, of g.
For the purpose of objectively testing the performance
of image restoration algorithms, the Improvement in SNR
(ISNR) is often used. This nietric is given by
where,Rij) and y ( i j ) are the original and degraded inten-
sity components, respectively, and j(i, j ) is the corrc-
sponding restored intensity field. Obviously, this metriccan
only be used for simulation cases when the original image is
available. While mean squared error (MSE) metrics such as
ISNR do not always reflec? the perceptual properties of the
human visual system, they s m e lo provide an objective
standard by which to compare different techniques. How-
ever, in all cases presented here, it is important to consider
the behavior of the various algorithms from the viewpoint of
ringing and noise amplification, which can be a key indicator
of' improvement in quality for subjective comparisons of
restoration algorithms.
thus providing a least squares fit to the data. This leads
directly to the generalized inverse filter, which is given by
the solution to
( H ' H ) ~ = ~~y
The critical issue that arises in this approach is that of noise
amplification. This is due to the fact that the spectral proper-
ties of the noise are not taken into account. In order to
examine this, consider the case when H (and, therefore, HT)
is block circulant, as described above. Such matrices can be
diagonalized with the use of the 2-D Discrete Fourier Trans-
form (DFT) [36]. This is because the eigenvalues of a block
circulant matrix are the 2-D discrete Fourier coefficients of
the impulse response of the degradation system which is used
in uniquely defining H, and the eigenvectors are the complex
exponential basis functions of this transform. In matrix form,
this relationship can be expressed by
H = W N '
(14)
where His a diagonal matrix comprising the 2-D DFT coef-
ficients of h(ij), and W1 is a matrix containing the compo-
nents of the complex exponential basis functions of the 2-D
DFT. Pre-multiplication of both sides of Equation (14) by
W', and post-multiplication by W, or in this case, taking the
2-D DFT of the first row of H , with the elements stacked into
an N x N image, gives the diagonal elements of 31
Using this diagonalization approach, the matrix inverse
problem of Equation (13) can be solved as a set of N2 scalar
problems. That is, using the DFT properties of block circu-
lant matrices, and pre-multiplying both sides of Equation
(13) by W1, the solution can be written in the discrete
frequency domain as
Notice that each block-row of H and each row of H(i) is a circular
shift of the prior block-row or row, respectively. Representing
H with a block circulant matrix could be further justified by
using the result that the asymptotic distribution of the eigenval-
ues of a block Toeplitz and a block circulant matrix are the same
[3 I]. The use of the block circulant approximation is important
because it leads to desirable discrete frequency domain proper-
ties that can be used in solving Equation (9) [2]. These properties
where @), d(!), and Y(!) denote the DFT of the restored
image, j(i, j ) , the PSF, h(ij), and the observed image, y(ij),
, where
as a function of the 2-D discrete frequency index
I = (k,, k2) for k,, k2 = 0, . . . , N-1, for an N x N point DFT, and
* denotes complex conjugate. Clearly, for frequencies at
which Pi(!) becomes very small, division by it results in
amplification of the noise. Assuming that the degradation is
lowpass, the small values of H(!) are found at high frequen-
MARCH 1997
IEEE SIGNAL PROCESSING MAGAZINE
29
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
cies, where the noise is
dominant over the image.
For the frequencies where
%(I)
is exactly zero (with
respect to the accuracy of the
particular computing envi-
ronment being used) $(I) is
also equal to zero (which is
the minimum norm least
squares solution). Figure 6
shows the effects of a gener-
alized inverse filter applied
to the degraded image in Fig.
5(b). The restored image has been truncated to lie within the
range of 10-2551, however, the actual dynamic range of this
image is much larger due to amplified noise. Clearly, this is
not an acceptable restoration approach in this case.
6. Result of Fig. 5(b) restored by
a Generalized Inverse filter,
ISNR = -15.6 dB.
In mathematical terms, the inverse problem represented in
Equation (9) is ill-posed, if described in continuous infinite-
dimensional space [85, 1181. In this case, the observation
equation becomes a Fredholm integral equation of the first
kind. The ill-posed nature of this problem implies that small
bounded deviations in the data may lead to unbounded devia-
tions in the solution. With respect to the discretized problem
of Equation (9), the ill-posedness of the continuous problem
results in the matrix H being ill-conditioned. It is interesting
to note that the finer the discretization of the Droblem. the
more ill-conditioned H becomes. Regularization theory is
often used to solve ill-posed or ill-conditioned problems. The
purpose of regularization is to provide an analysis of an
ill-posed problem through the analysis of an associated well-
posed problem, whose solution will yield meaningful an-
swers and approximations to the ill-posed problem [48].
Techniques to accomplish this span a vast array of mathe-
matical and heuristic concepts. In this section, some of the
classical direct, iterative, and recursive approaches to this
problem are reviewed.
Direct Regularized Restoration Approaches. Solving
Equation (9) in a regularized fashion can lead to direct
restoration approaches when considering either a stochastic
or a deterministic model for the original image, f. In both
cases, the model represents prior information about the solu-
tion which can be used to make the problem well-posed.
Stochastic Regularization. Stochastic regularization can
lead to the choice of a linear filtering approach that computes
the estimate, f , according to
(17)
The matrix (HRfiT + R,,), which needs to be invert
better conditioned than the matrix (HTH) in Equ
Equation (17) is the classical formulation of the Wiener filter
PI.
By assuming block circulant structures for each of the
matrices in Equation (17), it can be rewritten and solved in
the discrete frequency domain. The assumption of Rfand R,,
being block circulant implies that the image and noise fields
are stationary. This results in a scalar computation for each
2-D frequency component ! , given by
where S,(!)
original image and the noise, respectively.
and SEE(!) represent the power spectra of the
Having these power spectra represents significant prior
knowledge for the implementation of this filter. In most
cases, however, the noise variance is known, or can be
estimated from a flat region of the observed image [2]. In
addition, it is possible to estimate Sf(!) in a number of
different ways. The most common of these is to use the power
spectrum of the observed image, Syy(!), as an estimate of
Sf(!). Fig. 7 shows an ex-
ample of a Wiener filter res-
toration of Fig. 5(b), using a
periodogram estimate of the
power spectrum computed
from y . The periodogram es-
timate of the power spectrum
is simply defined according
to [78]
-
1 v
a direct Wiener filter, ISNR =
3.9 dB.
(19)
Deterministic Regulari-
zation. The use of determi-
nistic prior information about the original image can also be
used for regularizing the restoration problem. For example,
constrained least squares (CLS) restoration can be formulated
by choosing an
to minimize the Lagrangian
{ E f f } , which is the covariance where the term c? generally represents a high pass filtered
subject to knowledge of R,=
matrix off, and R,, = E{ nnT} , which is the covariance matrix version of the image 3. This is essentially a smoothness
of the noise. Using a stochastic model for f and n requires
constraint which suggests that most images are relatively flat
some prior knowledge of the statistics of the data which are with limited high-frequency activity, and thus it is appropri-
ate to minimize the amount of high-pass energy in the re-
then used to regularize the problem. The linear estimate
which minimizes Equation (16) is given by
stored image. Use of the C operator provides an alternative
30
IEEE SIGNAL PROCESSING MAGAZINE
MARCH 1997
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
way to reduce the effects of the small singular values of H,
occurring at high frequencies, while leaving the larger ones
unchanged. One typical choice for C is the 2-0 Laplacian
operator [37], given by
0.00
-0.25
0.00
0.00
-0.25
0.00
In Equation (20), a represents the Lagrange multiplier, com-
monly referred to as the regularization parameter, which
controls the tradeoff between fidelity to the data (as expressed
and smoothness of the solution (as
expressed by IICjif).
The minimization in Equation (20) leads to an equation of
the form
j=(HTH+aCTC)-LHTy.
(22)
This also may be solved directly in the discrete frequency
domain when block-circulant assumptions are used. The
critical issue in the application of Equation (22) is the choice
of a. This problem has been investigated in a number of
studies (see, for example, [25], and the references therein)
and optimal techniques exist for finding an a given varying
amounts of prior information about the noise and the signal.
One way to use Equation (22), and choose a based on prior
knowledge, follows a set theoretic approach [49, 46, 511.
With this method, a restored image is defined by an image
which lies in the intersection of the two ellipsoids defined by
and
Qf= cfl llCfl12 5 I? } .
(23)
(24)
The equation of the center of one of the ellipsoids which
bounds the intersection of QJy and Qf is given by Equation
(22) with a = (E/@*. The same solution may be obtained with
the Miller regularization approach [ 8 11.
Precise knowledge of
both bounds c2 and E2 may
not always be available. Sev-
eral ways to estimate these
bounds iteratively, based on
the partially restored image
at each step of the iteration,
have been presented in [42,
41, 52, 431. If the noise and
signal variances are known
or can be estimated, one
choice is a = - [49,46,
511. Figure 8 shows an ex-
BSNR
1
8. Result of Fig. 5(b) restored by
a Constrained Least Squares fil-
ter, ISNR = 2.0 dB.
1
ample of a CLS restoration applied to Fig. 5(b), using
a=-
BSNR '
As an illustration of the behavior of the restored image in
relation to the regularization parameter, Fig. 9(a) shows
several direct CLS restorations at different choices for a.
Notice that with larger values of a, and thus more regulari-
zation, the restored image tends to have more ringing, and yet
with smaller values of a, the restored image tends to have
more amplified noise effects. This is seen in the correspond-
ing error images shown in Fig. 9(b). The optimal solution, in
the MSE sense, lies somewhere in the middle of the two
extremes.
A more objective presentation of this idea can be seen in
Fig. 10, where the variance, bias, and MSE of the direct CLS
restoration filter are plotted as a function of a. The variance
and bias have been computed here in the frequency domain,
as in [25], according to
Vur [j(a)] = o : g
i=l (l?fil2
1%12
+ alcJ2)
2
and
Bias( ?(a)) = 0:
1q2a21ccr
N 2
i=l (lHc? + a l C , ? r
'
(26)
where @, c,, and F, represent the 2-D discrete frequency
components of the blur, Laplacian constraint operator, and
original image, respectively, and 0: represents the variance
of the additive noise. The bias of this estimator is a monotoni-
cally increasing function of a, while the variance is a mono-
tonically decreasing function of a. Notice that the minimum
MSE is encountered close to the intersection of these two
curves, which is the point having equal bias and variance.
Thus, one measure of objectively defining a good regulariza-
tion parameter is to choose that a which gives the best
compromise between these two types of errors. The proper-
ties of the bias and variance will be discussed in more detail
in a later section.
While direct approaches solved in the frequency domain
are among the most simple ways to restore noisy-blurred
images, they are subject to a number of restrictions, most
importantly the assumption that the image is globally station-
ary, and that a fair amount of prior information exists.
Iterative Approaches. Iterative image restoration algo-
rithms have been investigated in some detail over the last
decades (see, for example, [44, 46, 1061, and the references
therein, and [8, 49, 50, 51, 671). The primary advantages of
iterative techniques are that there is no need to explicitly
implement the inverse of an operator and that the process may
be monitored as it progresses. In addition, the effects of noise
may be controlled with certain constraints, spatial adaptivity
may be introduced, and parameters determining the solution
MARCH 1997
IEEE SIGNAL PROCESSING MAGAZINE
31
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on January 13, 2010 at 08:14 from IEEE Xplore. Restrictions apply.
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
