Lecture Notes for
EE 261
The Fourier Transform and its Applications
Prof. Brad Osgood
Electrical Engineering Department
Stanford University
Contents
1 Fourier Series
1.1
Introduction and Choices to Make
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Periodic Phenomena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Periodicity: Definitions, Examples, and Things to Come . . . . . . . . . . . . . . . . . . . .
1.4
It All Adds Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.5 Lost at c . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6 Period, Frequencies, and Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7 Two Examples and a Warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.8 The Math, the Majesty, the End . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.9 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.10 Appendix: The Cauchy-Schwarz Inequality and its Consequences . . . . . . . . . . . . . . .
1.11 Appendix: More on the Complex Inner Product . . . . . . . . . . . . . . . . . . . . . . . . .
1.12 Appendix: Best L2 Approximation by Finite Fourier Series
. . . . . . . . . . . . . . . . . .
1.13 Fourier Series in Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.14 Notes on Convergence of Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.15 Appendix: Pointwise Convergence vs. Uniform Convergence . . . . . . . . . . . . . . . . . .
1.16 Appendix: Studying Partial Sums via the Dirichlet Kernel: The Buzz Is Back . . . . . . . .
1.17 Appendix: The Complex Exponentials Are a Basis for L2([0, 1]) . . . . . . . . . . . . . . . .
1.18 Appendix: More on the Gibbs Phenomenon . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Fourier Transform
2.1 A First Look at the Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Appendix: Chase the Constant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Examples
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 How Does the Graph of f (ax) Compare with the Graph of f (x)? . . . . . . . . . . . . . .
2.5 Getting to Know Your Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 Convolution
1
1
2
4
9
10
12
16
21
26
33
36
38
39
50
58
59
61
62
65
65
75
75
78
79
91
ii
CONTENTS
3.1 A ∗ is Born . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 What is Convolution, Really? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Properties of Convolution: It’s a Lot like Multiplication . . . . . . . . . . . . . . . . . . . .
3.4 For Whom the Bell Curve Tolls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
91
95
97
99
3.5 Appendix: Evaluation of the Gaussian Integral
. . . . . . . . . . . . . . . . . . . . . . . . . 101
3.6 Convolution in Action I: A Little Bit on Filtering . . . . . . . . . . . . . . . . . . . . . . . . 102
3.7 Convolution in Action II: Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.8 Appendix: Didn’t We Already Solve the Heat Equation? . . . . . . . . . . . . . . . . . . . . 113
3.9 Convolution in Action III: The Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . 116
3.10 The Central Limit Theorem: The Bell Curve Tolls for Thee . . . . . . . . . . . . . . . . . . 128
3.11 Appendix: The Mean and Standard Deviation for the Sum of Random Variables
. . . . . . 130
3.12 More Details on the Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.13 Appendix: Heisenberg’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
4 Distributions and Their Fourier Transforms
135
4.1 The Day of Reckoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.2 The Right Functions for Fourier Transforms: Rapidly Decreasing Functions . . . . . . . . . 140
4.3 Appendix: A Very Little on Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.4 Appendix: The Riemann-Lebesgue lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
4.5 Appendix: Smooth Windows
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
4.6 Distributions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.7 Appendix: A Physical Analogy for Distributions
. . . . . . . . . . . . . . . . . . . . . . . . 165
4.8 Appendix: Limits of Distributions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.9 Appendix: Other Approximating Sequences for δ . . . . . . . . . . . . . . . . . . . . . . . . 166
4.10 The Fourier Transform of a Tempered Distribution . . . . . . . . . . . . . . . . . . . . . . . 169
4.11 Fluxions Finis: The End of Differential Calculus
. . . . . . . . . . . . . . . . . . . . . . . . 175
4.12 Approximations of Distributions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
4.13 Appendix: The Generalized Fourier Transform Includes the Classical Fourier Transform . . 179
4.14 Appendix: 1/x as a Principal Value Distribution . . . . . . . . . . . . . . . . . . . . . . . . 180
4.15 Operations on Distributions and Fourier Transforms . . . . . . . . . . . . . . . . . . . . . . 181
4.16 Duality, Changing Signs, Evenness and Oddness
. . . . . . . . . . . . . . . . . . . . . . . . 182
4.17 A Function Times a Distribution Makes Sense . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.18 The Derivative Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
4.19 Shifts and the Shift Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.20 Scaling and the Stretch Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
CONTENTS
iii
4.21 Convolutions and the Convolution Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
4.22 δ Hard at Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5 Sampling
209
5.1 X-Ray Diffraction: Through a Glass Darkly1
. . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.2 The III Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
5.3 The Fourier Transform of III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
5.4 Appendix: Periodic Distributions and Fourier series
. . . . . . . . . . . . . . . . . . . . . . 217
5.5 Appendix: How Special is III? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.6 Sampling Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
5.7 Sampling and Interpolation for Bandlimited Signals
. . . . . . . . . . . . . . . . . . . . . . 225
5.8
Interpolation a Little More Generally . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
5.9 Finite Sampling for a Bandlimited Periodic Signal
. . . . . . . . . . . . . . . . . . . . . . . 230
5.10 Appendix: Timelimited vs. Bandlimited Signals . . . . . . . . . . . . . . . . . . . . . . . . . 233
5.11 Appendix: Periodizing sinc Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
5.12 Troubles with Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
6 Discrete Fourier Transform
249
6.1 From Continuous to Discrete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
6.2 The Discrete Fourier Transform (DFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.3 Two Grids, Reciprocally Related . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
6.4 Appendix: Gauss’s Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
6.5 Getting to Know Your Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . 259
6.6 Periodicity, Indexing, and Reindexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
6.7
Inverting the DFT and Many Other Things Along the Way . . . . . . . . . . . . . . . . . . 262
6.8 Properties of the DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
6.9 Appendix: Different Definitions for the DFT . . . . . . . . . . . . . . . . . . . . . . . . . . 275
6.10 The FFT Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
6.11 Zero Padding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
7 Linear Time-Invariant Systems
293
7.1 Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
7.2 Examples
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
7.3 Cascading Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
7.4 The Impulse Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
7.5 Linear Time-Invariant (LTI) Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
iv
CONTENTS
7.6 Appendix: The Linear Millennium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
7.7 Appendix: Translating in Time and Plugging into L . . . . . . . . . . . . . . . . . . . . . . 306
7.8 The Fourier Transform and LTI Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
7.9 Matched Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
7.10 Causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
7.11 The Hilbert Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
7.12 Appendix: The Hilbert Transform of sinc . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
7.13 Filters Finis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
7.14 Appendix: Geometric Series of the Vector Complex Exponentials . . . . . . . . . . . . . . . 328
7.15 Appendix: The Discrete Rect and its DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
8 n-dimensional Fourier Transform
333
8.1 Space, the Final Frontier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.2 Getting to Know Your Higher Dimensional Fourier Transform . . . . . . . . . . . . . . . . . 345
8.3 Appendix: The Stretch Theorem and Friends . . . . . . . . . . . . . . . . . . . . . . . . . . 357
8.4 Higher Dimensional Fourier Series
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
8.5
III, Lattices, Crystals, and Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
8.6 Crystals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
8.7 Bandlimited Functions on R2 and Sampling on a Lattice . . . . . . . . . . . . . . . . . . . . 380
8.8 Appendix: The Poisson Summation Formula, Again . . . . . . . . . . . . . . . . . . . . . . 383
8.9 Naked to the Bone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
8.10 The Radon Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
8.11 Getting to Know Your Radon Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
8.12 Appendix: Clarity of Glass
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
8.13 Medical Imaging: Inverting the Radon Transform . . . . . . . . . . . . . . . . . . . . . . . . 395
A Mathematical Background
403
A.1 Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
A.2 The Complex Exponential and Euler’s Formula . . . . . . . . . . . . . . . . . . . . . . . . . 406
A.3 Algebra and Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
A.4 Further Applications of Euler’s Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
B Some References
413
Chapter 1
Fourier Series
1.1
Introduction and Choices to Make
Methods based on the Fourier transform are used in virtually all areas of science and engineering. By
whom? For starters:
• Circuit designers
• Spectroscopists
• Crystallographers
• Anyone working in signal processing and communications
• Anyone working in imaging
I’m expecting that many fields and many interests will be represented in the class, and this brings up an
important issue for all of us to be aware of. With the diversity of interests and backgrounds present not
all examples and applications will be familiar and of relevance to all people. We’ll all have to cut each
other some slack, and it’s a chance for all of us to branch out. Along the same lines, it’s also important for
you to realize that this is one course on the Fourier transform among many possible courses. The richness
of the subject, both mathematically and in the range of applications, means that we’ll be making choices
almost constantly. Books on the subject do not look alike, nor do they look like these notes — even the
notation used for basic objects and operations can vary from book to book. I’ll try to point out when a
certain choice takes us along a certain path, and I’ll try to say something of what the alternate paths may
be.
The very first choice is where to start, and my choice is a brief treatment of Fourier series.1 Fourier analysis
was originally concerned with representing and analyzing periodic phenomena, via Fourier series, and later
with extending those insights to nonperiodic phenomena, via the Fourier transform. In fact, one way of
getting from Fourier series to the Fourier transform is to consider nonperiodic phenomena (and thus just
about any general function) as a limiting case of periodic phenomena as the period tends to infinity. A
discrete set of frequencies in the periodic case becomes a continuum of frequencies in the nonperiodic case,
the spectrum is born, and with it comes the most important principle of the subject:
Every signal has a spectrum and is determined by its spectrum. You can analyze the signal
either in the time (or spatial) domain or in the frequency domain.
1 Bracewell, for example, starts right off with the Fourier transform and picks up a little on Fourier series later.
2
Chapter 1 Fourier Series
I think this qualifies as a Major Secret of the Universe.
All of this was thoroughly grounded in physical applications. Most often the phenomena to be studied
were modeled by the fundamental differential equations of physics (heat equation, wave equation, Laplace’s
equation), and the solutions were usually constrained by boundary conditions. At first the idea was to use
Fourier series to find explicit solutions.
This work raised hard and far reaching questions that led in different directions. It was gradually realized
that setting up Fourier series (in sines and cosines) could be recast in the more general framework of orthog-
onality, linear operators, and eigenfunctions. That led to the general idea of working with eigenfunction
expansions of solutions of differential equations, a ubiquitous line of attack in many areas and applications.
In the modern formulation of partial differential equations, the Fourier transform has become the basis
for defining the objects of study, while still remaining a tool for solving specific equations. Much of this
development depends on the remarkable relation between Fourier transforms and convolution, something
which was also seen earlier in the Fourier series days. In an effort to apply the methods with increasing
generality, mathematicians were pushed (by engineers and physicists) to reconsider how general the notion
of “function” can be, and what kinds of functions can be — and should be — admitted into the operating
theater of calculus. Differentiation and integration were both generalized in the service of Fourier analysis.
Other directions combine tools from Fourier analysis with symmetries of the objects being analyzed. This
might make you think of crystals and crystallography, and you’d be right, while mathematicians think of
number theory and Fourier analysis on groups. Finally, I have to mention that in the purely mathematical
realm the question of convergence of Fourier series, believe it or not, led G. Cantor near the turn of the
20th century to investigate and invent the theory of infinite sets, and to distinguish different sizes of infinite
sets, all of which led to Cantor going insane.
1.2 Periodic Phenomena
To begin the course with Fourier series is to begin with periodic functions, those functions which exhibit
a regularly repeating pattern. It shouldn’t be necessary to try to sell periodicity as an important physical
(and mathematical) phenomenon — you’ve seen examples and applications of periodic behavior in probably
(almost) every class you’ve taken. I would only remind you that periodicity often shows up in two varieties,
sometimes related, sometimes not. Generally speaking we think about periodic phenomena according to
whether they are periodic in time or periodic in space.
1.2.1 Time and space
In the case of time the phenomenon comes to you. For example, you stand at a fixed point in the ocean (or
on an electrical circuit) and the waves (or the electrical current) wash over you with a regular, recurring
pattern of crests and troughs. The height of the wave is a periodic function of time. Sound is another
example: “sound” reaches your ear as a longitudinal pressure wave, a periodic compression and rarefaction
of the air. In the case of space, you come to the phenomenon. You take a picture and you observe repeating
patterns.
Temporal and spatial periodicity come together most naturally in wave motion. Take the case of one
spatial dimension, and consider a single sinusoidal wave traveling along a string (for example). For such a
wave the periodicity in time is measured by the frequency ν, with dimension 1/sec and units Hz (Hertz =
cycles per second), and the periodicity in space is measured by the wavelength λ, with dimension length
and units whatever is convenient for the particular setting. If we fix a point in space and let the time
vary (take a video of the wave motion at that point) then successive crests of the wave come past that