Pierre Bremaud - Mathematical Principles of Signal Processing

Mathematical Principles of Signal Processing

Springer-Verlag Berlin Heidelberg GmbH

Pierre Bremaud

Mathematical Principles of Signal Processing Fourier and Wavelet Analysis

,

Springer

Pierre Bremaud Ecole Polytechnique Federale de Lausanne Switzerland and INRIAJEcole Normale Superieure France [email protected]

Library of Congress Cataloging in Publication Data Bremaud, Pierre. Matbematical principles of signal processing / Pierre Bremaud. p. cm. Includes bibliographical references and index. ISBN 978-1-4419-2956-3 ISBN 978-1-4757-3669-4 (eBook) DOI 10.1007/978-1-4757-3669-4 1. Signal processing-Matbematics. I. TitIe. TK5102.9.B72 2001 621.382'2'OI51-dc21 2001042957 Printed on acid-free paper.

© 2002 Springer Science+Business Media New York Originally published by Springer Science+Business Media New York, Inc in 2002 Softcover reprint of tbe hardcover I st edition 2002 All rights reserved. This work may not be translated or copied in whole or in part witbout tbe written permission oftbe publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection witb reviews or scholarly analysis. U se in connection witb any form of information storage and retrievaI, electronic adaptation, computer software, or by similar or dissimilar metbodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in tbis publication, even if tbe former are not especially identified, is not to be taken as a sign tbat such names, as understood by tbe Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Allan Abrams; manufacturing supervised by Joe Quatela. Typeset by The BartIett Press, Inc., Marietta, GA.

987 6 5 4 3 2 1 SPIN 10845428

ToMarion

Contents

Preface

xi

A Fourier Analysis in LI

1

Introduction

3

Al Fourier Transforms of Stable Signals A 1·1 Fourier Transform in L I Al·2 Inversion Formula . . . . . . . .

7 7 16

A2 Fourier Series of Locally Stable Periodic Signals A2·1 Fourier Series in L}oc . . . . . . . . A2·2 Inversion Formula . . . . . . . . . .

23

A3 Pointwise Convergence of Fourier Series A3·1 Dini's and Jordan's Theorems. A3·2 F6jer's Theorem . . . A3·3 The Poisson Formula References . . . . . . .

31

46

B Signal Processing

49

Introduction

51

BI Filtering B 1·1 Impulse Response and Frequency Response Bl·2 Band-Pass Signals . . . . . . . . . . . . . .

55 55 68

23 26 31 39 43

viii

Contents

B2 Sampling B2·1 Reconstruction and Aliasing . B2·2 Another Approach to Sampling B2·3 Intersymbol Interference . B2-4 The Dirac Formalism . . . . .

75 75 82 84 88

B3 Digital Signal Processing B3·1 The DFf and the FFf Algorithm B3·2 The Z-Transform . . . . . . . . . B3·3 All-Pass and Spectral Factorization

95 95 100 109

B4 Subband Coding B4·1 Band Splitting with Perfect Reconstruction . B4·2 FIR Subband Filters References. . . . . . . . . . .

115 115 120 126

C

Fourier Analysis in L 2

127

Introduction

129

Cl Hilbert Spaces C1·1 Basic Definitions. C1·2 Continuity Properties C1·3 Projection Theorem .

133

C2 Complete Orthonormal Systems C2·1 Orthonormal Expansions . C2·2 Two Important Hilbert Bases

145 145

C3 Fourier Transforms of Finite-Energy Signals C3·1 Fourier Transform in L 2 C3·2 Inversion Formula in L 2 • • • • • • • • •

155 155 159

C4 Fourier Series of Finite-Power Periodic Signals C4·1 Fourier Series in LToc . . . . . . . . . . . C4·2 Orthonormal Systems of Shifted Functions References. . . . . . . . . . . . . . . . . . . . .

161

D

Wavelet Analysis

133 136 139

150

161 163 166

167

Introduction

169

D1 The Windowed Fourier Transform D1·1 The Uncertainty Principle . . . . . . . . . D1·2 The WFf and Gabor's Inversion Formula .

175 175

D2 The Wavelet Transform D2·1 Time-Frequency Resolution of Wavelet Transforms D2·2 The Wavelet Inversion Formula . . . . . . . . . . .

185

178

185

187

Contents

03 Wavelet Orthonormal Expansions D3·1 Mother Wavelet . . . . . . . D3·2 Mother Wavelet in the Fourier Domain D3·3 Mallat's Algorithm. . . . . . . . . .

ix

195 195 202 211

04 Construction of an MRA D4·1 MRA from an Orthonormal System . D4·2 MRA from a Riesz Basis .. D4·3 Spline Wavelets . . . . . . . . . . .

217

05 Smooth MuItiresolution Analysis D5·1 Autoreproducing Property of the Resolution Spaces D5·2 Pointwise Convergence Theorem . . . D5·3 Regularity Properties ofWavelet Bases References . . . . . . . . . . . . . . . . . . .

229

217 220 223 229 231 234 237

Appendix

239

The Lebesgue Integral References. . . .

241

Glossary of Symbols

263

Index

267

261

Preface

Fourier theory is one of the most useful tools in many applied sciences, particularly, in physics, economics, and electrical engineering. Fourier analysis is a well-established discipline with a long history of successful applications, and the recent advent of wavelets is the proof that it is still very alive. This book is an introduction to Fourier and wavelet theory illustrated by applications in communications. It gives the mathematical principles of signal processing in such a way that physicists and electrical engineers can recognize the familiar concepts of their trade. The material given in this textbook establishes on firm mathematical ground the field of signal analysis. It is usually scattered in books with different goals, levels, and styles, and one of the purposes of this textbook is to make these prerequisites available in a single volume and presented in a unified manner. Because Fourier analysis covers a large part of analysis and finds applications in many different domains, the choice of topics is very important if one wants to devise a text that is both of reasonable size and of meaningful content. The coloration of this book is given by its potential domain of applications-signal processing. In particular, I have included topics that are usually absent from the table of contents of mathematics texts, for instance, the z-transform and the discrete Fourier transform among others. The interplay between Fourier series and Fourier transforms is at the heart of signal processing, for instance in the sampling theory at large (including multiresolution analysis). In the classical Fourier theory, the formula at the intersection of the Fourier transform and the Fourier series is the Poisson formula. In mathematically oriented texts, it appears as a corollary or as an exercise and in most cases receives little attention, whereas in engineering texts, it appears under its avatar, the formula

xii

Preface

giving the Fourier transfonn of the Dirac combo For obscure reasons, it is believed that the Poisson sum fonnula, which belongs to classic analysis, is too difficult, and students are gratified with a result of distributions theory that requires from them a higher degree of mathematical sophistication. Surprisingly, in the applied literature, whereas distribution theory is implicitly assumed to be innate, the basic properties of the Lebesgue integral, such as the dorninated convergence and the Fubini theorem, are never stated precisely and seldom used, although these tools are easy to understand and would certainly answer many of the questions that alert students are bound to ask. In order to correct this unfortunate tradition, which has a demoralizing effect on good students, I have insisted on the fact that the c1assical Poisson fonnula is all that is needed in signal processing to justify the Dirac symbolism, and I have devoted some time and space to introduce the Lebesgue integral in a concise appendix, giving the precise statements of the indispensable tools. The contents are organized in four chapters. Part A contains the Fourier theory in LI up to the c1assical results on pointwise convergence and the Poisson sum fonnula. Part B is devoted to the mathematical foundations of signal processing. Part C gives the Fourier theory in L 2 . Finally, Part D is concemed with the timefrequency issue, inc1uding the Gabor transfonn, wavelets, and multiresolution analysis. The mathematical prerequisites consist of a working knowledge of the Lebesgue integral, and they are reviewed in the appendix. Although the book is oriented toward the applications of Fourier analysis, the mathematical treatment is rigorous, and I have aimed at maintaining a balance between practical relevance and mathematical content.

Acknowledgments Michael Cole translated and typed this book from a French manuscript, and Claudio Favi did the figures. Jean-Christophe Pesquet and Martin Vetterli encouraged me with stimulating discussions and provided the illustrations of wavelet analysis. They also checked and corrected parts of the manuscript, together with Guy Demoment and Emre Telatar. Sebastien Allam and Jean-Fran~ois Giovanelli were always there when TEX tried to take advantage of my incompetence. To all of them, I wish to express my gratitude, as well as to Tom von Foerster, who showed infinite patience with my prornises to deliver the manuscript on time. Gif sur Yvette, France May 2,2001

Pierre Bremaud

Part

A

Fourier Analysis in L1

Introduction

In 1807 Joseph Fourier (1768-1830) presented a solution ofthe heat equation l

ae

a2e

-=K-,

at

a2x

where e(x, t) is the temperature at time t and at loeation x of an infinite rod, and is the heat eonduetanee. The initial temperature distribution at time 0 is given:

K

e(x,O) = f(x).

(The solution of the heat equation is derived in Seetion A 1·1.) In fact, Fourier eonsidered a cireular rod of length, say, 21T, whieh amounts to imposing that the funetions x --+ f(x) and x --+ e(x, t) are 21T-periodie. He gave the solution when the initial temperature distribution is a trigonometrie series f(t)

= L cne int . neZ

Fourier claimed that his solution was general beeause he was eonvineed that alI21Tperiodie funetions ean be expressed as a trigonometrie series with the eoefficients Cn

1

= cn(f) = 21T

1

2lT

0

f(t)e -int dt.

lThe definitive form of his work was published in Theorie Analytique de la Chaleur, Finnin Didot ed., Paris, 1822.

4

Part A Fourier Analysis in L I

Special cases of trigonometric developments were known, for instance, by Leonhard Euler (1707-1783), who gave the formula 1

.

1 .

1 .

"2 x = sm(x) - "2 sm(2x) + 3" sm(3x) -

"',

true for - l ( < x < +l(. But the mathematicians of that time were skeptical about Fourier's general conjecture. Nevertheless, when the propagation of heat in solids was set as the topic for the 1811 annual prize of the French Academy of Sciences, they surmounted their doubts and attributed the prize to Fourier's memoir, with the explicit mention, however, that it lacked rigor. Fourier's results that were in any case true for an initial temperature distribution that is a finite trigonometric sum, and be it only for this, Fourier fully deserved the prize, because his proof uses the general tricks (for instance, the differentiation rule and the convolution-multiplication rule) that constitute the powerful toolkit of Fourier analysis. Nevertheless, the mathematical problem that Fourier raised was still pending, and it took a few years before Peter Gustav Dirichlet2 could prove rigorously, in 1829, the validity of Fourier's development for a large class of periodic functions. Since then, perhaps the main guideline of research in analysis has been the consolidation of Fourier's ingenious intuition. The classical era of Fourier series and Fourier transforms is the time when the mathematicians addressed the basic question, namely, what are the functions adrnitting a representation as a Fourier series? In 1873 Paul Dubois-Reymond exhibited a continuous periodic function whose Fourier series diverges at O. For almost one century the threat of painful negative results had been looming above the theory. Of course, there were important positive results: Ulisse Dini3 showed in 1880 that if the function is locally Lipschitz, for instance differentiable, the Fourier series represents the function. In 1881, Carnille Jordan4 proved that this is also true for functions of locally bounded variation. Finally, in 1904 Leopold Fejeii showed that one could reconstruct any continuous periodic function from its Fourier coefficients. These results are reassuring, and for the purpose of applications to signal processing, they are sufficient. However, for a pure mathematician, the itch persisted. There were more and more examples of periodic continuous functions with a Fourier series that diverges at at least one point. On the other hand, Fejer had proven that if convergence is taken in the Cesaro sense, the Fourier series of such continuous periodic function converges to the function at all points.

2Sur la convergence des series trigonometriques qui servent arepresenter une fonetion arbitraire entre des limites donnees, J. reine und angewan. Math., 4,157-169. 3 Serie di Fourier e altre rappresentazioni analitiche delle funzioni di une variabile reale, Pisa, Nistri, vi + 329 p. 4Sur la serie de Fourier, CRAS Paris, 92, 228-230; See also Cours d'Analyse de l'lfcole Polytechnique, I, 2nd ed., 1893, p. 99. 5Untersuchungen über Fouriersehe Reihen, Math. Ann., 51-69.

Introduction

5

Outside continuity, the hope for a reasonable theory seemed to be completely destroyed by Nikola'i Kolmogorov, 6 who proved in 1926 the existence of a periodic locally Lebesgue-integrable function whose Fourier series diverges at alt points! It was feared that even continuity could foster the worst pathologies. In 1966 JeanPierre Kahane and yitzhak Katznelson7 showed that given any set of null Lebesgue measure, there exists a continuous periodie function whose Fourier series diverges at all points of this preselected set. The case of continuous functions was far from being elucidated when Lennart Carleson8 published in the same year an unexpected result: Every periodic locally square-integrable function has an almost-everywhere convergent Fourier series. This is far more general than what the optirnistic party expected, since the periodic continuous functions are, in particular, locally square-integrable. This, together with the Kahane-Katznelson result, completely settled the case of continuous periodic functions, and the situation finally tumed out to be not as bad as the 1873 result of Dubois-Reymond seemed to forecast. In this book, the reader will not have to make her or his way through a jungle of subtle and difficult results. Indeed, for the traveler with practical interests, there is a path through mathematics leading directly to applications. One of the most beautiful sights along this road may be Simeon Denis Poisson's9 sum formula

L J(n) = L nEZ

jen),

nEZ

where J is an integrable function (satisfying some additional conditions to be made precise in the main text) and where j(v)

=

L

J(t)e-2irrvt dt

is its Fourier transform, where ~ is the set of real numbers. This striking formula found very nice applications in the theory of series and is one of the theoretical results founding signal analysis. The Poisson sum formula is the culrninating result of Part A, which is devoted to the classical Fourier theory.

6Une serie de Fourier-Lebesgue divergente partout, CRAS Paris, 183, 1327-1328. 7S ur les ensembles de divergence des series trigonometriques, Studia Mathematica, 26,

305-306. 8Convergence and growth of partial sums of Fourier series, Acta Math., 116, 135-157. 9S ur la distribution de la chaleur dans les corps solides, J. Ecole Polytechnique, 1geme Cahier, XII, 1-144, 145-162.

Al Fourier Transforms of Stable Signals

A 1·1

Fourier Transform in LI

This first chapter gives the definition and elementary properties of the Fourier transform of integrable functions, which constitute the specific calculus mentioned in the introduction. Besides linearity, the toolbox of this calculus contains the differentiation rule and the convolution-multiplication rule. The general problem of recovering a function from its Fourier transform then receives a partial answer that will be completed by the results on pointwise convergence of Chapter A3. We first introduce the notation: N, Z, Q, ~, C are the sets of, respectively, integers, relative integers, rationals, real numbers, complex numbers; N+ and ~+ are the sets of positive integers and nonnegative real numbers. In signal theory, functions from ~ to C are called (complex) signals. We shall use both terminologies (function, or signal), depending on whether the context is theoretical or applied. We denote by L~(~) (and sometimes, for short, LI) the set offunctions f(t) 10 from ~ into C such that

L

If(t)1 dt <

00.

In analysis, such functions are called integrable. In systems theory, they are called stable signals. IOWe shall often use this kind of loose notation, where a phrase such as "the function f(t)" means "the function f : lR ~ c." We shall also use the notation "I" or "fO" with a mute argument. For instance, "f(· - a)" is the function t --+ f(t - a).

P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

8

Al.

Fourier Transforms of Stable Signals

Let A be a subset of IR. The indicator function of Ais the function lA : IR {O, 1} defined by

1o 1

lA(t) =

1-+

if tE A, ift

~

A.

The function I(t) is called locally integrable if for any closed bounded interval [a, b] C IR, the function I(t)l[a,bj(t) is integrable. We shall then write

I or, for short, I E Lloe' The set of functions

E

L~ loe(lR)

I (t) from IR into C such that

L

I/(t)1 2 <

00

is denoted by L~(IR). It is the set of square-integrable functions. A signal I(t) in this set is said to have a finite energy E =

L

I/(t)1 2 dt.

The function I(t) is called locally square-integrable if for any closed bounded interval [a, b] C IR, the function I(t)l[a,bj(t) is square-integrable. We shall then write

I

E

L~,loe(lR)

or, for short, I E L toe' We recall that in L~(IR) or L~(IR) two functions are not distinguished if they are equal almost everywhere with respect to the Lebesgue measure. EXERCISE AI.I. Give an example 01 a function that is integrable but not 01finite energy. Give an example 01 a function that is 01finite energy but not integrable 01 finite energy. Show that

A function I : IR 1-+ C is said to have bounded support if there exists a bounded interval [a, b] c IR such that I(t) = 0 whenever t ~ [a, b]. If the function I(t) is n times continuously differentiable (that is, it has derivatives up to order n, and these derivatives are continuous), we say that it is in Cn . If it is in Cn for all n E N, it is said to belong to COO • The kth derivative of the function I (t), if it exists, is denoted jCk)(t). The Oth derivative is the function itself: I(O)(t) = I(t); in particular, CO is the collection of continuous functions from IR to C. The set of continuous functions with bounded support is denoted by C~.

AI·I Fourier Transfonn in LI

9

Fourier Transform We can now give the basic definition. DEFINITION AI.I. Let s(t) be a stable complex signal. The Fourier transform (FT) ofs(t) is thefunctionfrom:IR into C:

s(v) =

L

s(t) e-2i1rvI dt.

(1)

(Note that the argument of the exponential in the integrand is -2i7rvt.) The mapping from the function to its Fourier transform will be denoted by s(t) ~ s(v)

or

:F: s(t) --* s(v).

Table A 1.1 gives the immediate properties of the Fourier transform. Table AI.I. Elementary Properties of Fourier Transfonns Delay Modulation Doppler

s(t - to)

~ e- 2ill'vtos(v)

e2i1rvot s(t)

~

s(v - vo)

s(at)

Fr --+

-IsA{V} lai a

AISI (t)

+ A2S2(t)

~ AISI(V) + A2 S2(V) ~

s*(t)

s(-v)*

EXERCISE

A1.2. Prove the assertions in Table Al.I.

EXERCISE

AI.3. Prove the modulation theorem: Fr 1 A s(t) cos(27rvot) --+ 2(s(v - Vo ) + sA( v

+ Vo )).

(2)

(See Fig. AI.I.) EXERCISE

AI.4. Show that the FT of a real signal is Hermitian even, that is, s(- v) = s(v)*.

Show that the FT of an odd (resp., even; resp., real and even) signal is odd (resp., even; resp., real and even). EXERCISE

A1.S. Defining the rectangular pulse recT(t) = I I- t .+ t1 (t)

s(v)

o

Hs(v

v

-vo

+ vo) + s(v -

o

Figure Al.l. Modulation theorem

vo)}

+vo

v

10

Al.

Fourier Transfonns of Stable Signals

and the cardinal sine

sinc (x)

sin(Jrx)

= --Jrx

show that (see Fig. Al.2) recT(t)

~ Tsinc (vT).

(3) T

1

-T/2

o

+T/2

1

T

2

T

3

T

Tsinc(vT) = recT(v)

reCT(t)

Figure A 1.2. Fourier transfonn of the rectangle function We will show that the Gaussian pulse is its own Fr, that is,

(4) In order to compute the corresponding Fourier integral, we use contour integration in the complex plane. First, we observe that it is enough to compute the Fr s( v) for v 2: 0, since this Fr is even (see Exercise A1.4). Take a 2: v (eventually, a will tend to 00). Consider the rectangular contour Y in the complex plane (see Fig. A1.3), Y

= Yl + Y2 + Y3 + Y4,

where the Yi 's are the oriented line segments

Yl : (-a, 0) -+ (+a, 0), Y2 : (+a, 0) -+ (+a, v), Y3 : (+a, v) -+ (-a, v), Y4: (-a, v) -+ (-a, 0).

v ')'4

-a

+a

Figure A1.3. The integration path in the proof of (4)

A 1·1 Fourier Transform in L I

We denote by Wehave

-Yi

11

the oriented segment whose orientation is opposite that of Yi.

i

e- rrz2

+ /z + h + 14,

dz = 11

where li is the integral of e- rr Z2 along Yi. Since the latter integrand is a holomorphic function, by Cauchy's theorem (see, for instance, Theorem 2.5.2, p. 83, of [Al], or Theorem 2.2, p. 101, of [A6]),

i

e- rrz2

= 0,

dz

and therefore,

+ /z + h + 14 =

h

O.

We now show that lim /z

a~oo

= a-+oo !im 14 = o.

For /Z, for instance, if we parameterize Y2 as folIows, Y2

then

/z

=

l

v

i dt

e-rr(a+itf

Therefore, since v .:::: a,

l/zl .:::: =

= {a + it;O,:::: t .:::: v},

l

a

=

l

e-rr(a-t)(a+t)

e- rra

21o

a

errat

v

e-rr(a2-t2)e-2irrat

dt .::::

l

e-rra(a-t)

1 dt = -(1 _

dt

e- rra 2 ),

Jra

where the last quantity tends to 0 as a tends to for 14, with sirnilar computations. Therefore, !im (h

a-HXl

a

i dt.

+ h) =

+00. A sirnilar conclusion holds 0,

that is, (5)

Using for YI the obvious parameterization !im { a--+oo

JYl

=

lim a--+oo

j+a e- rrt2

dt

= { e- rrt2 dt = 1.

-a

JJ!I!.

Parameterizing -Y3 as folIows, -Y3 = {iv

+ t; -a .:::: t

.:::: +a},

12

Al.

Fourier Transforms of Stahle Signals

wehave

1

= l+ a e-n(iv+tf dt -a

-)'3

Therefore,

Going back to (5), we obtain

•

which gives the announced result. EXERCISE

A1.6. Deduce Jrom (4) that, Jor all Cl > 0,

The Fr of the Gaussian pulse can be obtained by other means (see Exercise A 1.16). However, in other cases, contour integration is often necessary. Using contour integration in the complex plane, we show that, for a > 0,

= e-at IlR+(t)

s(t)

Fr

~

s(v) A

First observe that s(v)

=

1

00

o

. e- 2znvt e- at dt

1 2mv + a

= . =

1 2inv + a

=

1 1

00

0

Y

1 . a+2mv

(6)

. e-2znvt-a\2inv + a) dt

e- Z dz.

(The reader is refered to Fig. A1.4 for the definition ofthe lines y, YJ. Y2, and Y3.) Therefore, it suffices to show that

i

By Cauchy's theorem,

1

e-Z dz

Yl

+

e- z dz

1

e- Zdz

Y2

= 1. +

1

e- Zdz

)'3

= 0.

A 1·1 Fourier Transform in LI

13

2i7f1/

Figure A1.4. The integration path in the proof of (6)

The limit as A t 00 of I Yl e- Z dz is I y e- Z dz, and that of I Y3 e- Z dz = IoA e- t dt is 1. It therefore remains to show that the limit as A t 00 of J,Y2 e- Z dz is 0, and this foIlows from the bound

I{

e-zdzl ::: e- A IY21,

•

where IY21 = K x A is the length of Y2. EXERCISE

AI.7. Deducefrom (6) that

Convolution-Multiplication Rule THEOREM

AI.I. Let h(t) and x(t) be two stable signals. Then the right-hand side

oJ y(t)

=1

(7)

h(t - s)x(s) ds

is defined Jor almost all t and defines almost everywhere a stable signal whose FT is y(v) = h(v)x(v).

Proof"

By ToneIli's theorem and the integrability assumptions, f1xIR Ih(t - s)llx(s)1 dt ds = (l'h(t)' dt) (l'X(t)' dt) <

00.

This implies that, for almost aIl t, l'h(t - s)x(s)1 ds <

00.

The integral IIR h(t - s )x(s) ds is therefore weIl defined for almost aIl t. Also, 1,y(t)' dt

=

111

h(t - s)x(s)

dsl dt

::: 11'h(t - s)x(s)1 dt

ds <

00.

14

Al.

Fourier Transfonns of Stable Signals

Therefore, y(t) is stable. By Fubini's theorem, L(Lh(t - S)X(S)dS) e-2irrvt dt

=L

Lh(t - s)e-2irrv(t-s)x(s)e-2irrvs ds dt

•

= h(v)x(v).

The funetion y(t), the convolution of h(t) with x(t), is denoted by y(t) = (h

* x)(t).

We therefore have the convolution-multiplication rule, (h

Fr * x)(t) --+ h(v)x(v). A

(8)

EXAMPLE AI.L

The convolution of the rectangular pulse reeT (t) with itself is the triangular pulse of base [- T, + T] and height T, TriT(t) = (T - Itl)1[-T,+T](t).

By the convolution-multiplication rule, TriT(t)

~ (Tsine (vT)f

(9)

(see Fig. Al.5). EXERCISE ALS.

Let x(t) be a stable complex signal. Show that its autoeorrelation

funetion c(t)

=L

x(s

+ t)x*(s) ds

is well defined and integrable and that its FT is

Ix(v)1 2 • T2

A

T

-T

& I

I I

+T

i

C'>~ I ~C'> I . 1 j I

-~ -~ -~

0

~

TriT(t)

Figure A 1.5. FT of the triangle funetion

~

~

Al·l Fourier Transform in LI

15

EXERCISEAl.9. Showthatthenthconvolutionpoweroff(t) = e-atlt~o(t), where

a > 0, is fM(t)

(/*3

= f * f * f,

t n- 1

= (n -

I)!

e- at I t>o(t). -

etc.) Deducefrom this the FTofs(t)

= tne-atlt~o(t).

Riemann-Lebesgue Lemma

The Riemann-Lebesgue lemmal! is one of the most important technical tools in Fourier analysis, and we shall use it several times, especially in the study of pointwise convergence of Fourier series (Chapter A3). THEOREM

Al.2. The FT of a stable complex signal s(t) satisfies lim Is(v)1

Ivl-+oo

= O.

(10)

Proof' The Fr of a rectangular pulse s(t) satisfies Is(v)1 ::; K/lvl [see Eq. (3)]. Hence every signal s(t) that is a finite linear combination of indicator functions of intervals satisfies the same property. Such finite combinations are dense in Lb(l~) (Theorem 28 of the appendix), and therefore there exists a sequence sn(t) of integrable functions such that

lim (Isn(t) - s(t)1 dt

n-+oo

JIR

=0

and

Kn

A

ISn(v)1 ::; ~'

for finite numbers Kn • From the inequality Is(v) - sn(v)1 ::;

L

Is(t) - sn(t)1 dt,

we deduce that

::; -K n + lvi

i

IR

Is(t) - sn(t)1 dt,

from which the conclusion follows easily.

•

The following uniform version of the Riemann-Lebesgue lemma will be needed in the sequel. 11 Riemann, B., (1896), Sur la possibilite de representer une fonetion par une serie trigonometrique, Oeuvre Math., p. 258.

16

Al.

Fourier Transfonns of Stable Signals

Al.3. Let f(t) be a 2:rr-periodic locally integrable function, and let CbeinC I , where [a,b] ~ [-:rr, +:rr]. Then

THEOREM

g: [a,b]

f-*

lim l

....... 00

b a

fex - u)g(u) sin(Au) du = 0

uniformly in x. Proof

For arbitrary E: > 0, choose a 2:rr-periodic function h(t) in Cl such that

r:rr

If(x) - h(x)1 dx < E:

(Theorem 29 of the appendix). Integrating by parts yields /(A)

=l

b

=-

COS(AU) Ib hex - u)g(u) - A a

hex - u)g(u) sin(Au) du

+l

b

a

[hex - u)g(u)]

,

COS(AU) du. A

Since h E Cl and is periodic, h and h' are uniformly bounded. The same is true of g, g' (g is in Cl). Therefore, lim /(A)

....... 00

=0

uniformly in x .

Now,

11

b

fex - U)g(U)Sin(AU)dUI :S I/(A)I

+l

:S I/(A)I

+ a~~ Ig(U)ll b Ih(x -

:S I/(A)I

+

b

Ih(x - u) - fex - u)llg(u)1 sin(Au) du u) - fex - u)1 sin(Au) du

max Ig(u)IE:.

a:'Ou:'Ob

The conc1usion then follows because E: is arbitrary.

AI· 2

•

Inversion Formula

Al.lO. Show that the FT of a stable signal is uniformly bounded and uniformly continuous.

EXERCISE

Despite the fact that the FT of an integrable signal is uniformly bounded and uniformly continuous, it is not necessarily integrable. For instance, the FT of the rectangular pulse is the cardinal sine, a non-integrable function. When its FT is integrable, a signal admits a Fourier decomposition.

AI· 2 Inversion Fonnula

17

THEOREM At.4. Let set) be an integrable complex signal with the Fourier transform s(v). Under the additional condition

L

Is(v)1 dv <

the inversion formula set)

=

(11)

00,

L

s(v)e+2iJrvt dv

(12)

holdsfor almost all t.lf set) is, in addition to the above assumptions, continuous, equality in (12) holds for all t.

(Note that the exponent ofthe exponential ofthe integrand is +2irrvt.) EXERCISE

At.ll. Check that the above result is true for the signal (a E lR, a > 0, a E C).

Proof' We now proceed to the proof of the inversion formula. (lt is rather technical and can be skipped in a first reading.) Let set) be a stable signal and consider the Gaussian density function

with the Fr

We first show that the inversion formula is true for the convolution (s Indeed, (s

* h u )(t).

* hu)(t) = JR{ s(u)hu(u)e...L ';;2 (t)du, u

2u 2

(13)

and the Fr ofthis signal is, by the convolution-multiplication formula, s(v)hu(v). Computing this Fr directly from the right-hand side of (13), we obtain s(v)hu(v)

= (

J~

s(u)hu(u) ( { e

I

• u

J~ ~;;r

(t)e-2iJrvt dt) du

= J~{ s(u)hu(u)e ~,;;r (v) du. I

u

Therefore, using the result ofExercise A1.11, { s(v)hu(v)e2iJrvt dv

J~

= {( { s(u)hu(u)e J~ J~

I

~

,U

;;r

(v) dU) e2iJrvt dv

= JR { s(u)hu(u)e Zc;2' -' 0,

= 1, limha(u) = 1, a'\-O

forall a > 0,

ha(u) du

forall u

E

IR.

ALl. Let h a : IR ---+ IR be a regularizing function. Let set) be in Lb(IR).

Then lim a'\-O

f

JIR

I(s

* h a )(t) -

s(t)1 dt

= 0.

20

Al.

Fourier Transforms of Stable Signals

Proof" We ean use the proof of Theorem A1.4, starting from (15). The only plaee where the speeifie form of h" (a Gaussian density) is used is (16). We must therefore prove that lim ( J(u)h,,(u) "-1-0 JIR

=0

independently. Fix e > O. Sinee limuto J(u) = 0, there exists a = aCe) such that

e

h,,(u) du :s -. 1-a~ J(u)h,,(u) du :s -2el~ -a 2

Sinee J(u) is bounded (say, by M), ( J(u)h,,(u) du :s M ( h,,(u) du. JIR\[ -a.+a] JIR\[ -a.+a]

The last integral is, for suffieiently small a, less than e12M. Therefore, for suffieiently small a,

(

JIR J(u)h,,(u) du :s

e

e

2" + 2" = e.

•

The funetion h" is an approximation of the Dirae generalized funetion o(t) in that, for all O. In view of the last result, it suffices to prove (65) with fjJ(u)

= {f(x + u) -

f(x

+ O)} + {f(x -

f(x

1

11

-

n

0

8

=

rJ(c),

sin2(!nu) 1 ; fjJ(u) du

u

c n

< -

1ry

sin 2(!nu)

0

u2

du

+ -1

1 -IfjJ(u)1-

n ~

8

u2

du.

The last integral is bounded; and therefore, the last term goes to 0 as n for the penultimate term, it is bounded by Ac, where

A

A3·3

(67)

u) - f(x - O)}.

Since fjJ(u) tends to 0 as n ~ 00, for any given c > 0 there exists rJ rJ :'S 8, such that IfjJ(u)1 :'S c when 0 < u :'S rJ. Now,

o<

+ 0)

=

1

00

o

sin2(!v)

v

2

dv <

00.

t

00.

As

•

The Poisson Formula

The following corollary of F6jer's theorem will play the key role for the proof of the Poisson sum formula (Theorem A3.l2).

44

A3. Pointwise Convergence of Fourier Series

COROLLARY A3.1.

that, for some x

E

Let f be a 2rc -periodie locally integrable function and suppose lR.,

(a) thefunction f is continuous at x, and (b) its Fourier se ries Then A Proof'

sI (x) converges to some number A.

= fex).

From (b) we see that lim

ntoo

0'1 (x) = A.

From F6jer's theorem and (a),

lim

ntoo

•

0'1 (x) = fex).

We have already given a weak version of the Poisson sum formula in Section A2·2. A most interesting situation is when the function cI>(t) defined by (31) is equal to its Fourier series for all t E lR., that is,

Ls(t nEZ

+ nT) = ~ LS(f) e2in !ft

for all tE R

(68)

nEZ

The next theorem extends the result in Exercise A2.8. THEOREM A3.12. Let set) be a stable complex signal, and let 0 < T < fixed. Assume in addition that

(1) LnEZ set

(2)

LnEZ

00

be

+ n T) converges everywhere to some continuous function,

s( f) e2in !ft converges for all t.

Then the strang Poissonformula (68) holds. Proof' The result is an immediate consequence of both the weak Poisson summation result (Theorem A2.3) and the corollary of F6jer's theorem in Section A3·2. • Here are two important cases for which the strong Poisson sum formula holds. COROLLARY A3.1. Let set) be a stable complex signal, and let 0 < T < 00 be fixed. If, in addition, L set + nT) converges everywhere to a continuousfunction that has bounded variation, then the Poissonformula (68) holds.

Praof' We must verify conditions (1) and (2) ofTheorem A3.12. Condition (1) is part of the hypothesis. Condition (2) is a consequence of Iordan's theorem A3.6. • A3.1. If set) is continuous, has bounded support, and has bounded variation, the Poisson sumformula (68) holds.

EXAMPLE

A3·3 The Poisson Formula COROLLARY

A3.2. If a stable continuous signal s(t) satisfies

s(t) s(v)

=

0(1

=

oe

as Itl

+\tl"')

+llvl"')

~ 00,

as lvi

~ 00,

for some a > 1, then the Poisson formula (68) holds for alt Proof

45

(69)

°< T <

The result is an immediate corollary of Theorem A3.12.

00.

•

Convergence Improvement

The Poisson formula can be used to replace aseries with slow convergence by one with rapid convergence, or to obtain some remarkable formulas. Here is a typical example. For a > 0,

s(t) = e-2naltl ~ s(v) =

n(a 2

a

+ v2 )

.

Since

Ls(t +n)

= Le-2nalt+nl

nEZ

nEZ

is a continuous function with bounded variation, we have the Poisson formula, that is,

The left-hand side is equal to 2

1 - e- 2na

-1,

and the right-hand side can be written as

Therefore, 1

L

nO": 1

Letting a

~

n 1 + e- 2na

a 2 + n 2 = 2a 1 - e- 2na

1 2a 2 .

0, we have

The general feature of the above example is the following. We have aseries that is obtained by sampling a very regular function (in fact, C OO ) but also slowly

46

A3. Pointwise Convergence of Fourier Series

h(t-2T)

~ [jU[jU[j[j[j~ 1

-3T -2T

-T

0

T

2T

3T

-3T -2T

-T O T

2T

3T

Figure A3.1. Radar return signal

decreasing. However, because of its strong regularity, its Fr has a fast decay. The series obtained by sampling the Fr is therefore quickly converging. Radar Return Signal

Let s(t) be a signal ofthe form

s(t) =

(I>(t -

nT») f(t).

(70)

nE'L

(We may interpret h(t - nT) as a return signal of the nth pulse of a radar after reftection on the target, and f (t) as a modulation due to the rotation of the antenna. ) The Fr ofthis signal is (see Fig. A3.1)

s(v) = ~ Lh(!!.-) f(v T

nE'L

(t)

T

!!.-). T

(71)

EXERCISE A3.4. Show that if(1) f is integrable, (2) LnE'L h(t -n T) is integrable and continuous, and (3) LnE'L h(n/T) < 00, then (71) holds true. Find other conditions.

References [Al] Ablowitz, M.J. and Jokas, A.S. (1997). Complex Variables, Cambridge University Press. [A2] Bracewell, R.N. (1991). The Fourier Transform and Its Applieations, 2nd rev. ed., McGraw-Hil1; New York. [A3] Gasquet, C. and Witomski, P. (1991). Analyse de Fourier et Applieations, Masson: Paris. [A4] Helson, H. (1983). Harmonie Analysis, Addison-Wesley: Reading, MA. [A5] Katznelson, Y. (1976). An Introduetion to Harmonie Analysis, Dover: New York. [A6] Kodaira, K. (1984). Introduetion to Complex Analysis, Cambridge University Press. [A7] Körner, T.W. (1988). Fourier Analysis, Cambridge University Press.

References

47

[A8] Rudin, W. (1966). Real and Complex Analysis, McGraw-Hill: New York. [A9] Titchmarsh, E.C. (1986). The Theory of Funetions, Oxford University Press. [AlO] Tolstov, G. (1962). Fourier Series, Prentice-Hall (Dover edition, 1976). [All] Zygmund, A. (1959). Trigonometrie Series, (2nd ed., Cambridge University Press.

Part

B

Signal Processing

Introduction

The Fourier transform derives its importance in physics and in electrical engineering from the fact that many devices mapping an input signal x(t) into an output signal y(t) have the following property: If the input is a complex sinusoid e2invt, the output is T(v)e2invt, where T(v) is a complex function characterizing the device. For example, when x(t) and y(t) are, respectively, the voltage observed at the input and the steady-state voltage observed at the output of an Re circuit (see Fig. BO.I), the input-output mapping takes the form of a linear differential equation: y(t)

+ RCy(t) = x(t),

and it can be readily checked that T(v)

=

I 1+ 2irrvRC

The Re circuit is one of the physical devices that transform a signal into another signal, that satisfy the superposition principle, and that are time-invariant. More precisely: R

1'~1' C y(t)

x(t)

l"""

,1,,)

Figure BO.I. The Re circuit

52

Part B Signal Processing

(a) If Yl (t) and Y2(t) are the outputs corresponding to the inputs Xl (t) and X2(t), then AlYl (t) + A2Y2(t) is the output corresponding to the input AlXl (t) + A2X2(t); (b) If y(t) is the output corresponding to x(t), then y(t - -r) is the output corresponding to x(t - -r). Such physical devices are caIled (homogeneous linear) filters. A basic example is the convolutional filter, for which the input-output mapping takes, in the time domain, the form y(t)

=

1

h(t - s)x(s)ds,

where h(t) is caIled the impulse response, because it is the response of the filter when the Dirac pulse 8(t) is applied at the input. Indeed, h(t) =

1

h(t - s)8(s) ds.

If the impulse response is integrable, the output is weIl defined and integrable, as long as the input is integrable. Then, by the convolution-multiplication rule, the expression of the input-output mapping in the frequency domain is y(v) = T(v)x(v),

where T (v) is the frequency response, that is, the Fr of the impulse response: T(v) =

1

h(t)e-2i1Cvt dt.

Observe that if the input is x(t) = e-2i1Cvt, the output is weIl defined and equal to

1

h(s)x(t - s)ds

=

1

h(s)e- 2i1CV (t-s)ds

= T(v)e-2i1Cvt,

in accordance with what was said in the beginning of this introduction. In the particular case of the RC filter, the solution of the differential equation with arbitrary initial condition at -00 is indeed of the convolution type, with the impulse response

The RC filter is a convolutional filter, and it contains the typical features of the more general filters. In the general case, since a filter is a mapping, we shall have to define its domain of application. Depending on this domain, the inputoutput mapping takes different forms. In the above informal discussion of the RC circuit, there are a frequency-domain and a time-domain representation and also a representation in terms of a linear homogeneous differential equation. The latter is not general. In fact, when it is available, the transmittance is a rational function ofthe frequency v. The corresponding filters, caIled rational filters, form an important class, and Chapter BI gives the basic concepts concerning analog (that is, continuous-time) filters.

Introduction

53

In addition to filtering, there are two fundamental operations of interest in communications systems: frequency transposition and sampling. Frequency transposition is a basic technique of analog communications. It has two main applications, the first of which is transmission. Indeed, the Hertzian channels are in the high-frequency bands-in fact, much higher than the one ofbrute signals such as the electric signals carrying voice, for instance-and consequently, the latter have to be frequency-shifted. The second reason is resource utilization and is related to frequency multiplexing, a technique by which signals initially occupying the same frequency band are shifted to nonoverlapping bands and can then be simultaneously transmitted without mutual interference. From a mathematical point of view the theory of frequency transposition (or, equivalently, of band-pass signals, to be defined in Chapter BI) is not difficult. It remains interesting because of the special phenomena associated with this technique, such as cross-talk in quadrature multiplexing and dispersion phenomena. In digital communications systems, an analog signal s(t) is first sampled, and the result is a sequence of sampies s(n T), n E Z. It is important to identify conditions under which the sampie sequence faithfully represents the original signal. The central result of Chapter B2 is the so-called Shannon-Nyquist theorem, which says that this is true if the signal s(t) is stable and continuous and if the support of its Fr s( v) is contained in the interval [-1/ T, + I/Tl. The original signal can then be recovered by the reconstruction formula: s(t) =

L s (nT) sinc (f - n) . nEZ

The theory of sampling is an application of the results obtained in Part A, and in particular of the Poisson sum formula. The above reconstruction formula has many sourees, 1 and its importance in communications was fully realized by Claude Shannon and Harald Nyquist. The Shannon-Nyquist sampling theorem is the bridge between the analog (physical) world and the discrete-time (computational) world of digital signal processing. The reader will find in the main text abrief discussion of the interest of digital communications systems. Therefore, a large portion of this Part B is devoted to discrete-time signals (Chapters B2-B4). As we have already mentioned, the Poisson sum formula is the key to the sampling theorem. It also plays a very important role in the numerical analysis of the discrete Fourier transform considered as an approximation of the continuous Fourier transform (see Chapter B3) and also in the intersymbol interference problem (see Chapter B2). The study of the interaction between discrete time and continuous time is not limited to the sampling theorem. For instance, we prove that filtering and sampling ISee J.R. Higgins, Five short stories about the cardinal series, Bult. Amer. Math. Soc., 12, 1985,45-89.

54

Part B Signal Processing

cmumute for base-band signals. This is not a difficult result, but it is of course a fundamental one because in signal processing, one first sampies and then performs the filtering operation in the sampled domain, since one of the advantages of digital processing comes precisely from the difficulty of making analog filters. One advantage of analog processing is that it is instantaneous. To maintain competitivity, the signal processing algorithms have to be fast. For instance, the discrete Fourier transform is implemented by the so-called fast Fourier transform, an algorithm whose principle we briefty explain in this Part. Subband coding also has a fast algorithm associated with it. It is a data compression technique. The signal is not directly quantized, but instead, it is first analyzed by a filter bank, and the output of each filter bank is quantized separately. This allows one to dispatch the compression resources unequally, with fewer bits allocated to the subbands that are less informative (see the discussion in Chapter B4). Subband coding is the last topic of Part Band introduces the sections on multiresolution analysis in Part D.

BI Filtering

B 1·1

Impulse Response and Frequency Response

Convolutional Filter We introduce a particular and very important dass of filters. DEFINITION Bl.l. The transformation fram the stable signal x(t) to the stable signal y(t) defined by the convolution

y(t)

=

1

h(t - s)x(s)ds,

(1)

where h(t) is stable, is ca lied a convolutional filter. This filter is called a causal filter if h(t) = 0 for t < O.

The signal y(t) is the output, whereas the signalx(t) is the inputofthe linear filter with impulse response h(t). Informally, if x(t) is the Dirac generalized function 8(t) (an impulse at time 0), the output is (see Fig. Bl.1)

1

h(t - s)8(s) ds = h(t),

whence the terminology. A causal filter responds only after it has been stimulated. For this reason, it is sometimes also called a realizable filter (Fig. B 1.1. features a causal impulse response). For such filters, the input-output relationship (1) becomes (note the P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

56

BI. Filtering

8(t)

I

f\h(t~ >

0

0

impulse

>

V

impulse response

Figure B 1.1. Impulse and impulse response

upper limit of integration) y(t) DEFINITION

=

[t

oo

h(t - s)x(s) ds.

(2)

B1.2. The Fourier transform ofthe (stable) impulse response h(t), T(v) =

L

h(t)e-2i1Cvt dt,

(3)

is ca lied the frequency response. If the input is the complex sinusoid x(t)

= e2i1Cvt, by (1), the output is

y(t) = T(v)e2i1Cvt.

(4)

(Note that the output is weH defined by the convolution formula, even though in this particular case the input is not integrable.) EXERCISE Bl.1. Let y(t) be the output of a stable and causal convolutional filter with impulse response h(t) [see (2)]. Let

z(t)

=

1 t

h(t - s)x(s)ds,

t

~ 0,

be the output of the same filter, when the input x(t) is applied only from time t = 0 on. Show that

lim Iz(t) - y(t)1

tt+oo

= O.

A More General Definition Convolutional filters are only a special dass of filters. A more general definition is as foHows. Denote by C ~ the set of functions of lR. into C. DEFINmON Bl.3. Let D(12) be a set of functions from lR. into C with the two following properties:

(a) It is closed under linear operations; (ß) it is closed under translation. 12 : D(12)

1-+

C ~ is ca lied a homogeneous linear filter with domain D(12) if:

(i) 12 is linear, and (ii) 12 is time-invariant.

B 1·1 Impulse Response and Frequency Response

The meaning ofproperties (a) and (ß) is the following: (a) XI (t), X2(t) E

C

===}

x(t - T)

E

D('c).

AI, A2

+ A2X2(t)

AIXI (t)

E D('c);

and (ß) x(t)

E D('c),

T

E

The meaning of properties (i) and (ii) is the following: (i) XI (t), X2(t) AI, A2

c

E C,XI(t)~·

A2Y2(t); (ii) x(t) EXERCISE

YI(t),X2(t)

E D('c),

T

E

c

~

Y2(t)

B1.2. Show that if e 2i :rcvt

E D('c),

· e 2mvt

C

~

D('c),

lR

E

===}

D('c),

C

AIXI(tHA2X2(t) ~ AI(t)YI(tH

===}

lR, x(t) ~ y(t)

E

57

x(t - T) ~ y(t - T).

===}

then

T(v)e 2·mvt

(5)

for some complex number T(v). The function T(v) is called thefrequency response ofthe filter. Every frequency response is of the form

T(v) where G(v) EXAMPLE

= G(v)eiß(v),

(6)

= IT(v)1 is the amplitude gain and ß(v) = Arg T(v) is the phase.

Bl.l. Let D('c) = {x(t) :

or D('c) = {x(t)

+ e(t) :

L

Ix(t)1 dt < oo},

L

Ix(t)1 dt <

00

and e(t)

E

E},

where E is the set of complex finite linear combinations of complex exponentials. For any signal in D('c), the right-hand side of (1) is welt defined, and we can therefore define the filter ,C with domain D('c) by the input-output relationship (1). Thefrequency response, as defined by (5), is then the FT ofh(t). Let h(t) E L~(lR). We shalt see in Part C that the FT h(v) = T(v) of h(t) can be defined and that it is in L~(lR). We take D('c) = LUlR) and define ,C by the input-output relationship EXAMPLE B1.2.

y(t)

=

L

T(v)x(v)e 2i :rcvt dv,

(7)

where xCv) is the FT ofthe input x(t) E L~(lR). The right-hand side of(7) has a meaning since T(v) and xCv) being in L~(lR) implies that T(v)x(v) is in Lt(lR) (see Theorem 20 of the appendix). EXAMPLE B 1.3. If T ( v) is an arbitrary function, not necessarily in L~ (lR), one can always define a filter ,C by the input-output relation (7), provided one chooses for domain D('c) the set of signals x(t) such that the right-hand side has a meaning.

58

B 1. Filtering

1

o

-B

+B

Low-pass (B) ~2B

----7

~2B

o

-VQ

----7

+VQ

Band-pass (vQ, B)

Figure B 1.2. Low-pass and band-pass frequency responses Low-Pass, Band-Pass, and Hilbert Filters The low-pass and band-pass filters (see Fig. B1.2) that we now define belong to the category of Example B 1.2. One calls low-pass (B) a filter with frequency response T(v) = 1[-B,+B](v),

(8)

where B is the cut-offfrequency. One calls band-pass (B, va), where 0< B < va, a filter with frequency response T(v)

= 1[-vo-B,-vo+Bj(v) + l[vo-B,vo+Bj(v),

(9)

where Va is the center frequency, and 2B is the bandwidth.

Hilbert's filter (see Fig. B 1.3) belongs to the category of Example B 1.3. It is the filter with frequency response where T(O) = O.

T(v) = i sgn (v),

(10)

One possible domain for Hilbert's filter is the set of stable (resp., finite-energy) signals whose FT has compact support. The amplitude gain of Hilbert's filter is 1 (except for v = 0, where the gain is zero), and its phase is ß(v)

=

I

Jr /2

if v > 0,

0

if v

-Jr /2

if v < O.

+i

= 0,

r - ,- - - - - -

,0 - - - - - - - ' , -i

Hilbert filter

Figure B1.3. Hilbert frequency response

(11)

B 1·1 Impulse Response and Frequency Response

59

There is no function admitting the frequency response (10). There is, in fact, a generalized function (in the sense of the theory of distributions) with Fr equal to T(v). However, in signal theory, the Hilbert filter is used only in the theory of band-pass signals (see Section BI·2). For such signals the Hilbert filter coincides with a bona fide convolutional filter: EXERCISE Bl.3. Show that the output y(t) ofthe Hilbertfilter, corresponding to a stable signal x(t) having an FT x(v) that is null outside the frequency band [- B, + B], can be expressed as

y(t)

1

=-

R.

x(t - s)

2 sin 2 (n Bs) ns

ds.

Differentiation and Integration as Filters

Let D('c) be the set of signals

where

L

Ix(v)1 dv

L

=

x(t)

<

x(v)e2iJrvt dv,

and

00

L

Ivllx(v)1

(12)

<

00.

Such signals are continuous and differentiable with derivative

~ x(t) =

[(2inv)x(v)e2iJrvt dv. (13) JR. dt (Apply the theorem of differentiation under the integral sign; 15 ofthe appendix).

The mapping x(t) ~ dx(t)/dt is a linear filter, called the differentiating filter, or differentiator, with frequency response T(v)

= 2inv.

(14)

Let D('c) be the set of signals of the form (12), where

~ Ix(v)1 dv

JIR.

<

00

and

[lx(v)1 dv

JIR lvi

<

00.

The signal y(t)

= [

x.(v) e2iJrvt dv 2mv is in the domain of the preceding filter (the differentiator), and therefore,

JIR

~ y(t) = [ x(v)e2iJrvt dv = x(t). dt JIR The transformation x(t) ~ y(t) is a homogeneous linear filter, which is called the integrating filter, or integrator, with frequency response

1

T(v) = - . - . 2mv

(15)

60

BI. Filtering

y(t)

x(t)

y(t)

x(t)

.c2 *.c1

series

.c 2 +.c l

parallel

~----,-----:~ y (t)

Figure BlA. Series, parallel, and feedback configurations Series, Parallel, and Feedback Configurations

We now describe the basic operations on filters (see Fig. B1.4). Let C] and C2 be two convolutional filters with (stable) impulses responses h](t) and h 2 (t) and frequency responses T](v) and T2(V), respectively. The series filter C = C 2 * C] is, by definition, the convolutional filter with impulse response h(t) = (h] * h 2)(t) and frequency response T(v) = T](v)T2(V). It operates as folIows: The input x(t) is first filtered by Cl, and the output of C] is then filtered by C 2 , to produce the final output y(t). The parallel filter C = C] + C 2 is, by definition, the convolutional filter with impulse response h(t) = h](t) + h 2(t) and frequency response T(v) = T](v) + T2(V). It operates as folIows: The input x(t) is filtered by Cl, and "in parallel," it is filtered by C 2 , and the two outputs are added to produce the final output y(t). The feedback filter C = CI/(1 - C] * C 2 ) is, by definition, the convolutional filter with impulse response frequency response T(v) =

T](v)

1 - T](v)T2 (v)

This filter will be a convolutional filter if and only if this frequency response is the PT of a stable impulse response. 1fthis is not the case, one may define the feedback filter by the input-output relation A

y(v)

=

T](v)

A

1 _ T](v)T2 (v)x(v)

with, for instance, a definition along the lines of Example B 1.2.

B 1·1 Impulse Response and Frequency Response

61

The filter 'cl is the forward loop filter, whereas 'cl is the feedback loop filter. The forward loop processes the total input, which consists ofthe input x(t) plus the feedback input, that is, the output y(t) processed by the feedback loop filter. EXERCISE

B1.4. Consider the function 1 T(v) = 1 + 4n2v2'

Give the impulse response of the convolutional filter with the above jrequency response T (v). Interpret the filter as a feedback filter. Filtering of Decomposable Signals

We introduce the notion of a decomposable signal, because it allows one to rewrite the results conceming Fourier transforms and Fourier series in a unified manner, without recourse to symbolic expressions in terms of the Dirac generalized functions or, more generally, to the theory of distributions. DEFINITION

Bl.4. The signal set) is called decomposable ifit can be put into the

form set) where

=

l

e2i :rr:vt p,(dv),

(16)

p, is a complex measure on IR. whose total variation 1p,1 isfinite.

We recall that a complex measure of finite total variation is, by definition, a mapping /L : B(IR.) ~ O. We obtain sin(27r B~) sin(27r B~)" s(kT +~) = ak 27rB~ + f;;;oa k- j 27rB(~ _ jT)' g(t) =

We see that the error Is(kT

+~)

- akl

(76)

does not stay bounded for all bounded sequences {ad, because I . I L NO I~ - JT

=00.

(77)

2S ee H. Nyquist, (1928), Certain Topics ofTelegraph Transmission Theory, Trans. Amer. Inst. Elec. Eng., 47, 617-644.

86

B2. Sampling

A better pulse from this point of view is the "raised eosine" g(t)

= sine (2Bt)

eos(27f Bt) 1 _ 16B 2 t 2

'

(78)

whose Fr is g(v) = eos 2(7fV) 4B 1[-2B,+2Bj(v). A

(79)

In fact, sekT

+ D.) =

sine (4BD.) ak 1 _ 16B 2D. 2

+

'" sin(47fBD.) f#oa n- j 47f B(D. - jT)(1 - 16B2(D. - jT)2)'

and the error (76) is seen to remain bounded whatever the bounded sequenee {ad. Partial Response Signaling

Another disadvantage of the pulse (75) is that one eannot realize signals with an Fr that has an "infinite slope" (at - B and

+ B).

We shall see that, with clever encoding, we ean attain the Nyquist limit (74) (which says that in order to transmit a "symbol" an every T seeonds without intersymbol interferenee, a bandwidth of at least 2W = 2B ~ 1fT is needed), without resorting to an unrealizable pulse (with a very large slope). For example, in the duobinary encoding teehnique, instead of transmitting (7), one transmits S'(t) = L(an

+ an+l)g(t -

nT),

(80)

nEZ

that is, S'(t) = Lang'(t - nT),

(81)

nEZ

where g'(t) = g(t)

+ g(t + T).

(82)

With the pulse (75) ofminimal bandwidth 2B, starting from (80) we obtain s'(kT)

= ak + ak-l =

Cb

and from the sequence {Ck} and the initial datum ao we recover the sequenee {ak}' The interest of this teehnique is that we do not seek to implement Si (t) in the form (80) using the unrealizable pulse g(t), but rather in the form (81) with a realizable pulse g'(t). Indeed, 8'(V)

= (1 + e-2iJrvT)g(v) = 2T eos(7fvT)e-2iJrvT 1[-B,+Bj(v).

B2·3 Intersymbol Interference

87

This pulse has minimal bandwidth 2B, and, furthermore, it is easier to realize, not having an infinite slope. The above is a particular case of the technique of partial response signaling. 3 The general principle is the following: We pretend to use the unrealizable pulse g(t) given by (75), but in (71) we replace the symbol an by an encoding Cn , say, a linear encoding (83)

which gives S'(t)

= I>ng(t - nT). nEZ

In order to realize S'(t) it is rewritten in the form S'(t)

= .~:::>ng'(t - nT), nEZ

where g'(t)

= g(t) + y,g(t + T) + ... + Ykg(t + kT)

(84)

is a base-band (B) pulse, in general easily realizable, with FT 8'(v)

= T(v)g(v),

(85)

where k

T(v) = 1 + LYje-2inVjT = P(e-2invT)

(86)

j=!

and k

P(z)

= 1 + LYjz j .

(87)

j=!

By sampling at the time

t

= kT we obtain s'(kT) = p(z)ak = Ck.

We shall see in Seetion B3·2 that the sequence fad is deduced from the sequence { Ck} by inverse filtering ak =

1 P(z) Ck

(88)

(we assume that 1/ P(z) is stable and therefore that the corresponding filter is causal; these notions are discussed in detail in Section B3·2). 3See A. Lender (1981), Correlative (Partial Response) Teclmiques and Applications to Radio Systems, in Feher, K. (ed.), Digital Communications: Microwave Applications (Prentice-Hall: Englewood Cliffs, Ni), Ch. 7..

88

B2. Sampling

B2·4 The Dirae Formalism Do We Need Distributions Theory Here? In the applied literature, the Dirae formalism of generalized funetions is used profusely. It eonsists of a small set of symbolie roles that are justified by the classical Fourier theory of the previous ehapters. In signal proeessing, the Dirae formalism eulminates in the formula giving the Fr of a Dirae eomb. We shall see that the Poisson sum formula is, for all praetieal purposes, all that is needed to deal with such a mathematieal objeet in a rigorous way. We shall see in the next part that the Fourier theory has in the Hilbert spaee framework a high degree of formal beauty. There was yet another important step to be made in this direetion. The physicists had introdueed a very useful tool, the Dirae generalized funetion, associated with a formal ealeulus that was rather pleasant to use, but that laeked mathematieal foundations. These were established by Laurent Schwartz, with the elegant theory of distributions (or generalized funetions) and the equally elegant Fourier theory of tempered distributions. 4 Most engineers are familiar with the so-ealled Dirae funetion o(t), whieh is "defined" by the property

L

qJ(t)o(t) dt

= qJ(O),

for all funetions qJ(t). They are aware that there exists no such funetion in the usual sense with such property, and they take the above formula as a symbolie way of dealing with a limit situation. In the "prelimit," o(t) is replaeed by a proper funetion, depending on a parameter, say, n. There are many ehoiees for this proper funetion on(t), the simplest one being on(t)

= nree~(t).

Then for sufficiently regular funetion qJ(t) (say, eontinuous), lim ( qJ(t)on(t) dt

ntoo

JIR

= qJ(O).

Thus, in this point of view, the Dirae funetion is the limit of proper funetions beeoming more and more eoneentrated around the origin of times while their integral remains equal to 1. Another eandidate with these properties is the Gaussian pulse that we have already eneountered in the proof of the inverse Fourier formula: ha(t)

1

,2

= - - e--,;;z, a"fEi

where this time the positive parameter a tends to zero. Observe that the Fr ofboth on(t) and ha(t) (whieh we have previously eomputed) eonverge pointwise, as the 4Theorie des Distributions, Vols. 1 and 2,1950-1, Hermann, Paris.

B2·4 The Dirac Formalism

89

eorresponding parameters tend to the appropriate limits, to 1. This is eonsistent with the formal eomputation of the Fr of the Dirae funetion 8(v)

=

L

8(t)e-2irrvI dt

= e-2irrvO = 1.

Another generalized funetion that is omnipresent in the signal proeessing literature is the Dirae eomb (indeed a eosmetie too1!), also ealled the Dirae pulse train. It is the T -periodie generalized funetion Il T (t) =

L

nT).

8(t -

neZ

If we formally eompute its nth Fourier eoeffieient, we obtain -1

T

l

T

n dt 8(t)e- 2''" TI

0

= 1.

Now if we write the eorresponding formal Fourier series

-1

T

Le

2irr!!.v T

neZ

'

we observe that its eonvergenee is rather problematic. We ean, however, pursue the heuristies, and eonsider that the latter sum is the limit as N ~ 00 of the 1/ T -periodie funetion

1 +N

-

T

Le

2irr!!.v

-N

whieh is the Diriehlet kernel

T

'

!

1 sin(2rr(N + T) T sin(rrT) GraphieaIly, up to a multiplieative faetor 1/ T, such a funetion looks in the vicinity of 0 like a Dirae funetion: As N ~ 00, it becomes more and more coneentrated around 0, and its integral in a neighborhood of 0 tends to 1. Therefore, at the limit we have, invoking the 1/ T -periodicity, the Fourier transform of the Dirac comb

~T(V) = ~ L 8 (v - !!..) . T

neZ

T

This overdose of heuristics may weIl be fatal for the more critical mind. However, in most basic courses in signal analysis, it is administered with the best intentions, with the exeuse that it saves the student from a painful exposition to distributions theory. This apology of mathematical euthanasy is founded on wrong premiees. The first question that one should ask is: Do we need the Dirae comb in signal analysis? Looking back at the previous chapters, we ean immediately answer NO.1t is not needed to derive the Shannon-Nyquist theorem, because the Poisson formula is all that is needed there. Is the Poisson formula harder than distributions theory? Again, the answer is NO, without surprise, because the distributions theory version

90

B2. Sampling

of the Poisson sum formula is only a small ehapter of distributions theory. (I shall add that the heuristie derivation of the Poisson sum formula-see the eomment following the statement of Theorem A2.3 of Chapter l-is mueh more eonvineing than the usual heuristie derivation of the Fr of the Dirae eomb.)

In fact, the reader may skip this ehapter and proeeed to Chapters 3 and 4 without damage. On the other hand, the Fourier transform of the Dirae eomb is part of a well-established tradition in signal analysis that is bound to be etemal due to its aesthetie appeal. I have therefore devoted the next seetion to the expression of the classical results of Fourier analysis in the Dirae formalism. It is, however, a purely symbolie analysis. The Dirac Generalized Function The principal formal objeet of the Dirae formalism is the Dirae generalized funetion 8(t), and the first formal rule is the symbolie formula

L

qy(t)8(t - a)dt

EXAMPLE

= qy(a).

(Dl)

B2.1. By the first symbolie rule,

L

e-2iJrvt8(t - a)dt

= e-2iJrva,

that is, Fr 8(t ...,.. a) -+ e- 2·IJrva.

In partieular, the Fouriertransform ofthe Dirae generalizedjunetion is the eonstant junetion equal to l. B2.2. Let x(t) be a T -periodie signal, and let {xn}, n E Z, be the sequenee of its Fourier eoeffieients. In Seetion A2·I we defi-ned the FT x(t) symbolieally, by

EXAMPLE

Using the symbolieformula x(t)

=

L

e2iJrvt xCv) dv

and the symbolie rule (Dl), we then have x(t)

=L nEZ

r

J[{

e2iJrvtxn8(V -

-f)

= LXne2iJrft, nEZ

and we recover the inversionformula of Seetion A2·I.

dv

B2·4 The Dirae Fonnalism

91

EXAMPLE B2.3. Let x(t) be as in the previous example. /fit is the input of afilter with (stable) impulse response h(t) and withfrequency response T(v) = h(v), symbolic calculations give for the output

1

y(t) =

h(v)x(v)e2iJrvt dv

LXn JJ[{[ h(v)e2iJrvt8(v - -f) dv,

=

nEZ

that is,

The sequence

LYn} of Fourier coefficients ofy(t) is thus

A(n)A T xn,

Yn = h A

a result that we already know. The FT of the Dirac Comb

Consider the Dirae eomb I:lT(t) =

L 8(t - nT). nEZ

The seeond symbolie fomula, that we now introduee, gives the FT of this generalized funetion: (D2) EXAMPLE

B2.4.

The Poisson sum formula. The formal Plancherel-Parseval

equality

1

cp(t)I:lT(t) dt =

1

qJ'(v)Lr;:.(v) dv

gives, upon substituting into it I:lT(v)

= ~L

nEZ

8

(v - -f)'

the Poisson sum formula

Multiplication Rule

The third symbolie formula of the Dirae formalism eoneerns the multiplieation of a Dirae generalized funetion by a funetion in the usual sense: s(t)8(t - a)

== s(a)8(t - a).

(D3)

92

B2. Sampling

This rule is consistent with the first rule, in that

l

EXAMPLE

= s(a)cp(a) =

s(t)8(t - a)cp(t)dt

l

s(a)8(t - a)cp(t)dt.

B2.5. Sampling and Spectrum Folding. The train oJ sampled pulses

Si(t)

= Ls(nT)8(t -

nT)

netz:

may, in view oJ(D3), beJormally written Si(t) = s(t)Llr(t). Its symbolic FT is thereJore

Si(V) = s(v) * K;(v)

that is,

~

Sie v)

"~( = -1 ~ s vT netz:

Ifwe input TSi(t) into a low-pass [-I/T, output the signal set) with FT s(v)

= LS(V netz:

n) .

-

T

+ l/T]filter, we thereJore obtain at the

-f)

l[-t,+tl(v).

This is the equation describing spectrum Jolding (see Theorem B2.3). EXAMPLE

B2.6. The FT oJ a Radar Return Signal. Let us consider the signal

set)

= ( L h(t -

nT)) J(t)

netz:

= (h(t) * Llr(t))J(t) = v(t)J(t). Its FT is (by the convolution-multiplicationJormula) s(v)

=

l

V(fL)J(V - fL)dfL·

Now, on using the rule (D3): v(v)

= h(v)~r(v) =~ T

L h(v)8(v netz:

!!.-) T

B2·4 The Dirac Formalism

93

On the other hand,

Thuswe have

s(v) = ~ Lh(~) i(v - ~), nEZ

that is, Eq. (71) 0/ Section B2.3.

•

The examples above show how the Dirae symbolie ealeulus formally aeeounts for ea1culations of Fourier transforms. This symbolie ea1culus retrieves formulas already proven in the framework of the classical Fourier theory in LI, formulas that have been proved under eertain eonditions of regularity, and of integrability or summability. The symbolie ea1culus does not say under what eonditions the final symbolie formulas have a meaning, nor in what sense they must be interpreted (equalities almost everywhere? in LI?). For this reason, the Dirae symbolie ea1culus must be used with preeaution. From a mnemonic point of view, it ean be useful, as it allows one to obtain some formulas very quiekly, and "generally" these formulas are eorreet under eonditions that are "almost always" satisfied in praetiee. However, let us emphasize onee more the fact that these formulas have been obtained rigorously within the framework of Fourier transforms in LI.

B3 Digital Signal Processing

B3·1

The DFf and the FFT Algorithm

TheDFT Suppose we need to compute numerically the Fr of a stable signal s(t). In practice only a finite vector of sampies is available, s = (so, ... , SN-t>,

where Sn = s(nb.). The Fourier sum of this vector evaluated at pulsations 2k1t / N is the discrete Fourier transform (DFr). DEFINITION B3.1. The DFT (So, ... , SN-d, where

0/ s

L

=

(so, . .. ,SN-i) is

Wk

=

the vector S =

N-i

Sk =

sn e - i (21rkn/N).

n=O

The DFr is an approximation of the Fr, the quality of which depends on the parameters N and .1.. The first question to ask is: How to choose these parameters to attain a given precision? As we shall see, the answer is given by the Poisson sum formula. For the time being, we shall give the basic properties of the DFr without reference to a sampled signal. Let a = (ao, ... , aN-i) be a finite sequence of complex numbers. For the Nth root of unity, we adopt the following notation:

P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

96

B3. Digital Signal Processing

The finite sequence A

= (A o, ... , AN-I) defined by N-I

(89)

Am = Lanw';rn n=O is the DFT of a = (ao, ... , aN-d. THEOREM

B3.1. We have the inversion formula an

=

1 N-I N LAmw'Nmn . m=O

(90)

Proof'

N-I

N-I

~ m(k-n) = L...t ak L...t w N . k=O m=O ~

But if k =1= n, N-I

~ m(k-n) L...t w N = m=O

since

WN(k-n) _ 1 N

w Nk-n

1

-

=0

wZr = 1 when r =1= 0; on the other hand, for k = n, N-I

N-I

•

L w~(k-n) = L I = N. m=O m=O If we consider the periodic extensions of the finite sequences a = (ao, ... , aN - d and A = (Ao, ... , AN-d, defined by an+kN

= an,

(91)

Eqs. (89) and (90) remain valid since w 1. The ring 0/ convergence is defined by r2 = Iyl and rl = 0 (thus, in/act we have a disk 0/ convergence {Izl < lylJ that contains the unit circle). The Laurent expansion is in this case a power-series expansion in the neighborhood 0/ zero

(Izl < Iyl)· n;:::O To find the impulse response h n we must expand H(z) as apower series. Rut (_1)r-l(r - I)!(z - y)-r is the (r - I)st derivative 0/

_1_ = _ 2. (1 + ~ + ~ + ... + ~ + ... ),

z- y y Y y2 and there/ore,for Izl < y, (-Ir-l(r - I)!(z _ y)-r-l

1

L n(n y n=r-l

=- -

yn

00

1) ... (n - r

= _ 2. ~ (j + ~ yko

}!

zn-r+l

+ 2) - - , yn

I)! zj. _1_ . yJyr-1

Finally,

(z -

yY

(- IY ~ (j + r - I)! 1 ~ '1' ( 1) yr r- . j=O }.yJ

and, identifying this expression with hn with hn

= 0 if n <

= (-Ir

O.

j

Z,

L}:o h j zj, we obtain

(n + r -1)! (2.)r+n, n!(r - I)! Y

n 2: 0,

Izl

< y,

108

B3. Digital Signal Processing

Second Case: Ir I < 1. The Laurent expansion is then a power-series expansion in the neighborhood 0100: 1 --- = h_nz- n • Changing

Z

into

L

(z - r)'

l/ s,

nO":O

(-s1- r )-r = (-1)r-rrsr ( s - -r1)-r

1

Isl< - . Irl

We can use the previous calculations to obtain

and we obtain the anticausal filter h_ n

=

(n - I)! (r - 1)!(n - r)!

r

-n-r

,n

~

r,

where h n = 0 ifn > -r. Linear Recurrence Equations If Yn is the output of the filter with transfer function (129) corresponding to the stable input signal X n, we have y(w) = H(e-iw)x(w), that is, P(e-iw)ji(w)

=

Q(e-iw)x(w).

Now P(e-iw)y(w) is the Fourier surn of the signal Yn + L:~=l ajYn-j, and Q(e-iw)x(w) is the Fourier surn of X n + L:i=l bexn-e. Therefore, Yn

p

q

j=l

e=l

+ LajYn-j = X n + Lbexn-e,

(131)

or, syrnbolically, P(Z)Yn

= Q(z)xn.

The general solution of the recurrence equation (131) is the surn of an arbitrary solution and of the general solution of the equation without right-hand side p

Yn

+ LajYn-j j=l

= O.

This latter equation has for a general solution a weighted surn of terms of the form r(n)p-n, where p is aroot of P(z) and r(n) is a polynornial of degree equal to the multiplicity of this root minus one. If we are given X n , n E Z, and the initial conditions Yo, Y-l, ... , Y-p+l, the solution of (131) is cornpletely deterrnined.

B3·3 All-Pass and Spectral Factorization

109

In order that the general solution never blows up (it is said to blow up if limlnltoo /Yn/ = (0) whatever the stable input X n, n E Z, and for any initial conditions Y-p+l, ... , Y-l, Yo, it is necessary and sufficient that all the roots of P(z) have modulus strict1y greater than unity. A particular solution of (131) is

Yn

=L

k::::O

hkxn-k .

The output Yn is stable when the input X n is stable since the impulse response h n is itself stable, and therefore Yn does not blow up. Therefore, we see that in order for the general solution of (131) with stable input to be stable, it is necessary and sufficient that the polynomial P(z) has all its roots with modulus strict1y greater than 1. Xn

B3.1. The rational filter Q(z)/ P(z) is said to be stable and causal P(z) has all its roots outside the closed unit disk {/z/ ::: I}.

DEFINITION

if

Causality arises from the property that if P(z) has roots with modulus strictly greater than unity Q(z)/ P(z) = H(z) is analytic inside {/z/ < rz} where rz > 1. The LaUfent expansion of H(z) is then an expansion as an entire series H(z) = Lk::::O hkz k, and this means that the filter is causal (hk = 0 when k < 0). DEFINITION

invertible

B3.2.

The stable rational filter Q(z)/ P(z) is said to be causally I}.

if Q(z) has all its roots outside the closed unit disk {/z/ :::

In fact, writing the analytic expansion of P(z)/ Q(z) in the neighborhood of zero as Lk::::O WkZ k, we have

that is, X

B3·3

n=

L WkYn-k .

(132)

k::::O

All-Pass and Spectral Factorization

All-Pass Filters A particular case of a rational filter is the all-pass filter. THEOREM B3.3. Let Zi (l ::: i ::: L) be complex numbers with modulus strictly greater than 1. Then the transfer function L

*_ 1

H(z)=n~ i=l

Z - Zi

(133)

110

B3. Digital Signal Processing

satisfies

IH(z)1

Proof"

{

1

iflzl > 1.

(134)

Let ZZi* - 1 Hi() Z=--Z - Zi

be an arbitrary factor of H(z). If Izl Fejer's identity (z - ß)(z

1, we observe that IHi(z)1

-~) = -~zlz ß*

ß*

ßI 2

'

1, using

(135)

which is true for Izl = 1, ß E (~) ho(t - ~), 2B 2B

_1 2B JEa. . ~

where ho(t), h l (t), ho(t), h l (t) are the r~spective imp~lse responses corresponding to the frequency responses To(v), To(v), Tl(V), Tl(V). Sirnilarly the signal Yl (t) at level y in Fig. B4.1 is Yl(t)

= ~Xl(~)ho(t-

;).

118

B4. Subband Coding

Sampling at the rate 2B gives the sampie sequence

YI(n2B)

LX(~) ho(~B 2B

= Lk

_1 2B j

-

- ~). B

~) iio(!!.2B 2B

If we set 2B = 1 (this condition can be forced upon the system by a change of time scale), we find

YI(n)

=L

(142)

LX(j)h o(2k - j)iio(n - 2k),

k

j

with a similar expression for the output Y2(t) ofthe lower branch ofFig. B4.1. Down- and Up-sampling We shall now express the resuIts in terms of the operations of down-sampling and up-sampling, and then go back to (142). Let {xnlnez be a sequence of complex numbers and let m sequences {Ynlnez and {znlnez defined by

Yn

= Xnm '

= Xn, =0

nE Z,

E

N. Consider the

nEZ

and

{ Znm Zj For example, with m

if j is not divisible by m.

= 2, Xo

X2

Xl

Yo

X3

X4

Xs

Y2

YI

and

Xo

0

Xl

0

X2

0

X3

0

X4

0

Xs

Zo

Zl

Z2

Z3

Z4

Zs

Z6

Z7

Zg

Z9

ZIO

The sequence {YnlneZ is said to be obtained from the original sequence {xnlnez by down-sampling by a factor m. The corresponding operation is denoted as m,/... Up-sampling by a factor m, denoted as mt, is the operation that transforms {xnlnez into {znlnez. In this chapter, we are concemed with the case m = 2. For future use, we shall express the operation of down-sampling by 2 followed by up-sampling by 2 in terms of z-transforms (see Figure B4.3). Denote X(z) and R(z) thez-transforms ofthe sequences {x(n)lnez and {r(n)}neZ, respectively. The sequence {r(n)}neZ is therefore obtained from {x(n)}neZ by

B4·1 Band Splitting with Perfect Reconstruction

119

@1--_Y--'l(~_)_--1CWI-_~)_r_(n_)

x( n))

XW

RW

Figure B4.3. Down-sarnpling and up-sarnpling 1--------------

r--------------I

1

1

" - - - - - - - - ______ 1

ANALYSIS

SYNTHESIS

Figure B4.4. Subband coding in the Z-domain (1 split)

replacing all the entries with an odd index by a zero. Therefore, R(z) = LX(2n)z2n = nEZ

~ !LX(n)zn + Lx(n)(-zt) , nEZ

nEZ

that is, R(z)

= !{X(z) + X( -

(143)

z)}.

Going back to (142) and the similar expression for the lower branch of Fig. B4.1, we see that the whole system is equivalent in the z-domain to Fig. B4.4. From (143) we see that y(z)

= ! {X(z)Ho(z) + X( +

!{X(z)H1(z)

z)Ho( - z)} Ho(z)

+ X(Z)Hl( -

Z)}Hl(Z).

Separating the aliasing terms from the rest, y(z) =

! X(z){Ho(z)Ho(z) + H 1(z)H1(z))

Therefore, aliasing is eliminated if Ho( - z)Ho(z)

+ H 1(- Z)Hl (z) = 0,

(145)

and perfect reconstruction is obtained provided that ~

Ho(z)Ho(z)

- (z) = 2. + H 1(Z)Hl

(146)

120

B4. Subband Coding

B4· 2

FIR subband filters

Quadrature Mirrors Filters In or~r t'2., find a solution of (145) and (146), one ean first fix Ho and then find H I , Ho, HI in terms of Ho in order to satisfy the no-aliasing eondition (145). Then one ean determine Ho so that the perfeet reeonstruetion eonditions (146) ean be satisfied. Given Ho(z), one possible solution of (145) is 6 HI(z) = Ho(- z), {

~o(z)

H I (z)

= Ho(z), = - Ho( -

(147) z).

Assume that the filter Ho is symmetrie, that is, it has a symmetrie impulse response = ho(n), n E Z). Then

(h o( - n)

HI(z) = Ho(-z) = L(-lrho(n)zn nEZ

Ir ho( -

= L (-

n)zn

(symmetry of Ho)

nEZ

= L ( - l tho(n) (~)n Z

nEZ

Therefore, if Ho is symmetrie,

that is, in terms of pulsations, HI(e- iw )

= Ho(e-i(Jr-w»).

This means that the pulsation speetrum of HI is symmetrie with respeet to that of Ho with respeet to the frequeney n /2. This is why in this ease Ho and H I are said to be quadrature mirror filters (QMFs). Going back to (147)-and without assurning that Ho is symmetrie-the perfeet reeonstruetion eondition (146) beeomes, in terms of Ho: Ho(d - Ho( - Z)2 = 2.

(148)

One drawback of the solution (145) is the nonexistenee of a finite impulse response filter Ho satisfying it. However, we ean relax eondition (146) to Ho(z)Ho(z)

+ HI(z)HI(z) = 2z k

(149)

6Esteban, D., and Galand, C. (1977), Applications of quadrature mirror filters to splitband voice-coding schemes, Proc. IEEE Inf. Conf ASSP, Hartford, Connecticut, 191-195.

B4·2 FIR Subband Filters

121

for some K ::: 1, which means that we accept a delay of K time units to recover the input, and in this case FIR filters do exist.

B4.1. Taking the no-aliasing condition (147) into account, the relaxed condition (149) with K = 1 gives

EXAMPLE

Ho(z)2 - Ho( -

d

= 2z.

(150)

Afamous solution is the Haar filter Ho(z)

1

= -J2 (1 + z).

(151)

The relaxed condition (149) allows a "linear phase" corresponding to a delay K. For K ::: 2, we shall just mention that (149) does not have an exact solution with a FIR filter.

If in Fig. B4.I, the input signal x(t) is assumed to be band-pass [jB, (j + I)B], for some j ::: 1, the resulting output of the analyzer is the same as if the input had been frequency-shifted by j B, to obtain a base-band (B) signal. In fact, immediately after the sampler at rate B, at level y, we have the same signal, for both inputs (the pass-band signal x(t), or its base-band version). Therefore, the analyzer of Fig. B4.1 performs band splitting on the base-band (B) version of any band-pass [j B, (j + I)B] signal. Consequently, the analyzerof Fig. B4.4 behaves in the same way, with the additional feature ofbeing independent of B! This remark shows that the full program of subband coding can be achieved by a cascade ofthe analysis (resp., synthesis) structures ofFig. B4.4 (by anticipation, let us mention that this is similar to the structure of Mallat's algorithm in multiresolution analysis). Fig. B4.5 shows the analysis synthesis of the band [0, B] into four subbands [0, BI4], [BI4, B12], [BI2, 3BI4], [3BI4, B]. ---------------------

1---------------------1 1

I ____________________ J

~---------------------~

1

ANALYSIS

SYNTHESIS

Figure B4.5. Subband coding in the Z-domain (2 splits)

122

B4. Subband Coding

stage 1

stage 2

stage J stage 1

stage 2

stage J

Figure B4.6. Octave band filtering

In constant Q-filtering one decides not to split the high-frequency component. Thus at each stage of analysis only the "coarse" component (corresponding to low frequency) is further analyzed (see Fig. B4.6). Such a structure is also called a logarithmicjilter or an octave bandjilter. EXERCISE B4.2. (a): Verify that filtering by H(z) followed by up-sampling by 2 is equivalent to up-sampling by 2 followed by filtering by H(Z2). Show that the synthesis part in Fig. B4.6 with J = 3 is equivalent to Fig. B4.7. (b): With the Haar filter we have

-

Ho(z)

1 = -J2 (1 + z),

B4·2 FIR Subband Filters

123

Figure B4.7. Equivalent octave band filtering Give the impulse response of each of the fOUf filters in Fig. B4.7.

Another Solution Another class of solutions7 for the no-aliasing condition (145) is

= Z-l Ho( - Z-l), Ho(z) = Ho(C 1), ~l (z)

{

(152)

H 1(z) = zHo( - z).

The perfect reconstruction condition (146) then becomes Ho(z)Ho(Z-l)

+ H o(- z)Ho(- Z-l) =

2.

(153)

Since Ho is a real filter, HO(Z-l)

= H(z)*

for z

= e-ia>,

and (153) takes the form Ho(e-ia» 12

1

+ IHo(- e-i a»1 2 =

2.

(154)

We shall now exhibit a general solution of (152). We perform a change of notation that will be convenient in the chapter on multiresolution analysis: Ho(z) = H(z) =

L hnzn, neZ

1

ja> mo (w) = ,J2Ho(e- ), ml (w)

1 ja> = ,J2Hl(e).

7Smith, M.J.T., and Barnwell, m T.P. (1986), Exact reconstruction techniques for treestructured subband coders, IEEE Transactions ASSP, 34, 434-441.

124

B4. Subband Coding

In view of (152), we have ml(w)

= ei"'mo(w + JT)*,

and the perfect reconstruction condition (154) becomes Imo(w)12

+ Imo(w + JT)1 2 = 1.

(155)

The solution (152) is in terms of Ho(z), and therefore it suffices to obtain mo(w) satisfying (155). We seek a finite impulse response filter Ho(z), in which case mo(w) is a polynomial in e- i "'. We shall in fact look for a solution in the form 1 + ei"')N mo(w) = ( - 2 L(w),

where N :::: 1, and L(w) is a polynomial in e- i",. Letting Mo(w) = Imo(w)12,

we have Mo(w)

=

+ 1+

1

2N

i", 1

IL(w)1 2

= (cos2 (~))

N

IL(w)1 2.

But IL(w)1 2 is a real-valued polynomial in e- i "', and therefore it is a polynomial in cos(w). Since cos(w) = 1 - 2 sin 2(wj2), Mo(w)

= (COS2(~))N P(sin2(~)),

for some polynomial P. Condition (155) must be satisfied for all w, and therefore it is equivalent to (156) for all y E [0, 1]. Since two polynomials identical on [0, 1] are identical everywhere, the latter equality is for all y E IR. The polynomials yN and (1 - y)N have no common roots, and therefore, by Bezout's theorem, there exist two unique polynomials a and b, of degree::: N -1, such that (1 - y)N a(y)

This is true for all y

E

+ yNb(y) = 1.

IR, and in particular, replacing y by 1 - y,

(1 - y)Nb(1 - y)

+ yN a(1 -

y)

= 1.

By the uniqueness of a and b, it follows that b(y) = a(1 - y).

Therefore, (157) is (1 - y)N a(y)

+ yN a(1 -

y)

= 1.

(157)

B4·2 FIR Subband Filters

125

Therefore, P(y) = a(y) is a solution of (156). We have thuse proven that (156) admits at least one solution, and by the uniqueness in Bezout's theorem, this solution is the only one of degree :s N - 1. We have

L

N-l

a(y) = (1 - y)-N[l - yN a(1 - y)] =

(f+k-l) l + G(yN).

k=O

Since a is a polynomial of degree at N - 1, and therefore,

:s N -

1, it is equal to its Taylor series truncated

= L (f+k-l) l· N-l

a(y)

k=O

This solution is the unique one with degree :s N - 1. Observe that it is nonnegative for aH y E [0, 1], and therefore a solution to the initial problem. Call it PN and let P be the general solution. We have

(1 - y)N (P(y) - PN(y»

+ yN (P(1

- y) - P N(1 - y»

= 0.

This implies that P - PN is divisible by yN, that is, P(y) - PN(y) and (1 - y)N yN Q(y) + yN (1 - y)N Q(l - y»

=

yN Q(y),

= 0,

which implies Q(y) + Q(1- y» = 0. That is, Q is symmetrie with respect to 1/2, and therefore ofthe form Q(y) = R(1/2 - y) for an odd polynomial R. In summary, the general solution of (156) is P(y)

= N-l L (f+k-l) l + yN R k=O

(1 ) --

2

,

(158)

where R(y) is any odd polynomial such that P(y) so defined remains nonnegative for all y E [0, 1]. Having obtained Mo(w), it remains to extract its square root mo(w). But this can be done by spectral factorization, using Fejer's lemma. We shall elose this chapter on the basic principles of subband coding. Note, however, that other solutions were proposed, most notably "biorthogonal solutions,"8 which are more versatile and yield finite impulse response subband filters with better properties (of symmetry, for instance). We refer to the monograph [B12], where the reader will find a full and detailed treatment of this topic, as weH as additional references.

8Vetterli, M. Filter banks allowing peifect reconstruction, Signal Processing, 10 (3), 1986,219-244.

126

B4. Subband Coding

References [BI] [B2] [B3] [B4] [B5] [B6] [B7] [B8] [B9] [BIO] [B 11] [BI2]

Ablowitz, M.J. andJokas, A.S. (1997). Complex Variables, Cambridge University Press. Daubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf. Series in Applied Mathematics, SIAM: Philadelphia, PA. Gasquet, C. and Witomski, P. (1991). Analyse de FourieretApplications, Masson: Paris. Haykin, S. (1989). An Introduction to Analog and Digital Communications, Wiley: New York. Hirsch, M.W. and Smale, S. (1974). Differential Equations, Dynamical Systems, and Linear Algebra, Academic Press: San Diego. Kodaira, K. (1984). Introduction to Complex Analysis, Cambridge University Press. Lighthill, MJ. (1980). An Introduction to Fourier Analysis and Generalized Functions, Cambridge University Press. Nussbaumer, H.J. (1981). Fast Fourier Trans/orm and Convolution Algorithms, Springer-Verlag: New York. Orfanidis, S. (1985). Optimal Signal Processing, McMillan: New York. Papoulis, A. (1984). Signal Analysis, McGraw-Hill: New York. Rudin, W. (1966) Real and Complex Analysis, McGraw-Hill: New York. Vetterli, M. and Kovacevic, J. (1995). Wavelets and Sub-Band Coding, PrenticeHall: Englewood Cliffs, NJ.

Part C

Fourier Analysis in L2

Introduction

The modem era of Fourier theory started when the tools of functional analysisin particular, Lebesgue's integral and Hilbert spaces-became available. Fourier theory then seemed to have reached the promised land, which is called L 2, the space of square-integrable complex functions, indeed a Hilbert space.

F. Riesz and E. Fischer were the first to study Fourier series in the L 2 framework. 1 Many ideas of the modem theory of Hilbert spaces were already contained in the work of these two mathematicians, and they had a c1ear view of the geometrie aspect ofthe L 2-spaces. They were inspired by aseries of articles by David Hilbert written after 1904 on the theme of integral equations and in which he gives the properties of 4(Z). Note, however, that the notion of abstract Hilbert spaces made its appearance much later than one usually believes, in the years 1927-1930, with the work of John von Neumann, who was motivated by quantum mechanics. 2

In short, a Hilbert space is a vector space H on the field t E T} if and only if X is the limit in H of a sequenee of finite linear eombinations of elements of {XI, t E T}. Continuity of the Hermitian product CI.4. Let H be a Hilbert space over C with the Hermitian product ( " .). The mappingfrom H x H into C defined by (x, y) t-+ (x, y) is bicontinuous.

THEOREM

Proof:

We have

I(x + h l , Y + h 2 ) - (x, y)1 = I(x, h 2 ) + (h l , y) + (h l , h 2 )1· By Sehwarz's inequality, l(x,h 2 )1 ::: l(h l ,h 2 )1::: Ilh 1 1l1lh 2 11. Therefore, lim

II h l II.IIh2 11-1-0

In partieular, the norm X

IIxllllh 2 11, l(hl,y)1 ::: lIyllllhlll, and

I(x + h l , Y + h 2 ) -

t-+

(X,

y)1 = O.

•

IIx 11 is a eontinuous funetion from H to lR+

EXERCISE C1.5. Let (X, X, fL) be a measure space, where fL is a finite measure. Let {fn}n ::: 1 be a sequence of LUfL) converging to f. Apply Theorem Cl.4 to prove that limntoo fL(fn) = fL(f). Give a counterexample ofthis property when the hypothesis that fL is finite is dropped. (Hint: f = 1[0, I] , fn = (1-1/ n) 1[0, 1] +- .. .) Show that when fL is finite,

G

= L~(fL) n {f; fL(f) = O}

is a Hilbert subspace of L~(fL).

Note that when fL is not finite, G need not be a Hilbert subspaee of L~(fL). Wavelet multiresolution analysis will provide a speetacular eounterexample. Isometry Extension Theorem DEFINITION CI.5. Let Hand K be two Hilbert spaces with Hermitian products denoted ( " .) Hand ( " .) K, respectively, and let q; : H t-+ K be a linear mapping such that,for all x, y E H,

(q;(x), q;(Y»)K = (x, y)H.

(8)

Then q; is called a linear isometry from H into K. If, moreover, q; isfrom H onto K, then Hand Kare said to be isomorphie.

Note that a linear isometry is neeessarily injeetive, sinee q;(x) = q;(y) implies q;(x - y) = 0, and therefore, 0=

1Iq;(x - y)IIK = IIx - yllH,

and this implies x = y. In particular, if the linear isometry is onto, it is neeessarily bijeetive.

138

Cl. Hilbert Spaces

Recall that a subset A E E, where (E, d) is a metric space, is said to be dense in E if, for all x E E, there exists a sequence {xn}ne:! in A converging to x. The following result will often be used. It is ca1led the isometry extension theorem of Hilbert spaces or, for short, the isometry extension theorem.

CI.5. Let Hand K be two Hilbert spaces with Hermitian products ( ., .) Hand ( ., .) K, respectively. Let V be a vector subspace of H that is dense in H, and let cp : V f-+ K be a linear isometry from V to K (cp is linear and (8) holds for all x, y E V). Then there exists a unique linear isometry (l : H f-+ K such that the restriction of (l to V is cp. THEOREM

Proof We sha11 first define (l(x) for x EH. Since V is dense in H, there exists a sequence {xn}ne:! in V converging to x. Since cp is isometric,

In particular, {cp(xn) }ne:! is a Cauchy sequence in K, and it therefore converges to some element of K, which we denote (l(x). The definition of (l(x) is independent ofthe sequence {xn}ne:! converging to x. Indeed, for another such sequence {Yn}ne:!'

The mapping (l : H f-+ K so constructed is c1early an extension of cp (for x E V, one can take as the approximating sequence of x the sequence {xn}ne:! such that Xn

== x).

The mapping (l is linear. Indeed, let x, Y E H, a, ß E C, and let {xn}ne:! and {Yn}ne:! be two sequences in V converging to x and y, respectively. Then {axn + ßYn}ne:! converges to ax + ßy. Therefore, lim cp(axn + ßYn)

ntoo

= (l(ax + ßy).

However,

= acp(xn) + ßCP(Yn) ---+ a(l(x) + ß(l(y)· Therefore, (l(ax + ßy) = a(l(x) + ß(l(y)· cp(axn + ßYn)

The mapping (l is isometric since, in view of the bicontinuity of the Hermitian product and of the isometricity of cp,

where {xn}ne:! and {Yn}ne:! are two sequences in V converging to x and y, respectively. •

Cl·3 Projection Theorem

Cl· 3

139

Projection Theorem

Let G be a Hilbert subspace of the Hilbert space H. The orthogonal complement of G in H, denoted G1., is defined by G1. = {z EH: (Z, x) = 0 for all x E G}.

(9)

Clearly, G1. is a vector space over Co Moreover, it is c10sed in H since if {Zn}n~l is a sequence of elements of G 1. converging to zEH, then, by continuity of the Hermitian product, 0= lim(Zn,x) = (Z,x)

forallx

ntoo

E

H.

Therefore, G1. is a Hilbert subspace of H. Observe that a decomposition x = Y + Z where Y E G and Z E G1. is necessarily unique. Indeed, let x = Y' + z' be another such decomposition. Then, letting a = Y - y', b = Z - z', we have that 0 = a + b where a E G and b E G1.. Therefore, in particular, 0 = (a, a) + (a, b) = (a, a), which implies that a = O. Similarly, b = O.

Cl.6 (Projection theorem). Let x G such that x - Y E G1.. Moreover,

THEOREM

Y

E

H. There exists a unique element

E

lIy -xII = inf lIu -xII·

(10)

UEG

Proof" that

Let d(x, G) = infzEG d(x, z) and let d(x,

{Yn}n~l

be a sequence in G such

Gi :s d(x, Yni 1 :s d(x, G)2 + -.

(*)

n

The parallelogram identity gives, for all m, n ::: I, IIYn - Ym 11 2 = 2(lIx - Yn 11 2

Since ~(Yn

+ Ym)

E

+ IIx -

Ym 11 2 )

-

411x - ~(Ym

+ Yn)1I 2 •

G,

therefore, IIYn - Ymll 2

:s 2 (~ + ~).

The sequence {Yn}n~l is thus a Cauchy sequence in G, and it consequently converges to some Y E G since G is c1osed. Passing to the limit in (*) gives (10).

Uniqueness of Y satisfying (10): Let y'

IIx - Y'II

= IIx -

E

G be another such element. Then

Yll

= d(x, G),

140

CL Hilbert Spaces

and from the parallelogram identity lIy - y'1I 2 = 211Y - xll 2 + 211y' - xll 2 - 411x _

= 4d(x, G)2 Since !(y

+ y')

411x _ !(y

!(y + Y')11 2

+ y')1I 2.

E G,

Therefore, lIy - y'1I 2 ~ 0,

which implies that lIy - Y'1I 2

= 0 and therefore, y = y'.

It now remains to show that x - y is orthogonal to G, that is,

forallZEG.

(x-y,z)=O

This is trivially true if z = 0, and we shall therefore assume y + AZ E G for all A E IR,

IIx -

(y

+ Az)1I 2

:::

z

=j::.

O. Because

d(x, Gi,

that is,

Since

wehave - 2A Re {(x - y, z)} which implies Re {(x - y, z)} (pure imaginary) leads to

=

+ A211z 11 2 ::: 0

.for allA

E

IR,

O. The same type of calculation with A E ilR

Im{(x - y, z)} =

o.

Therefore, (x - y, z) = O.

That y is the unique element of G such that y - x E G.L follows from the observation made just before the statement of Theorem C 1.6. • The element y in Theorem C 1.6 is called the orthogonal projection of x on G (see Fig. CU) and is denoted PG(x).

Projection Principle The projection theorem states, in particular, that for any x E G there is a unique decomposition

x = Y +z,

Y

E

G,

Z E

G.L,

(11)

C1·3 Projection Theorem

141

xeH

Figure C 1.1. Orthogonal projection

and that y = Pa(x), the (unique) element of G closest to x. Therefore, the orthogonal projection y = Pa (x) is characterized by the following two properties: (1) y e G; (2) (y - x, z) = 0 for all z e G.

This characterization is called the projection principle and is useful in determining projections. Projection Operator

The next result features two useful properties of the orthogonal projection operator Pa· 'THEOREM

CI.7. Let G be a Hilbert subspace ofthe Hilbert space H.

(a) The mapping x ~ Pa(x) is linear and continuous; furthermore,

IlPa(x)1I :::: IIxll

forall xe H.

(ß) If Fis a Hilbert subspace of H such that F S;; G, then PF particular, Pa Pa (Pa is then called idempotent).

=

Proof:

0

Pa

= PF. In

(a) Let Xl, X2 eH. They admit the decomposition Xj

where Wj e Gl. (i

= Pa(Xj) + Wj

(i

= 1,2),

= 1,2). Therefore, Xl

+ X2 = =

where W e Gl.. Now, Xl

+ Pa(X2) + WI + W2 Pa(xt} + Pa(X2) + W,

Pa(XI)

+ X2 admits a unique decomposition of the type

= y+w, where W e Gl., y e G: namely, y = Pa(XI + X2). Therefore, Pa(XI + X2) = Pa(XI) + Pa(X2). Xl

+X2

One similarly proves that Pa(ax)

= aPa(x)

for all a e G, X e H.

142

Cl. Hi1bert Spaces

Thus PG is linear. From Pythagoras' theorem applied to x = PG(x) + w, IIPG(x)1I

+ IIwII 2 = IIx1I 2 ,

and therefore,

Hence, PG is continuous. (ß) The unique decompositions of x on G and G.L and of PG(x) on F and F.L are x

= PG(X) + w,

From these two equalities we obtain (*)

But (z E G.L) =} (z E F.L) since F ~ G, and therefore v = Z + W E F.L. On the other hand, PF(PG(X» E F. Therefore, (*) is the unique decomposition of x on Fand F.L; in particular, PF(X) = PF(PG(X». • The next result says that the projection operator PG is "continuous" with respect to G. THEOREM CI.S. (i) Let {Gn }n:::l be a nondecreasing sequence ofHilbert subspaces ofH. ThentheclosureGofUn:::l Gn isaHilbertsubspaceofH and,for all X E H,

lim PGn(x) = PG(x).

ntoo

(ii) Let {G n } be a nonincreasing sequence of Hilbert subspaces of H. Then nn:::l G n = G is a Hilbert subspace of Hand, for all x E H,

lim PGn(x) = PG(x).

ntoo

Proof: (i) The set Un>l G n is evidently a vector subspace of H (in general, however, it is not closedflts closure, G, is a Hilbert subspace (Theorem C1.3). To any Y E Gone can associate a sequence {Yn}n:::h where Yn E G n, and lim IIY-Ynll=O.

n->oo

Take Y = Pa(x). By the parallelogram identity, IIPGn(x) - PG(x)1I 2

= lI(x -

PG(x» - (x - PG n(x»1I 2

= 211x -

PGn (x)1I 2 + 211x - PG(x)1I 2

- 411x - !(PGn(x)

+ PG(x»II·

C1·3 Projection Theorem

But since PGn(x)

143

+ PG(x) is a vectorin G, IIx - !(PGn(x)

+ PG(x))1I 2 ~

IIx - PG(x)1I 2,

and therefore, IIPGn(x) - PG(x)1I 2 S 211x - PGn (x)1I 2 - 211x - PG(x)1I 2

S 211x - Ynll 2 - 211x - PG(x)1I 2. By the continuity of the norm, lim IIx - Ynll 2 = IIx - PG(x)1I 2, ntoo and, finally, lim IIPGn(x) - PG(x)1I 2 = O. ntoo (ii) Devise a direct proof in the spirit of (i) or use the fact

Gl..

= clos

(U G;) .

•

n~1

EXERCISE

Cl.6. Prove the lollowing two assertions:

• Let {G n } be a nonincreasing sequence 01 Hilbert subspaces 01 H. Then nn~IGn = 0 ifand only iflimntoo PGn(x) = Olorall xE H . • Let {Gn}n~1 be a nondecreasing sequence 01 Hilbert subspaces 01 H. Then clOSUn~1 G n = H ifand only iflimNtoo PGn(x) = xlorall XE H. NOTATION.

IIG I and G 2 are orthogonal Hilbert subspaces olthe Hilbert space

H,

GI ffi G2 := {z = XI

+ X2

: XI E G, X2 E G2}

is called the orthogonal sum 01 GI and G2.

Riesz's Representation Theorem C1.6. Let H be a Hilbert space over C and let I : H t-+ C be a linear mapping; I is then called a (complex) linear form on H. It is said to be continuous if there exists A ~ 0 such that DEFlNITION

I/(xd - I(X2)1

s

AlixI - x211

lor all XI, X2

E

H.

(12)

The infimum olthe constants A satisfying (12) is called the norm 01 I. EXAMPLE

C1.3. Let Y

E

Hand define I: H t-+ C by I(x)

= (x, y).

(13)

144

Cl. Hilbert Spaces

It is a linear form and, by Schwarz's inequality, If(XI) - f(X2) I = If(XI -

x2)1

I(XI - X2, y)1 :'S lIyllllxl - x211· Therefore, fis continuous. Its norm is lIyll. To prove this it remains to show that if K is such that I(XI - X2, y)1 :'S K IIXI - x211foralt Xl, x2 E H, then lIylI :'S K. It suffi,ces to take Xl = X2 = Y above, which gives IIYll2 :'S K lIylI. =

We now state and prove Riesz's representation theorem. THEOREM C1.9.

Let f : H 1-+ C be a continuous linear form on the Hilbert space H. Then there exists a unique y E H such that (13) is true for alt X E H.

Proof"

Uniqueness.

Let y, y'

E

H be such that

f(x) = (x, y) = (x, y')

for an X EH.

In particular, (x, y - y')

The choice x

=y-

y' leads to

=0

for all x

E

H.

lIy - y'II 2= 0, that is, y - y',

Existence: Consider the kernel of f, N = {u EH: f (u) = O}. It is a Hilbert subspace of H. We may suppose that f is not identically zero (otherwise, if f == 0, take y = 0 in (13». In particular, N is strictly included in H. This implies that N1. does not reduces to the singleton {O} and, therefore, there exists zEN 1., z =f- O. Define y by y = f(z)* IIzzll 2

For an x

E

•

N, (x, y) = 0; therefore, in particular, (x, y) = f(x). Also, (z, y)

= (z,

f(z)*

11 zZIl 2 ) = f(z).

Therefore, the mappings x --+ f(x) and x --+ (x, y) coincide on the Hilbert subspace generated by N and z. But this subspace is Hitself. Indeed, for an xE

H,

x = (x -

where u

E

f(x) f(z)

N and w is colinear to z.

z) + f(x) z= u + w, f(z)

•

C2 Complete Orthonormal Systems

C2·1

Orthonormal Expansions

The result ofthis section is the pillar ofthe L 2-theory ofFourier series and wavelet expansions. It concems the possibility of decomposing a vector of a Hilbert space along an orthonormal base. The Gram-Schmidt Orthonormalization Procedure The central notion is that of an orthonormal system:

C2.I. The sequence {en}n~O in a Hilbert space H is called an orthonormal system of H if it satisfies the following two conditions:

DEFINITION

(cx) (en,ek) = Oforall n =j:.k;and (ß) lien 11 = Iforalln:::: O.

An orthonormal system {en}n~O isfree in the sense that an arbitrary finite subset of it is linearly independent. For example, taking (eI, ... , ek), the relation k

Lcxiei =0 i=1

implies that

t

CXiei )

=0

1:::

e ::: k.

C2.I. Let {fn}n~O be a sequence of vectors of a Hilbert space H. Construct {e n }n~O by the Gram-Schmidt orthonormalization procedure:

EXERCISE

P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

146

C2. Complete Orthonormal Systems

• Set p(O) = 0 and eo = 10111 10 11 (assuming 10 i= 0 without loss 01 generality); • Witheo, ... , en and p(n) defined, let p(n + 1) be thefirst index p > p(n) such that I p is independent 01 eo, ... , en, and define, with p = p(n + 1),

Ip-

n

L(fp, ej}ej

.,,-11-------;;-11· I j=!

en +! Show that

{en}n~O

=

p - "t(fp, ej}ej

is an orthonormal system.

Hilbert Basis The following theorem gives the preliminary results that we shall need for the proof of the Hilbert basis theorem. THEOREM C2.1. Let {en}n2:0 be an orthonormal system 01 Hand let G be the Hilbert subspace 01 H generated by {en}n~!. Then:

(a) For an arbitrary sequence {an }n~O 01complex numbers, the se ries Ln>o anen is convergent in H if and only if {an}n~! E e~, in which case (14)

(b) For alt x

E

H, Bessel's inequality holds:

L I(x, e }l2 .::; IIx1l n

2•

(15)

n~O

(c) For alt x

E

H, the series L (x, en}en converges, and n~O

L(x,en}en = PG(x),

(16)

n~O

where PG is the projection on G. (d) For alt x, y E H, the series Ln>! (x, en}(y, en ) is absolutely convergent, and L(x, en}(y, en}*

= (PG(X), PG(y)}·

(17)

n~O

Prool:

(a) From Pythagoras' theorem we have

and, therefore, {LJ=oajej}n~o is a Cauchy sequence in H if and only if {LJ=o la j 12}n~0 is a Cauchy sequence in IR. In other words, Ln~o anen con-

C2·1 Orthononnal Expansions

147

verges if and only if Ln>O lan 12 < 00. In this case equality (14) follows from the continuity of the norm, by letting n tend to 00 in the last display. (b) Accordingto (a) ofTheorem Cl.7, IIxll :::: 11 PGn (x)lI, where G n is theHilbert subspace spanned by {el, ... , en }. But n

PGn(x)

= ~)x, ei}ei, i=O

and by Pythagoras' theorem,

IlpGn (x)11

n

2

=

L I(x, ei}1

2.

i=O

Therefore, n

IIxll 2 ::::

L I(x, ei}1

2,

i=O

from which Bessel's inequality follows on letting n -+

00.

(c) From (15) and result (a), it follows that the series Ln>o (x, en}en converges. For any m :::: 0 and for all N :::: m,

and, therefore, by continuity of the Hermitian product, ( X - L(x, en}en, em) n;::O

=0

for all m :::: O.

This implies thatx - Ln>o(X, en}en is orthogonal to G. Also, Ln>o(X, en}en Therefore, by the projectlon principle, PG(x)

E

G.

= L(x, en}en· n;::O

(d) By Schwarz's inequality in .e~, for all N :::: 0,

(~llx, e")(y, e"),r " (~llx, e")I') (~IIM)I') ~ IIx1l 2 11Y1l2. Therefore, the series L:'o (x, en ) (y, en )* is absolutely convergent. Also, by an elementary computation,

148

C2. Compiete Orthononnal Systems

Leuing N -+ product). DEFINITION

00,

we obtain (17) (using (16) and the continuity of the Hermitian •

C2.2. The sequence

{Wn}n~O

oJvectors oJ His said to be total in H

ifit gene rates H.

In other words, the finite linear combination of the elements of {w n }n~O forms a dense subset of H.

C2.2. Prove that a sequence {wn}n~O oJthe Hilbert space H is total in H if and only if there is no element oJ H orthogonal to alt the Wn, n ::: 0, except 0, that is, if and only if EXERCISE

((Z, w n ) = 0 Jor alt n ::: 0)

==}

(z = 0).

(18)

We are now ready for the fundamental result: the Hilbert basis theorem. THEOREM C2.2. Let {en}n~O be an orthonormal system oJ H. The Jollowing properties are equivalent:

(a) {en}n~O is total in H; (b) Jor alt x E H, the Plancherel-Parseval identity holds true:

IIxII 2 =

L I(x, e }1 n

2;

(19)

n~O

(c)Jor alt x

E

H, (20)

x = L(x, en}en. n~O

Proof

(a)=}(c)

Accordingto(c)ofTheoremC2.1, L(x, en}en = PG(x), n~O

where G is the Hilbert subspace generated by {en}n>O. Since {en}n>O is total, it follows by (18) that Gl. = {O}, and therefore PG(x) :: x. (c)=::}(b) This follows from (a) ofTheorem C2.I. (b)=::}(a) From (14) and (16), L

I(x, en}1 2 = IIPG(x)1I 2 ,

n~O

and (19) therefore implies

From Pythagoras ' theorem,

IIxll 2 =

IIPG(x)+x - PG(x)1I 2

= IIPG(x)1I 2 + IIx = IIxll 2 + IIx -

PG(x)1I 2

PG(x)1I 2 ;

C2·1 Orthononnal Expansions

149

therefore,

IIx -

PG (x)1I 2 = 0,

which implies x

=

PG(x).

Since this is true for an x E H, we must have G H.

==

H, that is, {en}n;::O is total in •

A sequence {en }n;::O satisfying one (and then an) of the conditions of Theorem C2.2 is caned a (denumerable) Hilbert basis of H.

VI

C2.3. Let 1/f be a function in L~ (IR) with the FT = 2~ 1/, where I = [-2rr, -rr] U [+rr, +2rr]. Show that {1/fj,n}jEZ,nEZ is a Hilbert basis of L~(IR), where 1/fj,n(x) = 2 j / 2 1/f(2 j / 2 x - n). EXERCISE

EXERCISE

C2.4. Let {gj}j;::o be a Hilbert basis of L 2 «O, 1]). Show that {gA -

n)ICn,n+1l0}j;::O,nEZ is a Hilbert basis of L 2 (1R). (Here, L 2(I) Denotes the Hilbert

space (equivalence classes) of measurable complex-valued functions defined on I, with the Hermitian product (f, g) = f(t)g(t)*dt.)

1/

Biorthonormal Expansions DEFINITION

C2.3. Two sequences {en}n;::O and {dn}n;::O of aHilbert space H form

a biorthonormal system

if

(0:) (e n , d k ) = Oforall n (ß) (e n , dn ) = Ifor all n

=f. k, ~

O.

This system is ca lIed complete if, in addition, each ofthe sequences {en}n;::O and {dn}n;::oforms a total subset of H.

Then we have the biorthonormal expansions x = L(x, en}dn, n;::O

x = L(x, dn}en n;::O

whenever these series converge. Indeed, with the first series, for example, calling its sum y, we have for any integer m ~ 0,

= L(x, en}(dn , em } n;::O

=

(x, e m ).

Therefore, (x - y, em )

=0

for an m

Since {en}n;::O is total in H, this implies x - y = O.

~

O.

150

C2. Complete Orthononnal Systems

Separable Hilbert Spaces An interesting theoretical question is: For what type of Hilbert spaces is there a denumerable Hilbert basis? Here is a first (theoretical) answer. DEFINITION C2.4. A Hilbert space H is called a separable Hilbert space contains a sequence {fn}n:::O that is dense in H.

if it

THEOREM C2.3. A separable Hilbert space admits at least one denumerable Hilbert basis.

Proof Let {fn }n:::O be a sequence defined in Definition C2.4. Construct from it the orthonormal sequence {en}n:::O by the Gram-Schmidt orthonormalization procedure. It is a Hilbert basis because (a) of Theorem C2.2 is satisfied. Indeed, forany zEH,

«(en , Z)

= 0 for all n ~ 0) ==>

«(fp, z)

= 0 for alln ~ 0).

In particular, (y, z) = 0 for any finite linear combination of {fp}p:::o, Because {fp}p:::o is dense, (y, z) = 0 for all y E H. In particular, (z, z) = 0, that is,

z=Q

C2·2

•

Two Important Hilbert Bases

The Fourier Basis The following theorem is the fundamental result of the theory of Fourier series of finite-power periodie signals. THEOREM

C2.4. The sequence

=

{en ( • )} def {

1 2i Jr ../Te

!l. . } T

,

n

E

'7J fLj,

is a Hilbert basis of L~([O, T]). Proof One first observes that {enO, n E Z} is an orthonormal system in L~([O, T]). It remains to show that the linear space it generates is dense in L~([O, T]) (Theorem C2.2). For this, let f(t) E L~([O, T]) and let fN(t) be its projection on the Hilbert subspace generated by {enO, -N ::: n ::: N}. The coefficient of en in this projection is cn(f) = (f, en)L~([O.Tl)' we have

+N

{T

L Icn(f)1 + Jo n=-N 2

0

If(t) - fN(t)1 2 dt

(T

= Jo

0

If(t)1 2 dt.

(21)

C2·2 Two Important Hilbert Bases

151

(This is Pythagoras' theorem for projections: 11 PG(X) 11 2 + IIx - PG(X) 11 2 = IIx 11 2 .) In particular, LnEZ Icn (f)1 2 < 00. It remains to show ((b) of Theorem C2.2) that lim

Ntoo

[T

10

We assume in a first step that qJ(X)

If(t) - fN(t)1 2dt

= 0.

f is continuous. For such a function, the formula

=

l

T

Jex

+ t)Jet)* dt,

where J(t)

=L

f(t

+ nT)l(o.nCt + nT),

nEZ

defines a T -periodic and continuous function qJ. Its nth Fourier coefficient is Cn(qJ)

- + t)f(t)* - dt ) e- 2.IITTXn dx, = T1 10[T( fex 1

{T _

= T 10 =

f(t)*

{{T _

10

fex

+ t)e-

2.

- 2· T1 10{T f(t)* {ft+T f(s)et

n

IITTX

n IITTS

}

dx dt

}

ds e

2·

n IITTt

dt

Since LnEZ Icn (f)1 2 < 00 and qJ(X) is continuous, it follows from the Fourier inversion theorem for locally integrable periodic functions that, for all x E IR, qJ(x)

=L

Icn(f)1 2e 2i Jl"j-x.

nEZ

In particular, for x = 0,

and therefore, in view of (21), lim

Ntoo

(T

10

If(t) - fN(t) 12 dt

= 0.

It remains to pass from the continuous functions to the square-integrable functions. Since the space C([O, Tl) of continuous functions from [0, T] into C is dense in L~([O, Tl) (Theorem 27), with any 8 > 0, one can associate qJ E C([O, Tl) such that IIf - qJlI ::::: 813. By Bessel's inequality, IIfN - qJNII 2 = 1I(f - qJ)N1I 2 :::::

152 11

f -

C2. Complete Orthonormal Systems

cP 11 2 , and therefore,

IIf - fNIl S IIf - cplI + IIcp - CPNII + IIfN - CPNII S IIcp - CPNII S

+ 211f - cpll 8

IIcp - CPNII + 2 3,

For N sufficiently large, 11 cP - cP Nil S 8/3. Therefore, for N suffieiently large,

IIf-fNIIS8.

•

The Cardinal Sine Basis

The LI-version of the Shannon-Nyquist theorem of Seetion B2·l eontains a eondition bearing on the samples themselves, namely, (22)

The simplest way of removing this unaesthetie condition is given by the L 2 _ version of the Shannon-Nyquist theorem. THEOREM

C2.5. Let set) be a base-band (B) signaloffinite energy. Then

lim ( Iset) -

Ntoo

llR

bn sine (2Bt - n

n=-N

where bn

L+N

=

I

+B

j2 dt = 0,

(23)

•

s(v) e2irrvfB dv,

-B

Proof' Let L~(lR; B) be the Hilbert subspaee of LUlR) consisting of the finiteenergy eomplex signals with a Fourier transform having its support eontained in [ - B, +B]. The sequenee (24)

where h(t) == 2B sine (2Bt), is an orthonormal basis of L~(lR; B). Indeed, the functions of this system are in L~(lR; B), and they form an orthonormal system sinee, by the Planeherel-Parseval formula,

=

I

+B

-B

e2irrvkz-;

dv

= 2B

x ln=k.

C2·2 Two Important Hilbert Bases

153

It remains to prove the totality of the orthonormal system (24) (see Theorem C2.2). We must show that if g(t) E L~(IR.; B) and

L

2~) dt = 0

g(t)h(t -

for all n

E

Z,

(25)

then g(t) == 0 as a function of L~(IR.; B) (or, equivalently, that g(t) = 0 almost everywhere). Condition (25) is equivalent (by the Plancherel-Parseval identity) to [+B g(v)e 2irrv ii;- dv

LB

=0

for all n

E

Z.

(26)

But we have proven in the previous section that the system {e2irrvn/2B }nEZ is total in LUIR.; B); therefore, (26) implies g(v) = 0 almost everywhere, and consequently, g(t) = 0 almost everywhere. Expanding s(t)

E L~(IR.;

B) in the Hilbert basis (24) yields +N

s(t) = lim LCn Ntoo -N

1

n h(t - - ) , v2B 2B Mn

(27)

where the limit and the equality in (27) are taken in the L 2-sense (as in (23», and Cn

=

[s(t)

Jrw.

~h(t -

..!!...-)dt. 2B

v2B

By the Plancherel-Parseval identity, Cn

=

I

+B

-B

1

.

n

s(v) - - e 2l1tv2ij dv.

./fii

•

An Apparent Paradox

Note that since s( v) is in L 2 and of compact support, it is also in LI, and therefore the Fourier inversion formula is true and the reconstruction formula takes the farniliar form s(t) = Ls(..!!...-) sinc(2Bt - n). nEZ 2B

(28)

This is essentially true, but not quite. Indeed, imagine that someone tells you the following: Look, I have an proof that aL 2-signal s(t), base-band (B) is almost everywhere zero! Here is my cute proof. Of course, the reconstruction equality is in the sense of equality of L 2-functions, and in particular, it holds only for almost all t. Now, let me change the original signal to obtain a new signal s'(t) differing froms(t) only atthe times n/2B, where I set s'(n/2B) = O. I now apply the reconstruction formula and obtain that s'(t) is almost everywhere zero. But s' (t) and s(t) are almost everywhere equal. Therefore, s(t) is almost everywhere zero! Quod erat demonstrandum.

154

C2. Complete Orthonormal Systems

The flaw in the above "proof' is that the Fourier inversion formula holds only almost everywhere, and maybe not at the sampling times. Therefore, formula (28) is true only if the Fourier inversion formula can be applied at all the times of the form nj2B. This is the case if s(t) is continuous, because the inversion formula then holds everywhere. We see that the continuity hypothesis always pops up. We cannot expect a much better version of the sampling theorem in the LI or L 2 framework. Indeed, since s(v) is integrable, the right-hand side of s(t) =

L

s(v)e2invtdv

is continuous, and s(t) is therefore almost everywhere continuous. We have a sampling theorem for sinusoids and for decomposable signals (Theorem B2.5), and those signals are neither in LI nor in L 2 . Note, however, that they are continuous.

C3 Fourier Transforms of Finite Energy Signals

C3·1

Fourier Transform in L 2

A stable signal as simple as the rectangular pulse has a Fourier transform that is not integrable, and therefore one cannot use the Fourier inversion theorem for stable signals as it iso However, there is aversion of this inversion formula that applies to all finite-energy functions (for instance, the rectangular pulse). The analysis becomes slightly more involved, and we will have to use the framework ofHilbert spaces. This is largely compensated by the formal beauty of the results, due to the fact that a square-integrable function and its Fr play symmetrical roles.

The Isometrie Extension We start with a technical result. We use 1(.) to denote the function I : lR 1-+ C; in particular, I(a + .) is the function la : lR 1-+ C defined by la(t) = I(a + t). THEOREM

C3.1. Let set)

E

LUlR). The mapping Irom lR into L~(lR) defined by t -+ set

+ .)

is uniformly continuous. Proof:

We have to prove that the quantity L'S(t

+ h + u) -

set

+ u)1 2 du

= L1S(h

+ u) -

s(u)1 2 du

tends to 0 when h -+ o. When sO is continuous and compactly supported, the result follows by dominated convergence. The general case where sO E LUlR) P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

156

C3. Fourier Transforms of Finite-Energy Signals

is obtained by approximating sO in L~(I~) by continuous compact1y supported functions (see the proof ofTheorem A1.4). • From Schwarz's inequality, we deduce that t -+ (s(t

+ .), s( . ») L~(R)

is uniformly continuous on ~ and bounded by the energy of the signal. The above function is (29)

t -+ LS(t +x)s*(x)dx

and called the autocorrelation function of the finite-energy signal s(t). Note that it is the convolution s(t) * s(t), where s(t) = s( -t)*. THEOREM C3.2. lfthe complex signal s(t) lies in L~(~) n L~(~), then its FT s(v) belongs to L~(~) and

L Is(t)1 2 dt

=L

(30)

Is(v)1 2 dv.

Praof: The signal s(t) admits s(v)* as FT, and thus by the convolutionmultiplication rule, (31)

Consider the Gaussian density function h,y(t)

I

,2

= --e-2,;2. a,J2ii

Applying the result in (14) of Chapter Al, with (s observing that ha(t) is an even function, we obtain L Is(v)1 2 ha(v)dv

=L

(s

* s)(t) instead of s(t), and

* s)(x)ha(x)dx.

(32)

Since ha(v) = e- 2:rr 2a2x 2 t I when a .,j.. 0, the left-hand side of (32) tends to Is(v)1 2 dv, by dominated convergence.

IR

On the other hand, since the autocorrelation function (s * s)(t) is continuous and bounded, the quantity L (s

tends when a

.,j..

* s)(x)ha(x)dx =

L (s

* s)(ay)hj(y)dy

0 toward L (s *s)(O)hj(y)dy

by dominated convergence.

= (s *s)(O) = L

Is(t)1 2 dt,

•

C3·1 Fourier Transform in L 2

157

From the last theorem, we have that the mapping cp: s(t) -+ s(v) from LboR.) n L~(lR) into L~(lR) thus defined is isometrie and linear. Sinee LI n L 2 is dense

in L 2, this linear isometry ean be uniquely extended into a linear isometry from LUlR) into itself (Theorem C1.5). We will eontinue both to denote by s(v) the image of s(t) under this isometry and to eall it the FT of s(t). EXERCISE

C3.1. Show thatfor s(t) E L~(lR), lim ( 1s(v)

Ttoo

J~

-1

+T

2

s(t)e-2irrVldt 1 dv.

(33)

-T

The above isometry is expressed by the Plancherel-Parseval identity: THEOREM

C3.3. If SI (t) and S2(t) are finite-energy, complex signals, then

L

SI(t)S2(t)* dt =

EXERCISE

L

(34)

SI(V)S2(V)* dv.

C3.2. Show that dv = Im ( Sin(JTv))2 JTV

1.

~

THEOREM

C3.4. If h(t) E Lb(lR) and x(t) E L~(lR), then y(t)

=

L

(35)

h(t - s)x(s) ds

is almost everywhere weil defined and in LUlR). Furthermore, its FT is y(v) = h(v)x(v).

(36)

Proof Letus first show that f~ h(t-s)x(s) ds is welldefined. Forthis weobserve .. that on the one hand L1h(t - s)llx(s)1 ds :::: L1h(t - s)l(l

+

Ix(s)1 2)ds

= L1h(t)1 dt + L1h(t -

s)llx(s)1 2 ds,

and on the other, for almost all t,

L1h(t - s)ll(x(s)1 2 ds <

00,

sinee Ih(t)1 and Ix(t)1 2 are in Lb(lR). Therefore, for almost all t,

L1h(t - sllx(s)1 ds <

00,

and y(t) is almost everywhere well defined. We now show that y(t)

E

LUlR).

158

C3. Fourier Transforrns of Finite-Energy Signals

Using Fubini's theorem and Schwarz's inequality, we have

LIL

h(t - S)X(S) dS

= =

LIL LL{L

2 I

dt

h(u)x(t - U)dUr dt

x(t - u)x(t - v)* dt} h(u)h(v)* du dv

~ (L Ix(s)1 dS) (L Ih(u)1 du Y< 2

00.

For future reference, we rewrite this as IIh

* XIlL~(lR) ~

(37)

IIhIlLt(lR)lIxIlL~(lR)'

The signal (35) is thus in L~(IR) when h(t) E LUIR) and x(t) furthermore, x(t) is in Lb(IR), then y(t) is in Lb(IR). Therefore, x(t) E Lb(lR)

n L~(IR) -+

E LUIR).

y(t) E Lb(lR) n L~(IR).

If,

(38)

In this case, we obtain (36) by the convolution-multiplication formula in L I . We now suppose thatx(t) is in L~(IR) (butnot necessarily in Lb(IR». The !signal XA(t) = x(t)l[-A,+Aj(t)

isinLb(lR)nL~(IR)andlimxA(t)

in L~(IR). Introducing YA(t)

= x(t)inL~(IR).Inparticular,limxA(v) = x(v)

=

L

h(t - s)xA(s)ds,

= h(V)XA(V), Also, lim YA(t) = y(t) in L~(IR) [use (37)], and thus limYA(v) = y(v) in L~(IR). Now, since limxA(v) = x(v) in L~(IR) and h(v) is • bounded, lim h(V)XA(V) = h(v)x(v) in L~(IR). Therefore, we have (36). we have YA(V)

EXERCISE

C3.3. Use the Plancherel-Parseval identity to prove that {dt

llR (t 2 + a 2 )(t 2 + b2 ) EXERCISE

7t:

ab(a

+ b)'

C3.4. Show that H

= {w(t) E L~(IR);tw(t) E L~(IR), vw(v) E L~(IR)}

is a Hilbert space when endowed with the norm IIwllH = (lIwll~2

1

+ IItwll~2 + IIvwll~2P .

Show that the subset 0/ H consisting 0/ the C oo -functions with compact support is dense in H. Hint: Select any q; in a Coo-function with compact support, with

C3·2 Inversion Formula in L 2

159

integral equal to 1, and equal to 1 in a neighborhood ofO, andfor any wEH, consider the function w(t)cp(t / n) ncp(nt).

*

C3·2

Inversion Fonnula in L 2

So far, we know that the mapping cp : L~(~) 1-+ L~(~) defined in Seetion C3·1 is linear, isometrie, and into. We shall now show that it is onto, and therefore bijeetive. THEOREM

C3.5. LetS(v) be the FT of s(t)

E L~(~).

Then

cp : s( -v) --+ s(t), that is, s(t)

= lim

Atoo

j

+A

-A

s(v)e 2i Jl'vt dv,

(39)

(40)

where the limit is in LU~), and the equality is almost everywhere.

We shall prepare the way for the proof with the following result. LEMMA

C3.1. Let u(t) and v(t) be two jinite-energy signals. Then

L

u(x)v(x)d.x

=

L

u(x)v(x)d.x.

(41)

Proof' If (41) is true for u(t), v(t) E L~(~) n L~(~), then it also holds for u(t), v(t) E L~(~). Indeed, denoting XA(t) = x(t)I[-A,+Aj(t), we have

L

u A(x )(v;)(x) d.x

=

L

(l7;)(x)v A(x) d.x,

that is, (UA, VA) = (UA, VA)' Now UA, VA, UA, and VA tend in LU~) to u, V, u, and V, respeetively, as A t 00, and therefore, by the eontinuity of the Hermitian produet, (u, v) = (u, v). The proof of (41) for stable signals is aeeomplished by Fubini's theorem:

L

u(x)v(x)d.x

L {L = L {L =L =

u(x)

v(y)e- 2i Jl'xy d Y } d.x

v(y)

u(x)e- 2i JI'XY d Y } dy

v(y)u(y)dy.

•

160

C3. Fourier Transforms ofFinite-Energy Signals

Proof of(39); Let g(t) be a real signal in L~(lR), and define f(t) g-(t) = g(-t). We have j(v) = g(v)*. Therefore, by (41):

L

g(x)j(x)dx =

=

= (g~)(t), where

L

g(x)f(x)dx

f

§(x)§(x)* dx.

Therefore, IIg -

But IIgll 2

f112

j) + IIfII 2

= IIgll 2 -

2Re (g,

= IIgll 2 -

211g11 2 + IIfII 2 .

= IIgll 2 and IIfll 2 = IIg1l 2 . Therefore, IIg - f112 = 0, thatis, g(t) = j(t).

(42)

In other words, every real (and, therefore, every complex) signal g(t) E L~(lR) is the Fourier transform of some function of LUIR). Hence, the mapping q; is onto. • C3.5. Show that if a stable signal is base-band (that is, compact support), then it also has afinite energy.

EXERCISE

if its FT has

We dose this seetion by showing how the LI Fourier inversion theorem was lirnited in scope, since it does not take much for a stable signal not to have an integrable FT. C3.6. Show that if a stable signal is discontinuous at a point t FT is not integrable.

EXERCISE

= a, its

C4 Fourier Series of Finite Power Periodic Signals

C4·1

Fourier Series in Lfoc

Let us consider the Hilbert space e~ of complex sequences a that LnEZ lan l2 < 00, with the Hermitian product (a, b}e~

= {an}, n

E

Z, such

= L>nb~

f:

and the Hilbert space L~([O, T], dt/T) of complex signals x = {x(t)}, t such that Ix(t)1 2 dt < 00, with the Hermitian product (x, Y}L~([O,T],~) = THEOREM

C4.1. Formula Sn

= -1 T

l

0

(43)

nEZ

dt Jorx(t)y(t)* T' T

T

n dt s(t)e- 2"17r'i/

E

lR.,

(44)

(45)

defines a linear isometry sO --+ {Sn} lram L~([O, T], dt / T) onto e~, the inverse

01 which is given by

s(t)

= L::Sne2i1l"f/ ,

(46)

nEZ where the se ries on the right-hand side converges in L~([O, T], dt / T), and the equality is almost everywhere. This isometry is summarized by the PlancherelP. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

162

C4. Fourier Series of Finite-Power Periodic Signals

Parseval identity:

LXnY~ = ~

nEZ

(47)

(Tx(t)y(t)* dt. 10

The result follows from general results on orthonormal bases of Hilbert spaces, since the sequence

Prao!"

{e n (-)}

~ {Jre 2i1t T'}'

nE Z,

is a complete orthonormal sequence of L~([O, Tl. dt / T) (Theorem C2.4).

•

The L 2 inversion theorem tells us that if s (t) is a T -periodic complex signal with finite power, then +N

L

L~([O, Tl)

Sn e2i1t (n/T)t ------+ set).

-N

Ntoo

In general, for an arbitrary sequence of functions of L~([O, Tl), convergence in L~([O, Tl) does not imply convergence almost everywhere. However, for sequences of partial Fourier series, we have the surprising Carleson's theorem: THEOREM C4.2. The Fourier se ries 01 a T -periodie signal s(t) with finite power converges almost everywhere to set).

This result shows that the situation for finite-power periodic signals is pleasant, in contrast with the situation prevailing locally stable periodic signals (remember Kolmogorov's result, Theorem A3.1). The proof ofCarleson's result is omitted; it is rather technical. It also shows that the L 2 framework is very adapted to Fourier series, since everything works "as expected." Discrete-Time Fourier Transform of Finite-Energy Signals Let lb be the space of sequences In, n discrete-time signals). THEOREM

C4.3. lb

c

E

Z, such that LnEZ Ilnl <

00

(stable

l~, that is, a discrete-time stahle signal has finite energy.

Prao!" Let A = {n: IX n I ~ I}. Since LnEZ IX n I < 00, then necessarily card(A) < 00. On the other hand, if Ixnl ::::: 1, Ix n l2 ::::: Ixnl, whence

• The situation for discrete-time signals is therefore in contrast with that of continuous-time signals, for which there exist stable signals with infinite energy that and finite-energy signals that are not stable.

C4·2 Orthonormal Systems of Shifted Functions

163

Let L~(2n) be the Hilbert space of functions j: [-n, +n] ~ C such that f~: Ij(w)1 2 dw < 00 provided with the Hennitian product (],

C4.4.

THEOREM

g)L~(2n) =

l+

_1 2n

-n

n

j(w)g(w)* dw.

There exists a linear isomorphism between LU2n) and .e~

dejined by fn =

l+

n

-n

j(w)e inw dw , 2n

j(w)

=L

fn e- inw .

(48)

nEZ

In particular, we have the Plancherel-Parseval identity

L fng: = -2n1 l+

n

f(w)g(w)* dw.

(49)

-n

nEZ

•

This is arestatement of Theorem C4.3.

Proof:

C4· 2

Orthonormal Systems of Shifted Functions

We give a necessary and sufficient condition in the frequency domain for a system of shifted functions to be orthonormal. THEOREM C4.5. Let g(t) be afunction of L~(lR.) andjix 0 < T < and sufficient condition for the family offunctions

00.

{g(. - nT)}nEZ

A necessary (50)

to fonn an orthononnal system of L~(lR.) is

L

Ig(v

+

nEZ

-f )1

2

= T

almosteverywhere.

(51)

Proof: The Fourier transform g( v) of g(t) E L~(lR.) is in L~(lR.) and, in particular, Ig(v)1 2 is integrable. By Theorem A2.3, LnEZ Ig(v + (n/T»1 2 is (I/T)-periodic and locally integrable, and T In~. g(t) g(t - nT)* dt is its Fourier coefficient (this follows from the Plancherel-Parseval formula: T

L

g(t)g(t - nT)* dt

=T

L

Ig(v)12e-2invnT dv

The definition of orthonormality of system (50) is

L

g(t) g(t - nT)* dt

= I n=o.

The proof then follows the argument in the proof of Theorem B2.6.

•

164

C4. Fourier Series of Finite-Power Periodic Signals

Riesz's Basis The following notion will play an important role in multiresolution analysis. DEFINITION

C4.1. A system offunctions of LU~) {w(· - nT)}nEz

(52)

is said to form a Riesz basis of some Hilbert subspace Vo of LU~) (a) it spans Vo, and (b)forall sequence, {ckhEZ ofe~(Z),

AL 1ck1 2 ::s k

where 0 < A ::s B <

00

(IL

Jffi.

kT)1

CkW(t -

2 dt

::s B L

kEZ

if

Icd,

(53)

kEZ

are independent ofthe Ck.

Thefunction LkEZ Ck w(t-kT) has theFouriertransform LkEZ cke-2inkTvw(v), and, therefore, by the Plancherel-Parseval identity, the term between the bounds in (53) is equal to

(IL

Jffi.

Cke-2itrkTVW(V)12 dv

kEZ

Also,

Now, any function c(v) E L~([O, I/T]) has the form LkEZcke-2itrkTv, where LkEZ ICkl 2 < 00, and (53)is therefore equivalent to AT

1

1fT

Ic(v)1 2 dv::s

1 IC(V)121~ 1fT

t

::s BT Jo

fT

Iw(v

Ic(v)1 2dv

+

f )1 1 2

dv

C4·2 Orthononnal Systems of Shifted Functions

for any c(v)

E L~([O,

165

IjT]). It then foIlows that

ATS~lw(v+ ~)12 SBT

a.e.

(54)

C4.6. Let (w(· - n T) }neZ be a Riesz basis of some Hilbert subspace Vo C L~(R). Define the function g E L~(R) by its Fourier transform

THEOREM

(55)

Then (g(. - nT)}nez is a Hilbert basis ofVo. In view of (54), the function gis weIl defined and in L~(R). Since (51) is obviously satisfied, it foIlows that the system {g(. - nT)}neZ is orthonormal.

Proof"

We must now show that the Hilbert space Vo spanned by (g(. - nT)}neZ is in fact identical to Vo.1t suffices to show that the generators of Vo belong to \10, and vice versa. Define

In view of condition (54), a( v) is (1 j T)-periodic and offinite power, and it therefore admits a Fourier representation

a(v) = L ane-2i7CvnT, neZ

for some sequence {an}

E l~(Z).

Since

w(v) = a(v)g(v),

it foIlows that w(t) = L ang(t - nT). neZ

Therefore, the generators of Vo are in Vo. The converse is true by the same argument since

g(v) = a(v)-Iw(v), where a(vr l is also, in view of condition (54), a (ljT)-periodic of finite power. •

C4.1. Let h(t) be the function of FT (ljJ2B)I[-B.+Bj(v) for some B > O. Show that there exists no orthonormal basis of L~(R; B) of the form

EXERCISE

{k g (t -

2~) }nd where g(t) = LCkh (t keZ

~), 2B

166

C4. Fourier Series of Finite-Power Periodic Signals

where {cn} nEZ is in CU7-), unless only one of the Cn is nonzero. Show that if the Fourier sum c(w) ofthe sequence {cn}nEZ is such that A < !c(w)!2 < B for some

o<

A

:s

B <

L~(lR; B).

00 and for all w,

then

{k g (t -

2~)}

nEZ

is a Riesz basis of

References [Cl] [C2] [C3] [C4] [C5]

Daubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf. Series in Applied Mathematics, SIAM: Philadelphia, PA. Gasquet, C. and Witomski, P. (1991). Analyse de Fourier et Applications, Masson: Paris. Halmos, P.R. (1951). Introduction to Hilbert space, Che1sea, New York. Rudin, W. (1966). Real and Complex Analysis, McGraw-Hill, New York. Young, N.Y. (1988). An Introduction to Hilbert Spaces, Cambridge University Press.

Part

D

Wavelet Analysis

Introduction

Although Fourier theory had reached in the L 2 framework a formal mathematical beauty, it was not entirely satisfactory for important applications in signal processing. Indeed, in many situations, the information contained in a signal is localized in time and/or in frequency. The typical example is a piece of music, which is perceived as a succession of notes welliocalized in both time and frequency. The usual Fourier transform is not adapted to the analysis of music because for a given frequency (a note) it is related to the total energy of all occurrences of this note in the entire piece. This led Dennis Gabor 1 to propose a windowed Fourier transform, whose idea is very natural. If f(t) is the signal to be analyzed, the local information at time t = bis contained in the time-Iocalized signal f(t)w*(t - b),

where w(t) is the time window function, a function negligible outside a relatively small interval around zero. Given a window w(t), the local information at time b is obtained by computing the Fourier transform of the last display: Wf(v, b):=

L

f(t)w*(t - b)e-2invt dt

=

(j, Wv,b),

where Wv,b(t)

= w(t -

b)e2invt.

ITheory ofCommunications, J. Inst. Elec. Engrg., Vol. 93, pp. 429-457,1946.

170

Part D Wavelet Analysis

We see that the time information is collected in a time interval around time b of width of the order of that of the time window. Now, from the Plancherel-Parseval identity,

Therefore, we see that the frequency information is collected in a frequency interval around v of the width of the time window's Fr. These observations point to a fundamental limitation of the windowed Fourier transform, in relation with the uncertainty principle, which states that time resolution is possible only at the expense of frequency resolution, and vice versa. Indeed, the wider the function, the narrower its Fourier transform, and vice versa. Of course, this is an imprecise statement, but it is already substantiated by the Doppler theorem

(Note that as a varies, the energy of lai! !(at) remains the same.) The so-called Heisenberg uncertainty principle makes the above limitation more explicit and states that

where a f is the mean-square width of a function ! E L~(lR) (see the definition in the first lines of Section D 1·1). The conclusion is that as long as we resort to windowed Fourier transforms, time resolution and frequency resolution are antagonistic: If one is increased, the other is necessarily decreased, keeping the area awafij ofthe time-frequency box above the lower bound of 1/41T. However, the real inconvenience of the windowed Fr is the fixed shape of the time-frequency box. In many occasions, it is interesting to have a time-frequency box that adapts itself to the time-frequency point analyzed. For instance, a discontinuity (abrupt change) of a signal takes place in a short time and involves high frequencies. Therefore, at high frequencies, the time dimension of the time-frequency box should be small. Also, since it takes time to determine the frequency of a low-frequency sinusoid, at low frequencies the time dimension of the time-frequency box should be large. Motivated by these imperatives, lean Morlet2 proposed the wavelet transform in order to take into account the need for an adaptive time-frequency box.

2 Sampling theory and wave propagation, in NATO ASI series, Vol. 1, Survey in Acoustic Signal! Image Processing and Recognition, C.H. ehen, ed., Springer-Verlag, Berlin, 233-

261,1983.

Introduction

171

In the wavelet transfonn, the role the family of functions Wv,b(t) plays in the windowed Fourier transfonn is played by a family

(t - b)

1fta,b(t) = lai -1/2 1ft -a-

a, b

,

E

IR, a

=I- 0,

where 1ft(t) is called the mother wavelet. The wavelet transfonn (WT) of the function f E L~(IR) is the function C j(a, b) = (j, 1fta,b) =

L

f(t)1ft;,b(t) dt,

By the Plancherel-Parseval identity, C j(a, b)

= (j, :(fra,b) =

a, b

E

IR, a

=I- 0.

L

!(V):(fr;,b(V) dv,

where

:(fra,b(V) = laI 1/ 2 e- 2i 1l'Vb:(fr(av). From the above expression, it appears that the wavelet transfonn C j (a, b) analyzes the function f(t) in the time-frequency box

[b

+ am -

au, b + am

8- m 8-] + au] x [-m - -, - + -

m m/

a

a

a

a

(where m is the center of 1ft, u is its width; and 8- are the corresponding objects relatively to :(fr; see the details in the main text). Assuming > 0, we see that the frequency window is centered at v = a and has width 28- / a; therefore, the ratio center/width

m

m

~

Q=

28-

is independent of the frequency variable a. The area of the time-frequency box is constant, but its shape varies with the frequency v = a analyzed. For high frequencies it has a large time dimension, and for small frequencies it has a small time dimension, which is the desired effect.

m/

The interest of the wavelet transfonn for signal analysts is that they can "read" it to extract infonnation about the time-frequency structure that is otherwise blurred in the brute signal by concurrent phenomena and subsidiary effects. For instance, they can detect the appearance time of a phenomenon linked to a particular frequency (e.g., the time at which a particular atom starts to be excited). A fonnula, called the identity resolution, allows us to reconstruct, under mild conditions that we shall make precise in due time, the function from its wavelet transfonn:

f(t)

= ~ { ( C j(a, b)1fta,b(t) dll ~b , K JlRJlR a

where K is some constant depending on 1{1. In this sense, wavelet analysis is continuous, in that the original function of L 2 is reconstructed as a continuous linear

172

Part D Wavelet Analysis

combination ofthe continuous wavelet basis 1J!a,b(t) = lal-l/21J!e~b) , a, bE lR., a =1= O. One would rather store the original function not as a function of two arguments, but as the doubly indexed sequence of coefficients of a decomposition along an orthonormal base of L 2 , {1J!j,nb,nEZ, called the wavelet basis,

f

=

L LU, 1J!j,k)1J!j,k. JEZ kEZ

where j

.

1J!j,n(t) = 2 2 1J!(2J t - n), and where 1J!(t) is the mother wavelet. The multiresolution analysis of Stephane Mallat3 is one particular way of obtaining such orthonormal bases and is the main topic of Part D. Similarly to the continuous wavelet transform, the coefficients of the multiresolution decomposition can be used to analyze a signal. But multiresolution analysis is also a tool for data compression. Indeed, with a good design of the mother wavelet, many wavelet coefficients are small and can be neglected. The coefficients that are not neglected are quantized more or less coarsely, depending, for instance, on the frequency index. This is of course reminiscent of subband coding, and this resemblance is not at all a coincidence. Mallat's algorithm of analysis-synthesis is of the same form as the subband analysis-synthesis algorithm. Multiresolution analysis can be considered as a systematic way of doing subband analysis. One of its advantages is to place the latter in a framework where the mathematical issues ensuring the efficiency of the algorithms are more easily dealt with. Perhaps one of the most striking advantages of multiresolution analysis over classical Fourier analysis is in the way it handles discontinuities. Consider, for instance, the signal on top of Figs. DO.la and DO.lb,4 which has a spike. In both figures the middle signal is a Fourier series approximation of the top signal, whereas the bottom signal is a wavelet approximation of the top signal. In Fig. DO.I a, only the first 60 coefficients of the expansions (Fourier or wavelet) are used to produce the approximated signals, and it appears that there is not a dramatic difference between the two approximations. This is not the case, however, when, as in Fig. DO.I b, one uses the 60 largest coefficients. The advantage of the wavelet approximation is then obvious.

3Multiresolution approximation and wavelets, Trans. Amer. Math. Soc., 315, 1989,6988. 4Reproduced with the kind permission of Martin Vetterli.

Introduction

173

0.5

of---------' o

100

200

300

400

500

600

700

800

900

1000

100

200

300

400

500

600

700

800

900

1000

200

300

400

500

600

700

800

900

1000

0.5

o o

0.5 01------"--"--"'\

o

100

(a)

0.5 01------'

o

100

200

300

400

500

600

700

800

900

1000

300

400

500

600

700

800

900

1000

300

400

500

600

700

800

900

1000

0.5

o o

0.5 01------'

o

100

200

(b)

Figure DO.I. Wavelet VS. Fourier

Dl The Windowed Fourier Transform

D 1·1

The Uncertainty Princip1e

Fourier analysis, weH as wavelet analysis have an intrinsic limitation, which is contained in the uncertainty principle. In order to state this result, we need a definition of the "width" of a function. Here is the one that suits our purpose. Root Mean-Square Widths

Let w : lR 1-+ C be a nontrivial function in L 2 • Define the centers of w and W, respectively, by mw

= -1

mfjj

= _1_

1

Ew IR

Ew

t Iw(t)1 2 dt,

r

JIR

v Iw(v)1 2 dv,

where E w is the energy of w(t): Ew =

L

Iw(t)1 2 dt =

L

Then define

and

P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

Iw(v)1 2 dv.

176

D 1. The Windowed Fourier Transforrn

w,

The numbers o"w and 0"1ii are the root mean-square (RMS) widths of w and respectively. Note that m w and mlii are not always defined. When they are weIl defined, o"w and 0"1ii are always defined but may be infinite. Therefore, we shall always assurne that

and

L

Ivllw(v)1 2 dv

<

00,

to guarantee at least the existence of the centers of w(t) and of its Fr. EXERCISE D1.I. Check if the centers m w and mlii are well defined, and then compute o"w and 0"1ii and the product O"wO"Iii in thefollowing cases:

w(t) w(t)

= l[-T,+Tl = e-a1tl ,

a > 0,

w(t) = e- at2 ,

a > O.

Suppose that the centerm w ofthefunction w Show that the quantity

EXERCISEDI.2.

E

L 2 iswell defined.

L

It - toI 2 Iw(t)1 2 dt

is minimized by to = m w . Heisenberg's Inequality THEOREM

DI.I.

Under the conditions stated above, we have Heisenberg's

inequality (1)

Proof: We assurne that the window and its Fr are centered at 0, without loss of generality (see Exercise D1.3). Denoting the L 2 -norm of a function f by IIfII, we have to show that ~

1

2

IItwll x IIvwll :::: 4n IIwll ' We first assurne that w is a COO-function with bounded support. In particular, td(v)

and therefore,

= (2inv) w(v),

(2)

D 1·1 The Uncertainty Principle

177

Thus, it remains to show that 1

x IIw'lI ::: "2 IIw1l 2 •

IItwll

(3)

By Schwarz's inequality,

x IIw'lI :::

IItwll

I(tw,

w')1 ::: IRe{(tw, w')}I.

Now, 2Re{(tw, w')}

= (tw, w') + (w', tw) = l =

t(ww'*

+ w'w*)dt

tlw(t)121~~ -lIW(t) 12 dt = 0 -

IIw1l 2 •

This gives (3) in the case where w E Coo . We now show that (3) is true in the general case. To see this, we first observe that it suffices to prove (3) in the case where w belongs to the Hilbert space H = {w

E

L~(lR); tw(t) E L~(lR), vw(v) E L~(lR)}, 1

with the norm 11 w 11 H = (11 W11 2 + IItw 11 2 + 11 vwll2) 2: (if w(t) is not in this space, Heisenberg's inequality is trivially satisfied). Then we use the fact that the subset of H consisting ofthe Coo -functions with compact support is dense in H (see Exercise • C3.4). The result then follows from the continuity ofthe Hermitian product. Equality in (1) takes place if and only if w(t) is proportional to a Gaussian signal e- ct2 , where c > O. We do the proof only in the case where it is further assumed that WES and is real. Observe that all the steps in the first part of the proof remain valid since for such a function, tlw(t)121~~ = O. Equality is attained in (1) if and only if the functions tw(t) and w'(t) are proportional, say, w'(t)

=-

ctw(t),

and this gives w(t) = Ae-ct2 ,

where c > 0 necessarily, because w(t) EXERCISE

E

L2.

D1.3. Show that, for arbitrary to E lR, Vo E lR, lI(t - to)wll

x

(Hint: Consider the function g(t)

~ 1 lI(v - vo)wll ::: 4n

2

IIwll .

(4)

= e-2invot w(t + to).)

The above resulttells us that in Heisenberg's inequality (1), the numbers a w and can be taken to be, respectively, the root mean-square width of w around any time to and the root mean-square width of w around any frequency vo. In particular, to can be the center of w(t), and Vo can be the center of w(v). This version is the most stringent, since the RMS widths around the centers are the smallest RMS widths (see Exercise D1.2). aliJ

178

D 1. The Windowed Fourier Transforrn

D 1· 2

The WFf and Gabor' s Inversion Formula

Windows

Let f(t) be the signal to be analyzed. The local information at (around) time t = b is contained in the time-Iocalized signal f(t)w*(t - b),

(5)

where w(t) is the time window function, the support of which is included in a relatively small interval about zero. Typical examples are the rectangular window w(t) = l[-a,+aj(t)

and the Gaussian window

wherea > O. Given a window w(t), the local information at time b is obtained by computing the Fourier transform of (5): Wf(v, b):=

L

f(t)w*(t - b)e-2irrvt dt.

(6)

The choice of w* instead of w in (5) is purely for notational cornfort. For example, Wf(V, b)

= (j, Wv,b),

(7)

the Hermitian product in L~(lR) of f(t) with Wv,b(t) = w(t - b)e2irrvt,

(8)

where we assume, of course, that fand w are complex-valued functions that have finite energy. L 2. One caUs the function Wf : lR x lR -+ C defined by (6) the windowed Fourier transform (or WFT) of f associated with the window w. DEFINITION

DI.I. Let w, f

E

When the rectangular window is chosen, W f is called the short-time Fourier transform of f. If the window is Gaussian, the function Wf is called the Gabor transform of f. Inversion Formula THEOREM

DI.2. Under the foUowing assumptions,

(a) w E Lb(lR)

(b)

L

n LUlR),

Iw(t)1 2 dt

= 1,

D 1· 2 The WFf and Gabor' s Inversion Formu1a

179

(c) Iwl is an evenfunction, we have, lor all I

E L~(lR),

the energy conservation formula

LLIWf (v,b)1 2 dvdb = LI/(t) 12 dt,

(9)

and the reconstruction formula, (inversion formula), lim ( I/(t)

Atoo

JR.

-1 JR.(

= O.

Wf(V, b)Wv,b(t) dv dbl2 dt

Ivl:",A

(10)

Note that assumption (c) is automatically satisfied if w(t) is areal function. Assumption (b) is just a convention: The window can always be normalized to have an energy equal to 1, and the wavelet transform is then just multiplied by a constant.

Proof

The proof is technical and can be skipped in a first reading. We define IA(t) =

1 JR.(

Wf(v, b)Wv,b(t) dv db.

Ivl:",A

Using the Plancherel-Parseval identity, (7) becomes Wf(v, b)

= (I, Wv,b),

where Wv,b(f.1,)

= e-2irr (/l--v)b W(Jl_ v)

(11)

is the Fourier transform of (8). Therefore, Wf(v, b) = e-2irrvb L I(Jl)w(Jl- v)*e 2irr /l- b dJl.

(12)

The function Jl -+ I(Jl)w(Jl - v)* is in LI n L 2. (It is in LI as the product of two L 2-functions; it is in L 2 because E L 2 and Wis bounded, being the Fourier transform of an L I-function.) By the Plancherel-Parseval identity,

!

L IWf(v, b)1 2 db = L

=L

IL

I(Jl)w(Jl- v)*e 2irr /l- b dJl

l

2

db

1!(Jl)w(Jl - v)*1 2 dJl,

and, therefore, L L IWf(v, b)1 2 dbdv = L L II(Jl)1 2Iw(Jl- v)1 2 dJldv

=L

{1!(Jl)1 2 L IW(Jl - v)1 2 dV} dJl

=L

{11(Jl)1 2 L Iw(v)1 2 dV} dJl.

180

D 1. The Windowed Fourier Transform

Equality (9) foHows because

L

L

Iw(t)1 2 dt = 1,

Iw(v)1 2 dv =

L

Ij(JL)1 2 dJL

=

L

IJ(t)1 2 dt (Plancherel-Parsevalidentity).

We show that the function JA is weH defined, that is, (v, b) -+ Wj(v, b)Wv,b(t) is integrable over [- A, + A] x R In view of (12), IA(t) := j+A { IWj(v, b)llwv,b(t)1 dv db

-A JIR

By Schwarz's inequality and the Plancherel-Parseval identity, and using assumption (b),

L

1.r-I{j(.)W(. - V)*}(b)IIW(t - b)1 db

:s ( =

=

=

L

2 )1/2 (

l.r-I{j(.)w(. - v)*}1 db

(L

L

Iw(t - b)1 2 db

)1/2

l.r-I{j(.)w(. - V)*}1 2 db )1/2

L

Ij(JL)w(JL -

v)*1 2 dJL

(lj12 * Iw 12 ) (v)

:= h(v).

This function h(v) is in LI, being the convolution product oftwo LI-functions. In particular, IA(t):s

j

+A -A Ih(v)1 dv < 00,

and JA is therefore weH defined. Using a previous calculation, we have JA(t)

where g(v):=

L

=

j

+A -A g(v) dv,

.r-I{j(.)w(. - v)*}(b)w(t - b)e2inv (t-b) db.

Dl·2 The WFT and Gabor's Inversion Forrnula

181

By the Plancherel-Parseval identity, g(v) =

= Therefore, fACt) =

L

!(f.1.)w(f.1. - v)*w(f.1. - v)e 2in /Lt df.1.

L

!(f.1.)lw(f.1. - v)1 2 e2in /Lt df.1..

i: (L A

!(f.1.)lw(f.1. - v)1 2 e2in /Lt df.1. ) dv.

In order to change the order of integration in the above integral, we first verify that (v, f.1.) -+ 1!(f.1.)llw(f.1. - v)1 2 is integrable over [- A, + Al x lR. But I!I E L 2 and Iwl 2 E L 1 ; therefore, I!I * Iwl 2 E L 2 , and the integral of an L 2-function over a finite interval is finite. But this integral is just

i: (L A

1!(f.1.)llw(v - f.1.)1 2d f.1.) dv

=

i: (L A

1!(f.1.)llw(f.1. - V)1 2d f.1.) dv

since Iwl is even. We can now apply Fubini's theorem to obtain fACt)

L(i: L (i: =L A

=

=

!(f.1.)lw(f.1. - v)1 2e2in /Lt dV) df.1.

!(f.1.)e 2in /Lt

A

Iw(f.1. - v)1 2 dV) df.1.

!(f.1.)C{JA(f.1.)e 2in /Lt df.1.,

where 0:::: C{JA(f.1.) :::: 1, in view of assumption (b). In particular, !C{JA fA

= F-

1

E L 2,

and

~

(jC{JA).

We now show that limA too fA = f in L 2. For this, we write, using the PlancherelParseval identity, IIf -

fAlli2 = IIF- 1!

- F- 1!C{JAlli2

= IIF- 1{!(1 -

C{JAmi2

Wehave

=

j

/L-A

-00

Iw(Y)1 2 dy

+

1+00 /L+A

Iw(Y)1 2 dy,

182

D 1. The Windowed Fourier Transform

and, therefore, if I/LI ::; A12,

o ::; 1 ::;

j

((JA(/L)

-A 12

-00

Iw(Y)1 2 dy

= y(A) -+

0

+

1+00 Iw(Y)1

2

AP

dy

(A -+ (0).

Also,

-+ 0

sinee

j

E

(A -+ (0)

L 2 . Therefore, finally,

lim

Atoo

•

111 - IAlli2 = O.

From (7) and the Planeherel-Parseval identity, we obtain the two expressions for the windowed Fourier transform: (13)

where WV.b is defined by (11). We assume (without diminishing the generality of the diseussion to follow) that w and W are funetions eentered at zero, that is,

L

tlw(t)1 2 dt

=0

and

L

vlw(v)1 2 dv

= O.

The RMS width a w is an indicator of the loealization of Wv.b about b, whereas ofthe loealization of WV.b about v. The reetangle [b - a w , b + + aJij] is the loeal time-frequeney box about (b, v) analyzed by the windowed Fourier transform at (b, v). It is of interest to have a sharp resolution, that is, to make the area 4awaJij of the time-frequeney box as small as possible. However, windows have the basic limitation eontained in the uneertainty prineiple, whieh says that aJij is an indieator a w ] x [v - aJij, v

1

(14)

awaJij:::: 4n '

with equality if and only if w(t) shows that.

==

Ae-ct2 ,

where c

=

4n2a~. The last result

D1.3. The Gabor window is optimal, in the sense that it minimizes the uncertainty aJijaw.

THEOREM

Dl·2 The WFf and Gabor's Inversion Formu1a

183

The windowed Fr is a continuous transform, in that the local time-frequency content of a signal is contained in a function of two continuous arguments. It would be interesting to have a discrete version, that is, a decomposition of the signal along a Hilbert basis. More specifically, one asks the question: Is there a window w(t) such that the family {lj!m,n}mE71,nE71, where

lj!m,n(t)

= e2irrmtw(t -

n),

(15)

is an orthonormal basis of L 2 (ffi.)?

01.1. Show that the answer to the previous question is positive for the rectangular window w(t) = 1[0,1](t).

EXERCISE

Although such "atomic" windowed Fr bases do exist, they turn out to be very bad from the view point of time-frequency resolution, as the following result, called the Balian-Low theorem,5, shows. THEOREM 01.4.

If{ lj!m,n}mE71,nE71, where lj!m,n is defined by (15)(with gEL 2 (ffi.»), is an orthonormal basis of L 2 (ffi.), then at least one of the following equalities is true:

L

t 2Iw(t)1 2dt =

EXERCISE

00

or

L

v 2Iw(v)1 2dv =

00.

01.2. Show that the system {lj!m,n}mE71,nE71, where lj!m,n(t)

= e2irrmt e-a (t-n)2,

()( > 0, is not an orthonormal basis of L 2 (ffi.).

SR. Balian, Dn principe d'incertitude fort en theorie du signal et en mecanique quantique, CR Acad. Sei. Paris, 292, Sero II, 1981, 1357-1361; F. Low, Comp1ete sets of wave packets, A Passion for Physics-Essays in Honor of Geoffrey Chew, 17-22, World Scientific: Singapore, 1985.

D2 The Wavelet Transform

D2·1

Time-Frequency Resolution of Wavelet Transforms

Definition of the Wavelet Transform

We mentioned in the introduction to Part D the shortcomings of the windowed Fourier transform. This chapter gives another approach to the time-frequency issue of Fourier analysis. The role played in the windowed Fourier transform by the family of functions Wv,b(t) = w(t - b)e+ 2i :rrvt,

b, v E IR,

is played in the wavelet transform (WT) by a family 1/Ia,b(t) -_

lai -1/2 1/1

(t - b) -a-

,

a, b

E

IR, a =I 0,

(16)

where 1/I(t) is called the mother wavelet. The function 1/Ia,b is obtained from the mother wavelet 1/1 by successively applying a change of time scale (accompanied by a change of amplitude scale in order to keep the energy constant) and a time shift (see Fig. D2.1). DEFINITION D2.1. The wavelet transform ofthe function f E

C j : (IR - {On x IR

1-*

C defined by

Cj(a, b)

= (j, 1/Ia,b) =

l

P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

LUIR) is the function

f(t)1/I:,b(t)dt.

(17)

186

D2. The Wavelet Transform

1lV2 1 1 1 1

1 1 1 1 1 3.51 1 1 1

,-------, 1

.

1 -1

1

1y'2 1

0 11

1+1 1 1

1

-1 L - J

)

1

t

17.5 1

-y'2

1 1

-V2 L.....

'ljJ(t)

'ljJ2,7.5(t)

'ljJ! ,3.5 (t)

Figure D2.1. Dilations and translations The Adaptive Time-frequency Box By the Plancherel-Parseval identity,

C j(a, b) =

(j, Vla,b)

=

1

(18)

j(v) VI;,b(V) dv,

where (19)

Let m", and a", be, respectively, the center and RMS width of the mother wavelet 1/1 , respectively defined by m",

= -1

E",

1lR I

11/I(t)1 2 dl,

aJ = _1_ r(I E", llR and similarly define m;;; and a;;;, where and width of 1/1a,b are, respectively,

m",)211/1(1)1 2 dt,

VI is the Fourier transform of 1/1. The center

b+am""

aa""

whereas the center and width of Vla,b are

1 a We shall simplify notations by writing

-m;;;,

1 a

-a;;;.

We see that C j(a, b) is the result of the analysis of the function frequency box (see Fig. D2.2)

[b+am-aa,b+am+aa]x

[ma -

~a

f

m + ~J. a a

in the time-

D2·2 The Wavelet Inversion Forrnula

187

a

o

i

b

Figure D2.2. Time-frequency tiling in wavelet analysis Let us assurne that fii > O. The frequency window is then centered at v = fii / a and has width 2a / a; therefore, Q-

center frequency fii -bandwidth - 2a

is independent of the frequency variable a. This is called constant-Q jiltering. Calling v = fii / a the center frequency, we see that the area of the box is constant and equal to 4a but that its shape changes with the frequency v = 1/a analyzed. For high frequencies it has a large time dimension, and for small frequencies it has a small time dimension (see Fig. D2.2). The interest of such features is discussed in the introduction to this chapter.

a,

We shall see in the next subsection that in order to guarantee perfect reconstruction of the signal from its wavelet transform, the center of :(f must be zero. Also, the center of the wavelet itself can be taken equal to zero without loss of generality. The Fourier transform of a wavelet has bumps at positive and negative frequencies (see Example D2.3, the Mexican hat). The centers of the bumps then play the role of the center of the wavelet in the first part of the above discussion (where fii was assumed to be nonzero ).

D2·2

Wavelet Inversion Formula

Under mild conditions, there exists a wavelet inversion formula similar to the WFT inversion formula. THEOREM

D2.1. Let 1jJ : ~

f-+

~ be a mother wavelet such that 1jJ E LI

L

11jJ(t)1 2 dt

=

1,

n L 2, (20)

188

D2. The Wavelet Transfonn

and

1~(v)12 dv =

(

K <

lvi

JR.

00.

(21)

Define 1/1a,b E L 2 n LI by (16). To the function J E L 2 is associated its wavelet transJonn C j : IR\{O} X IR -+ C, defined by (17). Then

~

{ IC j(a, b)1 2 da ~b = ( IJ(t)1 2 dt a JR.

{

K JR.\/OjJR.

(22)

and

(C j(a, b)1/Ia,b(t) da ~b ,

J(t) = K1 {

(23)

a

JR.\/OjJR.

in the sense that Je -+ J in L 2 as e -+ 0+, where Je(t) = -1 ~~ K

dadb C j(a, b)1/Ia,b(t) 2a

R.x/lal:::eJ

Proof' First observe that 1/Ia,b has the same energy as From (18) and (19), C j(a, b)

= lal 1j2

.

1/1, equal to unity.

f !(v)~*(av)e-2i7rvb

dv

The function inside the curly brackets is in L I because it is the product of two L 2-functions, and it is in L 2 as it is the product of an L 2-function with a bounded function (~ is bounded because 1/1 E LI). By the Plancherel-Parseval identity,

L'C j(a, b)1 2 db

= laILIF;I{!(v)~*(av)}12 (b) db = laILI!(v)121~(av)12 dv,

and therefore,

11

R. R.

dbda IC j(a, b)1 2 2- = a

11

R. R.

~ 2 da IJ(v)1 2 11/I(av)1 dvla I A

= LI!(V)1 2 K

dv

=K

LIJ(t)1 2 dt.

D2·2 The Wavelet Inversion Formula

189

This proves (22). To prove (23), first compute

L

C j(a, b)1/!a,b(t) db

l(a) =

lal l / 2

=

L L

F;I{!(v)V;*(av)}(b)1/!a,b(t)db

= lal l / 2

!(v)V;*(av)F;I{1/!a,b(t)}(v)dv,

where we have used the Planchere1-Parseval identity. Now,

= lal l / 2 V;(av)e 2i Jl'vt,

F;I{1/!a,b(t)}(V)

and therefore,

and, for c > 0, J€(t) =

~

(

l(a)

=

~

{

( { !(v)IV;(av)1 2 e2i Jl'vt dV) da .

K J11al?€}

d~ a

lai

K J11al?€} JR

With a view to applying Fubini's theorem we must check that the function ~

~

2

1

(a, v) -+ IJ(v)II1/!(av)1 ~

is integrable in the domain lR. x {la I ~ c}. We have 1

=

11

~

R lal?€

~

IJ(v)II1/!(av)1 -

lai

JR

=

1+ + {

11

=

1+

J1VI?1

1

Ix I

= 11

1

1V;(x)1 2

J1xl?€IVI

+1

-I

1!(v)1 dv <

dx) dv

+h

1!(v)1 ( (

-I

:s K

1V;(x)1 2

( J1xl?€lvl

1

-I

But

lai

J1al?€

= { 1!(v)1 (

da dv

1V;(av)1 2 da) dv

= { 1!(v)1 ( ( JR

1

00

lxi

dX) dv

(25)

190

D2. The Wavelet Transfonn

because!

E

L2e~) and, in particular, !

lz =

1

l!ev)1

Ivl:::l

(1

E

L 2 e[ - 1, + 1]). Also, 2

~

11/Iex)1 dX) dv Ixl:::slvl Ix I

But

l11/let)1 2 dt = 1, and, therefore, using Schwarz' inequality,

lz

!

1

< -1

(1

<

l!ev)1 dv lvi

- e Ivl:::l

- c

IJev)1 2 dv )1/2 A

Ivl:::l

(1

-dV)I/2 Ivl:::l v 2

We can therefore change the order of integration in (25): ~

KJset)

2

= { !ev)e 2i ;rvt ( ( 11/Ieav)1 da) dv JlR Jlal:::s lai

=

l

!ev)e 2i ;rvt gsev)dv

= F-1{!gs}(t).

Since we want to prove that Js -+ J in L 2 , we must evaluate the L 2 -norm of J- Js:

K 2 11J

- Jsll 2 = K 2 I1F- 1{j

-

Js}1I 2 = lI!eK - gs)1I 2

1 1

= { IK - gsev)1 2 1!ev)1 2 dv = + = A + B. JlR Ivl:o:eIvl>e1/ 2

On

{lvi

:s c- I / 2 }, gsev) = { Jlal:::s

l1freav)1 2 da lai

= { Jlxl:::SIVI

l1frex)1 2 dx lxi

1/ 2

D2·2 The Wave1et Inversion Formu1a

191

where

Ke

:s K

and

lim K e

e-+O+

= K.

Therefore,

Also,

since

j

E

L 2 . We have therefore proved that

IIf - fell

--+ 0

as

8

--+ 0+.

The proof is lengthy because we have only required D2.2. for almost all t. THEOREM

Proof"

If f

E

L 2 n LI and

We start from (24):

1

[Ca) -da

Ja

a2

=

11

Ja Ja

j

E

LI, the inversionformula (23) is true

~ ~ 2 2· dvda f(v)Il{!(av)1 e "rvt -

a2

= [ j(v)e2irrvt ( [ lJa

= K

f to be in L 2 •

L

lJa

1~(av)12 da) dv lai

g(v)e2irrvt dv.

This quantity is almost everywhere equal to K f (t) by the Fourier inversion formula in LI. • Recall that if f is continuous the equality in the Fourier inversion formula holds for all t and, therefore, (23) is then true for all t E R Oscillation Condition

Since ~ is continuous (l{! is, to say,

E

LI), the assumption (21) implies that ~(o) = 0, that

L

l{!(t) dt

= o.

(26)

192

D2. The Wavelet Transfonn

In most situations it suffices to verify (26), and then (21) follows. For example, if 1/I(t) and t1/l(t) are integrable, then Vi is Cl; therefore, if Vi(o) = 0, the quantity IVi(v)1 2 /lvl is integrable in a neighborhood of zero and therefore on ~, since at infinity there is no problem, due to the hypothesis 1/1 E L 2 (which implies that

Vi E L2 ).

EXAMPLE

02.1 (Modet's pseudo-wavelet). 1/I(t)

= ye-

t2 / 2

Morlet used the mother wavelet cos(5t),

where y is a normalization factor that makes the energy equal to unity. The theoretical problem here is that

Vi(O) =

L

1/I(t) dt > O.

However, the numerical results obtained with this wavelet were satisfactory because the value of Vi(O) is in fact very smalI. EXAMPLE

02.2 (Haar wavelet).

The Haar wavelet ifO ~ t < ~, if ~ ~ t < 1,

otherwise, satisfies the conditionsfor the reconstructionformula (23) to be valid. Here ~

1/I(v)

= le. -;rrv

1 - cos(7l' v) 7l'V

.

EXAMPLE 02.3. In practice, a mother wavelet 1/1 should be weil localized in addition to Vi, and it should also be oscillating (so as to guarantee at least that 1/I(t) dt = 0). Derivatives ofthe Gaussian pulse are goodfor this purpose. For example, the second-order derivative, called the Mexican hat (see Fig. D2.3),

IR

1/I(t) "" (1 - t 2 )e- t2 / 2 with the Fourier transform ~

2

1/I(V) "" v eis interesting because both

2 2 2 rr v

1/1 and Vi are rapidly decreasing C OO -functions.

We shall now give a pictorial example. Fig. D2.4 shows a simple signal and Fig. D2.5 shows its wavelet transform. The mother wavelet used is not given, since it is irrelevant to this qualitative illustration. In the latter figure, the time axis is horizontal, and the time axis vertical, the bottom part corresponding to high frequencies. We observe the good time localization and the fact that sharp discontinuities are represented in the bottom part.

D2·2 The Wavelet Inversion Formula

193

~ilr-----·~----II (a)

-10

-8

-6

-4

-2

0

2

4

6

8

2

4

6

8

JJ(/:::

10

(b)

-10

-8

-6

-4

-2

0

1 10

(c)

j:~ -10

-8

-6

-4

-2

0

2

4

Figure D2.3. The Mexican hat

6

8

10

194

D2. The Wavelet Transform

0.8

0.6

0.4

0,2

01 - - - - - - - '

~.2

~

o

__

~

500

L _ _ _ _ _- - - '

__ __ ___ __ __ __ __ 1000 1500 2500 4000 2000 3000 3500 ~

~

~

Figure D2.4. Spike

~

~

~

+ sinusoid

Figure D2.5. The wavelet transform of the signal in Fig. D2.4

~~

D3 Wavelet Orthonormal Expansions

D3·1

Mother Wavelet

The wavelet analysis of Chapter D2 is continuous, in that the original function of L 2 is reconstructed as an integral, not as a sumo One would rather store the original function not as a function of two arguments, but as the doubly indexed sequence of coefficients of a decomposition along an orthonormal base of L 2 • Multiresolution analysis is one particular way of obtaining such orthonormal bases. In the remainder, we adopt a slightly different definition of the Fourier transform. The FT j(w) ofthe signal f(t) is now defined by j(w)

=

L

f(t)e- iwt dt.

The inversion formula, when it holds true, takes the form f(t) = _1 [j(w)e iwt dw, a.e.,

2n

JR

and the Plancherel-Parseval identity, when it holds true, reads [ f(t)g(t)* dt = _1 [j(w)g(w)* dw.

JR

2n

JR

Also, the necessary and sufficient condition for {cp( . - n) }nEZ to be an orthonormal sequence of L~(IR) (Theorem C4.5) now reads

L 1 0, and then take (without further loss of generality) qJ'(0) = 1.

(32)

DEF1NITION D3.3.

A wavelet orthonormal basis 01 L 2 basis olthelorm {1/Ij,n}j,nez, where

= L~(lR) is an orthonormal (33)

The function expansion

1/1 is then called the mother wavelet of the wavelet basis. The

1 = LL(f, 1/Ij,k)1/Ij,k jeZ

(34)

keZ

is called the wavelet expansion of I. A wavelet orthonormal basis can be obtained from an MRA in the following way. Let Wj be the orthogonal complement of Vj in Vj+l: (35)

Vj+l = Vj EB Wj. From property (d) of the definition of MRA, L2

= EBWj .

(36)

jeZ

Also, from (e),

1E

Wo ~

1(2 j .) E

Wj.

Therefore, ifwecan exhibit an orthonormal basis of Wo ofthe form {1/1( . -n)}nez, then {1/1 j.n }neZ is an orthonormal basis of Wj. Therefore, by (36) {1/1 j,n b,nez is an orthonormal basis of L 2 • Recall that Pj is the projection on Vj . We have Pj+I!

= Pj/ + L (f, 1/Ij.k)1/Ij,k keZ

for all

1E

L2.

(37)

Pj 1 is the result of observing 1 at the resolution level j: As j increases, the resolution increases (note that Vj C Vj+l); the difference

Pj+I! - Pj/

= L(f, 1/Ij,k)1/Ij.k keZ

is the additional detail required to pass from the resolution level j to the higher resolution level j + 1. A first issue is: How to compute the mother wavelet 1/1 from the scaling function cp? The next question is: How to obtain a scaling function cp? Finally, one would like to obtain a mother wavelet with "good" numerical properties, that is fast convergence of the wavelet expansion (34).

202

D3. Wavelet Orthonormal Expansions

D3·2

Mother Wavelet in the Fourier Domain

We address the first issue, that of explicitly finding a mother wavelet given a scaling function.

D3.1. We seek to obtain the mother wavelet corresponding to the Haar scaling function. Recall that VI is the Hilbert space of L 2 -functions that are constant almost everywhere on the intervals (nI2, (n + 1)/2]. The mother wavelet 1/1 must be ofthis type since Wo C VI. Thefunction ofnorm 1, with support (0,1], EXAMPLE

1/I(t)

=

1

!],

+1

ift

E

(0,

-1

ift

E

(!' 1],

does it. To see this, it suffices to verify that any f E VI with support (0, 1] is a linear combination of cp and 1/1 and that cp and 1/1 are orthogonal. Orthogonality is obvious. Any fE VI such that supp(f) E (0, 1] is oftheform f(t)

=

l

!],

a

ift

E

(0,

ß

ift

E

(!' 1].

and we therefore have the decomposition (see Fig. D3.4) a+ß

a-ß

f = - 2 - cp + -2-1/1· The function 1/1 is called the Haar mother wavelet.

5

4" ----,

l---

/

f(t)

L --

1

1

4"

01

I

1 2

1

~

~

~x 4

'Cl

°

1

'ljJ(t)

+1

lx 2

-1

21 1 1 1 1

1

--~

Figure D3.4. Haar decomposition

D3·2 Mother Wavelet in the Fourier Domain

203

We now give a simple example of wavelet analysis to illustrate the notions of projection and detail. Fig. D3.5a gives (from top to bottom) a signal and its successive projections on the nested subspaces at decreasing resolution levels, whereas Fig. D3.5b gives (from top to bottom) the successive projections on the detail subspaces at decreasing resolution levels. In particular, the second function in Fig. D3.5b is the difference between the first and second functions in Fig. D3.5a. The general case will now be treated. We shall obtain a necessary condition for the scaling function ({J to be a scaling function. First, since {({J(' - n)}nEZ is an orthonormal system, we have, by Theorem C4.5,

L I$(w + 2kn)1

2

= 1,

a.e.

(38)

kEZ

The scaling function ({J E Vo and therefore, ({J E VI. Requirements (a) and (e) in the definition of an MRA imply that {({JI,n}nEZ is a Hilbert basis of VI. and we therefore have the expansion ({J = LnEZ hn({JI,n, that is, ({J

= v'2

L h n({J(2 . -n),

(39)

nEZ

where (40)

In the Fourier domain (39) reads

~ ((J(w)

=

1M ~hne'""' inW~(W) 'i({J "2

-v2

nEZ

'

that is, (41) where mo(w) is the 2n-periodic function defined by mo(w)

=

1M ~hne '""'

-v2

-inw .

(42)

nEZ

It is called the low-pass filter MRA, because mo(O) = 1 (recall the running assumption that $(0) = 1; see (32)). Substituting identity (41) in (38) gives

204

D3. Wavelet Orthonormal Expansions

300~

200

100 O~

__

~

____- L____

50

100

::~'

~

____L -_ _

150

200

'~'

~

_ _ _ _- L____~____L-~~____~

250

100

150

400

450

500

AM' '~ ,~-, LJJ

- L_ _ _ _~_ _ _ _~_ _ _ _L -_ _ _ _L -_ _~~

- J_ _ _ _~_ _ _ _- L_ _ _ _

50

350

'''A~'

100~~-·, __ -L____ O~

300

200

250

300

350

400

450

500

300~

200

100 O~

__- J_ _ _ _ 50

~

_ _ _ _- L_ _ _ _~_ _ _ _- L_ _ _ _~_ _ _ _~_ _ _ _~_ _ _ _~_ _~~

100

150

200

250

300

350

400

450

500

300~ 200 100 O~

__

~

____- L____

50

100

~

150

____L -__

200

~

____- L____

250

300

~

____L -_ _

350

400

~

____

450

~

500

300~

200

100 O~

__- J_ _ _ _ 50

~

_ _ _ _- L_ _ _ _- L_ _ _ _- L_ _ _ _~_ _ _ _~_ _ _ _L -_ _ _ _~_ _~~

100

150

200

250

300

350

400

450

500

300

350

400

450

500

(a)

50

100

150

200

250

,oo~ -10: 50

100

150

200

250

300

350

400

450

500

,oo~ -10: 50

100

150

200

250

300

350

400

450

500

100~ -10: 50

100

150

200

250

300

350

(b)

Figure D3.5. Haar wavelet analysis

400

450

500

D3·2 Mother Wavelet in the Fourier Domain

205

Therefore,

or, equivalently, (43) The filter with frequeney response eiwmo(w + n)* is ealled the high-pass filter of the MRA. Eqn. (43) shows that the high-pass and the low-pass filters altogether extraet the whole energy eontained in the band [-n, +n]. We now eharaeterize the spaees V-I and Vo. This will be a preliminary to the eharaeterization of W_I, the orthogonal eomplement of V-I in Vo. Onee this is done, we shall obtain the eharaeterization of Wo and then the mother wavelet itself. LEMMA

D3.1. f E V-I

if and only if it has an FT of the form f(w)

= m(2w)mo(w)qJ'(w),

for some 2n -periodic function m

Proof" Indeed, any f {CP-I,n}nEZ, that is,

E

E l~.

+n]).

V-I ean be deeomposed along the orthonormal basis

f(t) where {Cn}nEZ

E L~ ([ -n,

(44)

=

1

1 I>kCP( -t - k), ",2 kEZ 2

(45)

M

Taking the FT, we obtain f(w)

= hI>ke-i2kWqJ'(2w). kEZ

This is (44) (using (41) and defining m(w)

= v'2 LkEZ Cke-ikw).

f

Conversely, eonsider a funetion defined by (44), where m is a 2n-periodie funetion in L~([-n, +n]). We show that fis in L~(lR.). First, observe that it is of the form h(w)qJ'(w), where h is a 2n-periodic funetion in L~([ -n, +n]) (sinee m E L~([ -n, +n]) and sinee mo is bounded in view ofEq. (43». Now [ Ih(w)qJ'(w)1 2 dw

JR

=L

j+Jr Ih(w)1 21qJ'(w + 2k1T)1 j +Jr 2

kEZ

=

-Jr

-Jr

Ih(w)1 dw <

+00.

2

dw

206

D3. Wavelet Orthononnal Expansions

f

f

This proves that E L~(lR). Since E L~(lR), it is the Fr of a function I E LUlR). Tracing back the computations in the first part of the proof, we obtain that (45) holds true, with {cn}nEZ E e~, which implies that I E V-j. • LEMMA

D3.2.

I

E

Vo

if and only if it has an FT 01 the lonn f(w) = d(w)$(w),

lor some 2rr -periodie function d Proof Indeed, let {fPo,n }nEZ, that is,

I

E

E

(46)

L~([ -rr, +rr]).

Vo. It can be decomposed along the orthonormal basis

= L dkfP(t -

I(t)

k),

(47)

kEZ

where {dn}nEZ

E

e~. Taking the Fr, we obtain

f(w) =

L dkeikwqJ(w) = d(w)$(w), kEZ

where d E L~([-rr, +rr]). Arguing as in the proof ofLemma D3.l, we can show • that any function of the form (46) is the Fr of a function I E Vo.

f

Consider the mapping U : Vo !--+ L~([ -rr, +rr]) defined by UI = d (where d is defined by (46)). Clearly, this mapping is linear, and IIUfllh([-rr,+rr])

= IIdlli~([_rr,+rr]) = 2rr

By the polarization identity, for all (j, g) L~(R)

I, g E

L Id l

k 2

kEZ

= 2rrll/ll~.

(48)

Vo,

1

= 2rr (UI, U g) L~([-rr,+rr])'

(49)

We are now ready to state and prove the Fourier characterization of Wo, the Hilbert space of details at level O. THEOREM

D3.2. The function f(w)

I

E

Wo

if and only if

= ei~mo(~ + rr)* V(w)$(~) ,

(50)

lor some 2rr -periodie function v in L~( -rr, +rr). Proof

Observe that it is equivalent to prove that the function lEW_I if and

only if (51)

for some 2rr-periodic function v in LU -rr, +rr). Let lEW-I, that is, I E Vo and 1..1 V-I. Being in Vo, I has a representation oftype (46). By (49) and the characterization (44) of V-I, the orthogonality of I and V-I is equivalent to

0=

(2rr

10

d(w)m(2w)*mo(w)* dw,

D3·2 Mother Wavelet in the Fourier Domain

for all2Jr-periodie funetion m 0=

E L~([ -Jr,

207

+Jr]). This ean also be written

Ln: m(2w)* [d(w)mo(w)* + d(w + Jr)mo(w + Jr)*] dw.

The funetion in the square braekets is therefore orthogonal to all g and therefore,

E L~([O,

d(w)mo(w)* + d(w + Jr)mo(w + Jr)* = 0

+Jr]), (52)

almost everywhere in [0, +Jr] (and therefore almost everywhere in [-Jr, +Jr)). Define mo(w) = (mo(w), mo(w + Jr».

In view of the identity (43), this is a unitary veetor in C 2 eonsidered as a 2dimensional veetor spaee (with sealar field C). The veetor m'o(w) = (mo(w

+ Jr)*, -mo(w)*)

is unitary and orthogonal to mo(w). Defining do(w) = (d(w), d(w + Jr»,

we have, by (52), do.lmo(w). Therefore,

do

= Ä(w)m'o(w),

where Ä(w) =

(do, m'o(w»)

= d(w)mo(w + Jr) - d(w + Jr)mo(w).

In partieular, Ä(w + Jr)

= -Ä(w + 2Jr),

a.e.

or, equivalently, Ä(w) = -Ä(w + Jr), a.e., whieh implies in partieular that Ä is 2Jr-periodie. It is also in L~([ -Jr, +Jr)). Indeed, j(w) = d(w)qJ(w) = Ä(w)mo(w + Jr)*qJ(w),

and therefore,

1n: IÄ(w)1 2 dw = 111: IÄ(w)12(lmo(w)12 + Imo(w + Jr)1 2) dw

208

D3. Wavelet Orthononnal Expansions

where the last equality follows from (48). Defining

gives the representation (51). Conversely, suppose that j(w) = eiwmrf,w

+ n)*v(2w)qy(w),

for some 2n-periodic function v in L~([ -n, +n]). That is, j(w) = d(w)ifI..w),

where d(w)

= eiwmriw + n)*.

Since Imo(w)1 ::: 1, this implies that d(w) E L~([-n, +n]). Therefore, I E Vo (Lemma D3.2). Also, from the expression of d(w), do(w) = eiwv(w)m'o(w), and therefore do..lmo(w),

that is, d(w)mo(w)* + d(w + n)mo(w + n)* = O. By Lemma D3.1 and Eq. (48), • this implies that 1.1 V-I. But also I E Vo. Therefore, I E Wo. We are now ready for the main result of this subsection, the Fourier characterization of the mother wavelet in terms of the scaling function and of the high-pass filter. THEOREM

r

D3.3. The junction 'tjJ is a mother wavelet if and only if :V;(w)

= eiW/2mo(~ + n V(w)qy(~) ,

lor some 2n-periodicjunction v such that Iv(w)1

(53)

= 1 almost everywhere.

Proof" Since:V; is of the form (50) with lvi = 1, it is in L~(IR) (by the now standard argument) and, therefore, it is the Fr of a function 'tjJ E L~(IR), which is in Wo by Lemma D3.2. By Lemma D3.2 again, any function g E Wo has an Fr of the form g(w)

r

= eiW/2mo(~ + n s(w)qy(~) ,

for some 2n-periodic function s in L~([ -n, +n]. In particular, since g(w)

Since sv*

E L~([ -n,

= s(w)v(w)*:V;(w).

+n], s(w)v(w)*

= LCke-inw, nEZ

v-I

= v*,

D3·2 Mother Wavelet in the Fourier Domain

for some sequence (cn}nEZ

E l~,

and therefore,

g(t)

= I:>k1fr(t -

209

n).

nEZ

Therefore, the translates of orthonormal because

1fr

generate Wo. The system

L 11fr(w + 2br)1

2

= 1,

{1fr(. -

n) }nEZ is

a.e.,

kEZ

as can be checked by the usual routine. Conversely, let 1fr be an orthonormal wavelet. Being in Wo, it is of the form (50). By the usual calculations, we find that

L 11fr(w + 2krr)1

2

= Iv(w)1 2 ,

kEZ

and therefore, by the orthonormality condition, Iv(w)1 2

= 1.

•

In summary, the mother wavelet is of the form (54) where ml(w) = eiWmo(w+rr)*v(2w) is a high-pass filter (Iml(rr)1 that the scaling function cp and the low-pass filter are related by

q;(w) = mo (~) q;(~) .

= 1). Werecall (55)

We also have the identity (56) These three relations fuHy describe the MRA: (55) is called the dilation equation and teHs us that Vj - 1 is obtained by low-pass filtering Vj ; (54) teHs us that Wj - 1 is obtained by high-pass filtering Vj; and (56) guarantees that there is no loss of energy. The choice v

==

1 leads to 1fr(w)

= eiW/2mo(~ + rr)* q;(~),

or, equivalently, up to the sign, 1fr(t)

=.J2

L(-It-1h"'-n_lcp(2t - n),

(57)

where h n is defined by (40). EXAMPLE D3.1 (The Haar Wavelet). We shall obtain the Haar wavelet by the general method just described. Recall that the scaling function is cp(t) = I[o,1](t)

210

D3. Wavelet Orthonormal Expansions (b)

(a)

-

r-

0.5

0.5

0

o

-0.5

-0.5

-1

-1

-3

-2

-1

-3

3

2

0

-2

o

-1

2

3

(d)

(c)

1.5

~

1.5 ,-----------~--~--~--____,

0.5

0.5

OL-------~--~--~--~

o

0.1

0.2

0.3

0.4

0.5

0.1

0.2

0.3

0.4

0.5

Figure D3.6. Haar scaling function and the corresponding wavelet (left: scaling function; right: wavelet; top: time domain; bottom: frequency domain) and, therefore, hn

= v'2

L

q;(x )q;(2x - n)* dx

forn = 0, I, otherwise, and, using (57), 1jI(t) = q;(2t) - q;(2t - I).

Thus, we recover the Haar wavelet (Fig. D3.6). EXAMPLE

D3.2 (Shannon wavelet).

Here

fi(w) = I[-n,+nj(w),

and, therefore, q;(t) =

sin(nt) nt

D3·3 Mallat's A1gorithm

211

Wefirst choose mo such that (41) holds, i.e., $(2w) = mo(w)$(w).

Therelore, necessarily, mo(w)

= $(2w)

on

[-n, +n],

that is,

By periodicity,

=L

mo(w)

$(2w + 2kn).

kEZ

Ourchoice 011/1 is as in (53), with v(w) = _ie- iw : V!(2w)

= - e-iwmo(w + n)*$(w) = - e- iw

(L

$(2w + 2kn

kEZ

=-

e- iW ($(2w

= _e- iw

+ 1») $(w)

+ n) + $(2w -

(1 [_lI"._lj](w) + 1[+lj,+lI"](w»).

This gives the Shannon wavelet (Fig. D3.7) 1/I(t) = cos(ln(t _ 2

D3·3

n»

!» 2

sin( !n(t 2

!» 2

~n(t _ ~)

Mallat's Algorithm

Mallat's algorithm6 is a fast algorithm for obtaining from the projection at a given level the wavelet's coefficients at coarser levels of resolution. Let 1 be a function in L~(IR).lts projection on Vj , the resolution space at level j, is

Pjl

= LCj,n'Pj,n, nEZ

where (58) 6Mal1at, S. A theory of multireso1ution signal decomposition: The wave1et representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1989, 674-693.

212

D3. Wavelet Orthonormal Expansions (a)

(b)

0.5

0.5

0

0

-0.5

-0.5

-1 -15

-10

-5

0

5

10

-1 -15

15

(c)

1.5

-10

-5

0

10

15

(d)

1.5

0.5

5

0.5

0.1

0.2

0.3

0.4

0.5

0.1

0.2

0.3

0.4

0.5

Figure D3. 7 Shannon sealing funetion and the eorresponding wavelet (left: sealing funetion; right: wavelet; top: time domain; bottom: frequeney domain)

Its projeetion on Wj , the spaee of details at level j, is Dj/

= Ldj,nVrj,n, nEZ

where (59)

and wehave (60)

Denote by Cj and d j the sequenees {Cj,n}nEZ and {dj,n}nEZ, respeetively. The purpose of Mallat's algorithms is to decompose the funetion f, that is, to pass from CM to dM-I, dM-I, ... , do, Co, and to reconstruct that is to pass, from co, do,d l , " " d M to CM. The sequenee d M - lo d M - lo ... , do, Co is the wavelet encoding ofthe wavelet data CM' We shall explain the interest of this eneoding onee we have derived Mallat's algorithm.

D3·3 Mallat's Algorithm

213

Since the function cp(t/2) is in V-I, and V-I C Vo, and since {cp(. - n)}nEZ is a Hilbert basis of Vo, we have the decomposition

1

1

2CP(2 t ) = L

ancp(t

+ n),

nEZ

where

11m cp(-t)cp(t+n)dt. 1

a n =2

2

lR

Therefore,

=2

L::!. 2

1

1

2

2

.

-cp( -(2' t - 2n))

= 2-9- LakCP(2jt -

2n

kEZ

+ k),

that is, CPj-l.n

= V2L a kCPj,2n-k.

(61)

kEZ

Sirnilarly, since the function 1/1'(t/2) is in W_I, and W_I C Vo, and since {cp(. n)}nEZ is a Hilbert basis of Vo, we have the decomposition

where ßn

= ~ [ 1/1'(~t)cp(t+n)dt. 21lR

2

Therefore, it follows by computations sirnilar to those above that 1/1'j-l,n

= V2LßkCPj,2n-k. kEZ

Denoting the low-pass filter by mo(w)

=L

ane inw

nEZ

and the high-pass filter by ml(w)

= Lßneinw, nEZ

we have, from (55) that qy(2w) = mo(w)cp(w),

(62)

214

D3. Wavelet Orthonormal Expansions

and, from (54) that

In Theorem D3.3, we now make the particular choice of the mother wavelet corresponding to v(w) = 1: that is, L

= L ( _l)n+la;_neinw.

ßn einw

nEZ

nEZ

Therefore, (63)

Substituting (61) in (58), we obtain Cj-l.n

=

(64)

hLaZcj,Zn-k. kEZ

Similarly, substituting (62) in (59), we obtain d j - 1•n

=

(65)

hLßZCj,Zn-k. kEZ

These are the basic recursions of the decomposition algorithm (see Fig. D3.8). The recursions for the reconstruction algorithm (see Fig. D3.9) are obtained from (60), (61), and (62). This gives Cj,n

EXERCISE

=h

[azk-nCj-l,k L kEZ

(66)

D3.1. Show that for the Haar wavelet, Cj-l,k

@

+ ßZk-ndj-l,k]'

(64)

=

(64)

+ Cj,Zk h

Cj,Zk-l

(64)

(64)

~CM-I ~CM-2 ~

(~

dM-I

(6~

dM-2

~

Cl

(64)

~Co

(6~

Figure D3.8. Mallat's decomposition algorithm

/

/

~CM-I·~CM

s

Figure D3.9. Mallat's reconstruction algorithm

D3·3 Mallat's Algorithm

215

and dj -

I •k

=

Cj,2k-1 -

"fi

Cj.2k

We shall now evaluate the algorithmic complexity of the decomposition algorithm. (Similar results hold for the reconstruction algorithm.) For this we suppose that the low-pass and high-pass filters, mo and ml, respectively, have finite impulse responses, that is, the sequences {an }nEZ and {ßn }nEZ have a finite-number (say, K) of nonzero terms. Suppose that the infinite-dimensional vector CM has in practice a finite number N of nonzero terms (say, after truncation). Then there are approximately N /2 terms in CM-I, and therefore, in view of (64), the passage from CM to CM-I costs approximately K N /2 multiplications; so does the passage from CM to d M -I. For the decomposition algorithm, we therefore have approximately N (K2

N

N)

N

+ K 4' + ... + K 2 M + K 2M = K N

multiplications. The complexity of Mallat's algorithm is therefore linear in data size. Note that Mallat's algorithm encodes N numbers into N numbers. Thus the compression gain seems to be null. However, only a few terms in the sequence of details dj,e, e E Z, j = M - 1, ... ,0, are nonnegligible, provided the MRA is sufficiently smooth. The smoothness issue is discussed in Chapter D5.

D4 Construction of an MRA

D4·1

MRA from an Orthonormal System

The Fourier structure of an MRA is now elucidated, and we know how to obtain a wavelet basis when an MRA is given. This chapter gives two methods for obtaining anMRA. In the previous chapter, we started from a nested family of resolution spaces {Vj bEZ and we discovered a scaling function q; in rather simple examples. Now,

obtaining the scaling function from a given nested family of resolutions spaces can be a difficult task in general. However, if we are interested in a wavelet basis rather than in a given family of resolution spaces, we might as weH start from a given function q; E L 2 with the property that {q;(. - n)}nEZ is an orthonormal system, and define the resolution spaces in an ad hoc manner guaranteeing that q; is indeed the corresponding scaling function. If q; is to be the scaling function, there is but one choice for the resolution spaces, namely, Vj

= span {q;j,n

:n

E

Z}.

An inspired choice of q; will make the Vj 's nested as required, and this has to be verified because there is no reason why it should be so when one starts from an arbitrary orthonormal system {q;(. - n)}nEZ, A necessary and sufficient condition for this is that q;(t)

= L cnq;(2t nEZ

P. Brémaud, Mathematical Principles of Signal Processing © Springer Science+Business Media New York 2002

n),

(67)

218

D4. Construction of an MRA

for some sequence

{cn}nEZ E

e~(Z) or, equivalently, that the dilation equation

(~) fi(~)

fi(w) = mo

(68)

holds for some 2JT -periodic function mo in L~( -JT, +JT). We must also verify that conditions (d) in the definition of, an MRA are satisfied. By Theorem D3.1, it suffices that fibe continuous at the origin and that Ifi(O) I = 1.

Meyer's Wavelet Define cp by

.

fi(w)

=

2JT

lflwl S

3'

. 2JT 3 -

-

l f - < Iwl <

° where v is a smooth function

4JT 3 '

(69)

otherwise, (C k or C OO ) such that

v(x)

=

{~

ifx SO, ifx:::l

(70)

and v(x)

+ v(l

- x)

=

1.

(71)

U sing (71) it is easy to verify that

L lfi(w + 2kJT)1

2

=

1,

kEZ

and, therefore, {cp(. - n)}nEZ is an orthonormal system. We must now verify that the Vj are nested, and for this it suffices to verify that Vo c Vj or, equivalently, that cp E Vj. But this is true if and only if there exists a 2JT -periodic function mo of finite power such that

fi(w) = mo

(~) fi(~) .

It turns out that

mo(w)

= Lfi(2w+4kJT) kEZ

accomplishes what is required. In fact,

~(w) =~ cp(w)cp "2 '

D4·1 MRA from an Orthononnal System

since the supports of $(w + 2klr) and of $(w /2) do not overlap if k

$(~) = 1

if w

E

=1=

219

O. But since

supp(~,

we have

$(w)$(~) = $(w), as desired. We obtain a mother wavelet by formula (53) of with v(w) gives

=

1. This

which gives e iw / 2 sin( :;j;(w)

=

~ v (2~ Iwl -

1))

~ v (2~ Iwl -

1))

e iw / 2 cos(

0 EXERCISE D4.1.

.f 2JT 3

1 -

4JT

if3

::slwl::s ::s Iwl ::s

4JT

3' 8JT

(72)

3'

otherwise.

Let P be a probability measure on lR with support in [-8, +8] C

[-1' +1]' and define q;(t) by its Fourier trans/orm

Check that q;(t) is indeed in LUlR) and that the system {q;(. -n ]}nEZ is orthonormal. Check that the dilation equation (55) holds with

mo(~) =

I

~

4JT

q;(w)

iflwl::S 3'

o

otherwise.

Show that q;(t) so defined is the scaling function 0/ some multiresolution analysis and that a mother wavelet is given by its Fourier trans/orm

Examine the case where P is the Dirac measure at O.

220

D4. Construction of an MRA

D4·2

MRA from a Riesz Basis

Now we do not impose orthonormality. To be specific, we have an L 2 -function W such that W(t)

= I>nw(2t -

(73)

n),

nEZ

where {cn}nEZ

E

eUZ), and we define the resolution spaces by Vj

= span {Wj,n

:n

(74)

Z}.

E

Of course, condition (73) guarantees that these spaces are nested. In order to obtain a Hilbert basis of Vo, we use Theorem C4.6 which says that under the "frame" condition

o< a

L Iw(w + 2krr)1

~

~ß<

2

(75)

00,

kEZ

the system {cp(. - n)}nEZ is a Hilbert basis of Vo, where ~

cp(w)

w(w)

= "~ Iw(w + 2k:rr)1 2 .

(76)

kEZ

Here we shall also have to verify that

For this we can use the following result. THEOREM

D4.1. Let W

E L~(lR.)

o< a

~

satisfy

L Iw(w + 2krr)1

2

~ ß <

(77)

00,

kEZ

and define

Vj

= span {Wj,k : k E Z}.

(78)

Suppose that the Vj are nested. Then

nVj

(79)

= 0.

JEZ

Proof The inequalities (77) are equivalent to the existence of A > 0, B < such that

0< AllfII 2 ~

L l(f, wo,k)1

2

~ BllfII 2 ,

kEZ

for all

f

E

Vo, and therefore equivalent to 0< AllfII 2 ~

L

kEZ

l(f, wj,k)1 2 ~ BllfII 2 <

00,

00

(80)

D4·2 MRA from a Riesz Basis

221

for all 1 E Vj , 1 # 0, and allj E Z. With any 1 E njEZ Vj and s > 0, one can associate a compactly supported and continuous function 1 E L~(~) such that 11 1 - 111 :::: s, and therefore on denoting the orthogonal projection on Vj by Pj , we have

111 Therefore, for all j

E

Pjlll

= IIPj(f - j)11 :::: 111 - 111 :::: s.

Z,

By (80),

Now, with c > 0 such that suppIC[-c,+c]

M=supll(x)l,

and

XEIR

we have 1(1, wjk)1 2 = ITj

1

Ixl-, w

we have, for Iwl

~

Jr - 2'

O 1. From Hölder's inequality,

Ix J(f +

g)p-l dJl :::::

[Ix jP dJl

r/ [Ix p

(f + g)(p-l)q

r/

q

The Lebesgue Integral

259

and

Ix g(f

+ g)p-l dlL

S

[Ix gP dlL flP [Ix (f + g)(p-l)q flq

Adding together the above two inequalities and observing that (p - l)q = p, we obtain

One may assume that the right-hand side of (128) is finite and that the lefthand side is positive (otherwise the inequality is trivial). Therefore, !x(f + g)P dlL E (0,00). We may therefore divide both sides of the last display by [Jx (f + g)P dlL q. Observing that 1 - 1/q = 1/P yields the desired inequality (128). For the last assertion of the theorem, take p = q = 2. •

t

THEOREM 22. Let p ::: 1. The mapping vp

:

L~(IL)

vp(f) = (Ix'/'p dlL

1-+

[0,00) defined by

)I IP

(129)

defines a norm on L~(IL). ProoJ-

Clearly, vp(cxf) = Icxlvp(f) for all cx

UX

E

C, I

E L~(IL).

Also, (vp(f) = 0) I/IP dlLjlP = 0 ===> (f = 0). Finally, vp(f + g) S vp(f) + vp(g) for all I, g E L~(IL), by Minkowski's inequality. Therefore, vp is a norm. •

Riesz-Fischer Theorem We shall denote vp(f) by IIfll p. Thus L~(IL) is a normed vector space over C, with the norm 11 . 11 p and the induced distance

dp(f, g)

= 111 -

gllp.

THEOREM 23. Let p ::: 1. The distance d p makes 01 L~ a complete normed space.

In other words, L~(IL) is a Banach space for the norm ProoJ-

11 .

I p•

To show completeness one must prove that for any Cauchy sequence

(fn}n~1 of L~(IL) there exists I E L~(IL) such that limntoo dp(fn, f) = O.

Since {fn}n~1 is a Cauchy sequence (that is, limm,ntoo dp(fn, Im) = 0), one can select a subsequence (fn,}i~1 such that

dp(fni+l - In) S Ti. Let k

gk

=L

i=1

I/ni+l - Inil,

(*)

260

Appendix 00

L

g=

;=1

I/ni+' - Ini I·

By (*) and Minkowski's inequality we have IIgk I p :s 1. Fatou's lemma applied to the sequences {gfk~:1 gives IIgli p :s 1. In particular, any member of the equivalence c1ass of g is finite /L-almost everywhere, and therefore

L (jni+' (X) 00

In, (X) +

ln/x))

;=1

converges absolutely for /L-almost all x. Call this limit I(x) (set I(x) this limit does not exist). Since

= 0 when

k-I

In,

+L

;=1

(jni+' - Ini) = Ink'

we see that

I = lim Ink /L-a.e. ktoo

One must show that I is the limit in Lt(/L) of Unkk~:I' Let e > O. There exists an integer n = N(e) such that II/n - Im I p :s e whenever m, n 2: N. For all m > N, by Fatou's lemma we have

Jx[ 1I -

ImIPd/L:S liminfll/ni - ImIPd/L:S e P •

I~OO

x

Therefore, I - Im E Lt(/L), and consequently, I the last inequality that lim

m--+oo

111 - Imllp

E

Lt(/L). It also follows from

= O.

•

Terminology. For p 2: 1, Lt(/L) is a Banach space (a complete normed vector space) over Co This phrase will implicitly assume that the norm is defined as in (129). When /L is the Lebesgue measure on jRn, we write Lt(jRn) instead of Lt(/L) (with a slight symbolic inconsistency). In the proof of Theorem 23 we obtained the following result. THEOREM 24. Let Unk:1 be a convergent sequence in Lt(/L), where p 2: 1, and let I be the limit. A subsequence {/ni k::1 can then be chosen such that

lim f,n

;too

'

=I

/L- a.e.

(130)

Note that the statement in (130) is about functions and not about equivalence c1asses. The functions thereof are any members of the corresponding equivalence c1ass. In particular, since when a given sequence of functions converges /L-a.e. to two functions, these two functions are necessarily equal/L-a.e.

References THEOREM 25.

If{fnln:o:! converges both to f in L~{J.1) and to g f-L-a.e., then f

261

=g

f-L-a.e.

A most interesting special case of LP -space, particularly in view of its relevance to signal processing, is when p = 2. In this case THEOREM

26. L~(f-L) is a complete normed space with respect to the norm

IIfII =

[Ix

Ifl 2 df-L

f/2

This norm is derived from a Hermitian product, namely, (j, g) =

Ix

fg* df-L

in the sense that

IIfII 2

= (j, f).

L~(f-L) is a Hilbert space over C (see Section CI·I).

Approximation Theorems We now quote the approximation results used in the main text.

27. Let f E L~(lR.), P ~ 1. There exists a sequence {fnln:o:! of continuous functions fn : IR t-+ C with compact support that converges to f in

THEOREM

L~(IR).

(To have compact support means, for a continuous function, to be null outside some c10sed bounded interval.) Let f E L~(IR), P ~ 1. There exists a sequence {fnln:o:! offunctions fn : IR t-+ C which are finite linear combinations ofindicatorfunctions ofintervals, that converges to f in L~(IR).

THEOREM 28.

29. Let fE Lt([-n, +nD be a 2n-periodicfunction (that is, f(t) = f(t + 2n)forall t E IR, and J~: If(t)1 d t < (0). There exists a sequence {fnln:o:! of functions fn : IR t-+ C with continuous derivatives that converges to f in Lt([ -n, +n D.

THEOREM

References [Dl] [D2] [D3] [D4] [D5]

de Barra, G. (1981). Measure Theory and Integration, EIlis Horwood: Chichester. Halmos, P.R. (1950). Measure Theory, Van Nostrand: New York. Royden, H.L. (1988). Real Analysis, 3rd ed., MacMillan: London. Rudin, W. (1966). Real and ComplexAnalysis, McGraw-Hill: New York. Taylor, A.E. (1965). General Theory of Functions and Integration, Blaisdell, Waltham, MA, Dover edition, 1985.

Glossary of Symbols

P(X), the collection of all subsets of set X.

card (X), or

lXI, the cardinal of set x; the number of elements in X.

N, the integers. N+, the positive integers. Z, the relative integers. IR, the reals. IR+, the positive reals. C, the complex numbers.

z*, the complex conjugate of z

C.

E

(a, b], interval of IR open to the left, c10sed to the right; and similar notation for the other types of intervals. Re(z), the real part of z E C.

Im(z), the imaginary part of z

J : IR

1-+

E

C.

C, a function from IR to C; equivalent notation: J, JO, J(t).

J(n), the nth derivative of J; J(O)

= J.

J * g, the convolution product of J and g: Cf * g)(t) =

L

J(t - s)g(s)ds =

L

g(t - s) J(s)ds.

264

Glossary of Symbols

f*n, the nth convolution product of 1 by itself: 1*° = I; 1*(n+1) = 1

* f*n.

lA, the indicator function of a set A; lA (t) = 1 if t h(t)

E

A, =

°

otherwise.

= l(t)l[o,Tj(t).

sinc (f) = sin(:rrt)/m, the cardinal sine function. rectT (t)

=

l[_~,+~j(t),

the rectangle function.

l, the Lebesgue measure on lR; l([a, b])

=b-

a.

a.e., almost everywhere with respect to the Lebesgue measure. f-L-a.e., almost everywhere with respect to the measure f-L. L~(lR), the set (equivalence c1asses) of measurable functions

that

flR

I/(t)IP dt <

f:

1 : lR

L~([a, b]), the set (equivalence c1asses) ofmeasurable functions

such that

~ C such

00.

I/(t)IP dt <

1 : Ca, b]

~ C

00.

L~,loc(lR), the set (equivalence c1asses) of measurable functions that I(t)l[a,bj(t) E L~(lR) for all Ca, b] c lR.

1 : lR ~

C such

Lfoc' short for L~,loc(lR). l~(Z), the set of complex sequences

{Xn}nEZ

such that

LnEZ IXn 12

<

00.

Cn , the set of ntimes continuously differentiable functions

1 : lR ~ C.

COO, the set of infinitely differentiable functions 1 : lR

C.

~

1 : lR ~ C. C~, the set of continuous functions 1 : lR ~ C with bounded support. C([O, T]), the set of continuous functions 1 : [0, T] ~ C. Co, the set of continuous functions

V, the set oftest functions q; : lR

C; C oo and compact support.

~

V', the set of distributions on lR; the set of linear forms on V. S, the set of functions

1 : [0, T]

~

C in C oo and with all its derivatives rapidly

decreasing. Sr. the set of functions 1 : [0, T] order r rapidly decreasing.

~

C in

er and with all its derivatives up to

S', the set of tempered distributions on lR; the set oflinear forms on S. (x, Y) H, the Hermitian product of x, y EH, H Hilbert space.

IIx 11 H

= (x, x) H )1/2, the norm ofx EH, H Hilbert space.

265

x..ly; x is orthogonal to y; (x, y) H = O. G.L, the orthogonal complement of G. PG, the orthogonal projection on G.

= (I/2n) J~rr f(t)e- int dt, the nth Fourier coefficient of f. Sn(f) = L~: ck(f)e+ ikt , the Fourier series.

cn(f)

S(f)

= LnEZ cn(f)e+ int , the formal Fourier series development.

j(v)

= JR. f(t)e-2irrvt dt, the Fourier transform of f.

H(z)

= LnEZ hnz n, the z-transform of {hn}nEZ,

CPj,n(t)

= 2 j / 2cp(2 j t -

n).

Index

Aliasing, 79, 100 all-pass, 110 almost everywhere, 248 amplitude gain, 57 analytic signal, 69 autocorrelation function, 14, 156 autoreproducing Hilbert space, 230 B-splines, 223 band-pass, 58 base-band, 68 Bessel's inequality, 146 biorthonormal system, 149 Borel function, 244 Borel set, 244 Borel sigma-field, 244 bounded support, 8 bounded variation, 36 Butterworth filter, 67 C.dJ., 246 cardinal sine, 10 Cauchy sequence, 135 causal filter, 55, 101, 109 complete, 135 complex envelope, 69 complex signal, 7

convolution, 14 convolution-multiplication rule, 14,25 convolutional filter, 55, 101 counting measure, 246 cut-off frequency, 58 Decomposable signal, 61 dense, 138 differentiating filter, 59 dilation equation, 209 Dini's theorem, 34 Dirac comb, 91 Dirac measure, 246 Dirichlet integral, 32 Dirichlet kerneI, 32, 36 dispersive channel, 72 distance, 135 distribution function, 246 dominated convergence, 252 down-sampling, 118 Elementary Borel function, 249 Fejer's idenlity, 110 Fejer's kerneI, 40 Fejer's lemma, 112 Fatou's lemma, 252

268

Index

feedback filter, 60, 105 finite-energy signal, 8 finite measure, 246 finite-power signal, 24 FIR,115 formal Fourier series, 31 Fourier coefficient, 24 Fourier inversion formula, 17 Fourier sum, 100 Fourier transform, 9 Franklin's wavelet, 222 frequency response, 56, 101 frequency transposition, 69 Ff,9 Fubini's theorem, 255 Gabor transform, 178 Gaussian pulse, 10 Gibbs's phenomenon, 38 Gram-Schmidt,145 group delay, 73 Hölder's inequality, 257 Haar filter, 121 Haar mother wavelet, 202 Haar wavelet, 192 Heisenberg's inequality, 176 Hermitian product, 133 Hilbert basis theorem, 148 Hilbert space, 135 Hilbert span, 136 Hilbert subspace, 136 Hilbert's filter, 58 Impulse response, 55, 101 indicator function, 8 integrating filter, 59 integration by parts, 256 isometry extension theorem, 138 isomorphic, 137 Jordan's theorem, 36 Kernel of an MRA, 229 Lebesgue measure, 246 Lebesgue-Stieltjes integral, 256 linear form, 143 linear isometry, 137

localization principle, 33 locally integrable, 8, 23 locally square-integrable, 8 locally stable, 23 low-pass, 58 Mallat's algorithm, 211 measurable function, 244 measure, 246 measure space, 246 metric space, 135 Mexican hat, 192 Meyer's wavelet, 218 Minkowski's inequality, 258 monotone convergence, 252 Morlet's pseudo-wavelet, 192 mother wavelet, 185,201 MRA,196 multiresolution analysis, 196 Norm, 134 norm of a linear form, 143 Nyquist condition, 85 Octave band filter, 122 orthogonal, 134 orthogonal complement, 139 orthogonalprojection,140 orthogonalsum, 143 orthonormal system, 145 Parallelogram identity, 134 partial response signaling, 87 periodic signal, 23 phase, 57 phase delay, 73 Plancherel-Parseval identity, 157, 162 Poisson sum formula, 91 polarization identity, 134 pre-Hilbert space, 133 probability measure, 246 product measure, 254 product sigma-field, 254 projection principle, 141 projection theorem, 140 pulse amplitude modulation, 84 Pythagoras' theorem, 134

QMF,120

Index quadrature components, 68 quadrature mirror filter, 120 quadrature multiplexing, 70 quasi-positive delta sequence, 231 Radon measure, 246 realizable filter, 55, 101 rectangular pulse, 9 regularization lemma, 19 regularizing function, 19 reproducing kernel, 230 residue theorem, 105 resolution level, 196 Riesz basis, 164 Riesz's representation theorem, 144 Riesz-Fischer theorem, 259 root mean-square width, 176 Sampie and hold, 78 scalar product, 133 scaling function, 196 Schwarz inequality, 134 Shannon wavelet, 210 short-time Fourier transform, 178 sigma-additivity, 246

sigma-fie1d, 243 sigma-finite measure, 246 spectral decomposition, 62 spectral factorization, 112 spectrum folding, 79 stable, 7, 100 synchronous detection, 70 Tonelli's theorem, 255 total, 148 transfer function, 104 triangular pulse, 14 Uncertainty princip1e, 175 up-sampling, 118 Wavelet orthonormal basis, 201 Weierstrass theorem, 41 WFT,178 WFT inversion formula, 179 window function, 169, 178 windowed Fourier transform, 178 WT, 185 Z-transform, 104

269