An Introduction To Frames and Riesz Bases (PDFDrive) PDF

 

Applie Ap plied d  and N um ume erica ricall  H arm on ic A nal alys ysis is S e rie s

d ito r 

John Jo hn   J  Be Benede nedetto tto Univer Un iversity sity o  off Ma Mary ryland land  

E d it o ri a l A d v i s o r y B o a r d

k ram ram    ldr ldroub oubii NIH,, B iomed NIH iomediical Eng En g ineeri ineerin n g/ Instr nstrum umenta entatio tion n Ingrid D aubec Ingrid ubech hies Princceton U niver Prin niverssity

Do uglas C ochr Dou ochra an Arizona rizona   St State ate   Univer Universsity H ans ans   G Feichtin Fe ichting ger Univ Un iversity ersity of Vien Vienn na M u rat ratKu Ku nt

Ch ristoph Chris topher er   H eil G eor eorgia gia   Institu nstitute te   of T ec echnol hnolog ogyy

Sw iss   Fe Swiss Fed deral In Inst stitute itute o f T ech echnolo nology, gy,   Lausa Lausanne nne

Jam es  James   McC Mc Clellan Georgi Ge orgia a Instit Institut ute e o f Techno chnolo logy gy

Wim S weld welden ens s Luce ucent nt T ec echno hnolog logies ies Be Bellll Labo Labora ratories tories  

M icha ichael el   U nser nser    NIH,, B iome NIH iomed d ical Engineer En gineering/ ing/ Instrum Ins trumen entatio tation n Victor W icker ickerh hause auserr W as ashi hington ngton   U nivers niversitityy M 

M artin Vette Vetterl rlii S wiss wiss   Fede Federral Insti Institu tute te of Tec Techn hnolog ology, y, Laus Lausanne

 

nic c al H armo armoni Num m eric erical   pplied and Nu

lysis is nalys na

Publish Publis h e d titles ATLAB MATLAB ntial Equations E quations with M Partial tial Differe Differential Introduction oduction to Par J. J.M M . Cooper: Intr 967 -5) 0-8176-3 76-3967 (ISBN   0-81 (ISBN an d Harmonic C.E. Theor eoryy and and  (ISBN E .M. Feman Wavelet let Th DdAttellis Attell is and Femandez-Be dez-Berdaguer: rdaguer:   Wave Harmonic Analysis in 3-5) 0-8176-395 0-8176 -3953-5) Sciences Sciences  Ap plied plie

Strohmer:   G abo r Analysis inger and  nd   T. Strohmer: H.G. Feicht Feichtinger 6-3959-4)   -8176-3959-4) (ISBN 0 -817

nd

Algorithms   Algorithms

ier Transforms a nd J.C.   W illiams: Four Fo urier T.M. T.M. Peters, J.H.T. Bates, G .B. Pike, P. M unger, and J.C. 176-3941-1) -1) N 0- 8 176-3941 (ISBN Biomedicall E ngineering (ISB Biomedica s Science nces ions in the the   Physical and E ngineering Scie Distribut tributions oyczynss tU U ==

L: 1u, / k ) l

2.

k=1

llglll =   1   s u c h T h e   u n i t b a l l i n   W   is  co m p a c t , so w e  c a n   fi nd   g E W w i t h llgl that 

A   = ~   l(g, / k ) l

2

=  i n f

{ ~   l ( f ,  h ) l

giv v en   f ow   gi is   c l e a r   t h a t   A >   0.  N ow I t  is

= ~ I  

k)l 2  /k l ( f ,  /

E W,

2

:

ll f l l

fE W ,

= 

} ·  

0,  w e h a v e   f :f. :f. 0, 

l l j l l h ) l 2   l l f ll 2  

~ A  

ll f l l 2 · 

0 

k} k '= 1 i n V   i s   a f r a m e   f or   V   i f C o r o ll a r y   1   1   3   A   f a m i l y   o f e l e m e n t s  { f k}k = V. n l y  i f s p a n { fk } and o C o r o l l a r y   1 . 1 .3   sh o w s   t h a t a   f r a m e   m i g h t c o n t a i n   m o r e e l e m e n t s   t h an   fo r V   a n d   { g k } k = 1 {fk k }k '= 1  is a   fr a m e for n e e d e d t o   b e a   ba si s . I n   pa r t i c u l a r ,  i f {f

is  fk } k'= 1  U   { g k } k = 1 is  is a n a rb i tr a ry   fi n it e   co ll e c ti o n o f v e c t o r s i n   V , t h e n   {fk} is  s a i d to   b e o v e r c o m p l e t e   is   n ot   a   ba s is   is  al s o a   fr a m e f or V .   A f r a m e   w h ic h   is or  re d u n d a n t.  C o n s i d e r  no w   a v e c t o r s p a c e V  e q u i p p e d w i t h   a f r a m e   {fk} k '= 1  a n d d ef i n e   a   l in e a r m a p p i n g m

T:

em  

7 

V ,  T { c k } k = 1   =

L  c k f k ·  

(1 .4 )

k=1

o r.  r.   T h e   r1,t ,to T is u s u a l l y   ca l l e d t h e   p r e - f r a m e o p e r a to r , o r   th e   s y n t h e s i s  op e r1 a d j o i n t o p e r a t o r  is g iv e n b y  

T*: V 

t

em, em,

T   f =   { ( ,  f k ) } ; : 1 ,  

1 .5 )  

an d   is c a ll e d t h e   a n a l y s i s   op e ra to r . B y   c o m p o s i n g   T w i t h i t s   a d j o i n t T* ,   w e o b t a i n  t h e   f r a m e   o p e r a t o r m 

S :  V - t V ,   S f =  T T *  f

f, fk /k.k.   =  L  f, 

1 .6 )

k=1

N o t e   t h a t i n t e r m s   o f  t h e   f ra m e   o p e r a t o r ,   m

L 

k ) l 2 ,  f E V ;  /k l ( f ,  / (Sf, f ) =   k=1

1 .7 )  

 

1.1 S o m e   b asi c   fa cts ab o u t fr a m e s  

5

ered   as s o m e k i n d o f " lo w er   er   f ra m e   co n d i t ion c a n t h u s b e c o n s id ered the low lo w er  b o u n d "   o n  the f r a m e o p e r a t o r .

i.ee., i f nition n ,   i. oose   A = B in t he d e f i nitio we   c a n ch oose }k=l   is tigh t i f we A   fram e   { f k }k=l m 

l(f, l( f, h ) l 2 = A  l l f l l 2 ,

Vf E

k= l

v.

( 1.8 1. 8 )

e  alled d the f r a m e  (1 .8) is s i m p ly   calle alu e   A in   (1. F o r   a  t i g h t   f r am e , th e e x a c t   v alu b ound. ound.   W e n o t e t h a t (1. (1.  7) c o m b in e d w i t h   L e m m a A . 6 . 6 i m m e d i a t ely l e a d s t i o n o f  f E V   in te r m s o f t h e f r a m e   el e m e n ts: t o a r e p r e s e n t a ti ame fra f o r V w ith fr ight fra fr a m e fo ition   1 1   4   A ss u me t h a t { f k } k = l is a ttight roposition P ropos  ),, a n d tity o p erato r o n V  ) identity the   iden b o und A . T h e n   S = A I (h ere I is the  1

f

m

= A  I : U , f k ) f k ,

V f  E

v.

(1.9 (1 .9))

k=l

 framee  a n d we w a n t t o   is  a  t i g h t  fram is   th a t if { k } ~ is  (1 .9)) is A n i n t e r p r e t a t i o n of (1.9 fine   we   c a n s i m p l y  d efine ;;= = l c k h , we I:;; inatii o n f = I: lineaa r  c o m b inat as   a line e x p r e s s   f E  V   as presee n t a  (f,  g k). F o r m u l a   (1.9) is s i m i l a r t o  the r e pres gk = fk a n d   t a k e ck =  (f, is  the f a c tor 1 / A  in   fferenc nc e is  basis: s: t h e   only d i ffere t i o n (1 .2 ) v i a   a n   o rt h o n o r m a l basi eral   f ra m e s we no now w p r o v e t h a t w e  still still   ha v e   a r e p r e s e n ta tatio n  (1 . 9). F o r g e n eral for   an a p p r o p r i a t e c ho ice o f   :;;=ll (f, gk) h for o f e a c h f E V o f  th e  fo rm f = I :;;= ults   ab o u t is   o n e o f t h e   m ost i m po r tant r e s ults { g k } k = l · T h e o b t a i n e d t h e o r e m is positii on: decomposit alled   the fra m e   decom is  c alled (1..10 ) is  fram e s , a n d (1 m   1 1   5 Let { eorem T h eore

k } ~

m e o p e ra to r S . be a f r a m e f o r V   with   fra me

Then  l f-adjo int. rt ible a n d s e lf-adjo ( i )   S is in v e rtible d   as ( i i) E v e ry f E V c a n be repres e n te d  m

m

(1 .1 0 )

f = I : u , s - 1 h ) f k =   I : U , f k ) s - 1 fk . k=ll k=

k=l

r e se n ta tio n f = thee re p re (iii)) I f f  E V also h as th (iii

k=l,l,   t h e n   ck}  k= nts {  ck}  efficients c o efficie m 

I: I:;;=l ;;=l   c k fk   f o r

a lar s o m e sc alar

m

m 

S - 1 h ) l 2 + L i c k - ( f , S - 1 fk W ·  l(f,S L l c k l 2 = L l(f, k=l k=ll k= k=l k=l w p ro v e now int.   W e no lf-ad d jo int. is   c le a r t h a t  S is s e lf-a Sincee S =   T T * , it is Sinc ive.   L e t f   E  V a n d a s s u m e   t h a t S f =  0. T h e n   injecc t ive. t h a t S is inje

Proof

m

0 =  (S f,f) = L I U ,fk W ,

k=l

 

6

1.  1.  F r a m e s i n F initeinite-d d i m e ns n s i o n a l  l   I n n e r  r   Prod Produ u c t S p a ces

i m plyi plyin n g   b y  t h e  f r a m e c o n d i t i o n t h a t   f =  0. T h a t S is i n j ectiv ectivee   a c tu a l l y im p li e s t h a t S is  is  s u r j e ctiv ctivee ,   b u t  l e t u s give give   a  d i re c t p r o o f . T h e  f r a m e c o n 

d i t i o n i m plies plies   by C o rolla r y   1 .1.3 .1.3   t h a t s p a n { f k } ~ =   V , so  so   t h e p r e-fr e-fraa m e op erato eratorr  T i s s urje urjecc t ive. ive.   G iven iven   f   E V we c a n t h e r efor eforee find find   g E   V s u c h  t h a t Tg   = f ; we c a n c h o o s e g   E N / = R r - , so so   it follo fo llow w s t h a t R s   = R r r - =  V . Th u s S   is is   s u r j e ctiv ctivee , as c la im e d .   E ac h   f E V   h a s th e r e p r e s e n t a ti tio n

ss- 1J

J 

TT*S- 1 f m

2)s-1J,h )h ;

k=1   k=1

usin g that S   is s e lf-a lf-ad d j oint oint,, w e a r r i v e at   m

1

L _ u,s- 1 h ) h .

= 

k=1 k=1

T h e s e c o n d r e p r e s e n tati tatio o n  in  i n (1.10 (1 .10)) is  is  o b t a i n e d  i n  t h e s a m e   way, u s i n g  t h a t f = s - 1 S f . F o r th e   p r o o f o f (iii), (iii),   su p p o s e that that   f =  I: Z '= 1 c k h ·  W e c a n w rite

{(f,S {(f, S - 1h) } k = 1 +   { ( f , S - 1h )} k '= 1 ·

{c {ck} k}k= k=1 1  =   { c k } ~ 1  

B y   t he c hoic hoicee o f {  ck} ck } k= 1 w e h a v e m 

L  ( c k -

( f , S - 1 f k ) ) fk = 0,

k=1 k=1

{ ( f , S - 1fk)} fk)}k' k'= =1

i .e., { c k } ~

{ ( ,

s - 1h )

~ 1  

E

Nr

={

= R ;j . ;

s ince ince  

S - 1 J, h )  }k'=1 E   R r · ,

we ob ta in (iii). (iii).  

0 

E v e r y   f r am e in   a  f init in it e -dim e nsio nsion n al sp a c e c o n tain s a s u b f a m i l y   w hic h is is   a b a sis ( E x e r c i s e 1.1 1. 1 ). I f { fdk'= 1   is  is  a   f r am e b ut   n o t a   b as i s, th e r e e x i s t n o n-ze n-zerr o   s e qu e n c e s {dk}k dk}k'= '= 1 s u c h t h a t I: Z '= 1  d k h   = 0. T h e r e f o r e f E   V c a n b e w r itte n m

m

k=1

k=1

m

L. ( f ,  s -

k=1 sh o w ing   that

h ) + dk) h ,  

1 

f

h a s   m a n y r e p re r e s e n t a t i o n s   as s u p e r p o s i tion s o f t he   f r a m e 1 k)  }k'= 1 h a v e e le m e n ts. T h e o r e m   1 .1. .1.5 5 sh o w s t h a t th e   c o e ff i cien cients ts   { ( f , m i n i m a l £2 - n o r m a m o n g   all s e q u e n c e s {ck} {ck}k'= k'= 1 for fo r w h i c h f = I :Z '= 1  c k h ·

s-

 

1.1 Some basic facts about frames

The numbers ( , s - 1 fk),

k

= 1, ... , m

7

are called frame coefficients. Note that because S : V --+ V is bijective, the sequence { S - 1 k}k : 1 is also a frame by Corollary 1.1.3; it is called the canonical dual of { k}/:'= 1 . Theorem 1.1.5 gives some special information in case { k}/:'= 1 is a basis: k= 1 is a basis for V . Then there exists {fk} that a unique family {gk}/:'= 1 in V such

Corollary 1.1.6 Assume that

f

=

m

L U , g k ) k , 'r/f E

k=1

In terms o f the frame operator, {gk}k : 1 (fi,gk) = 8j,k·

v.

=

{1.11) { S - 1 k}/:'= 1 . Furthermore

The existence of a family {gk}/:'= 1 satisfying {1.11) follows from Theorem Theor em 1.1.5; 1.1.5; we le leav avee the t he proof of the uniqueness uniqueness to the reader. Applying {1.11) on a fixed element f i and using that { k}/:'= 1 is a basis, we obtain that (fi, 9k) = 8j,k for all k = 1, 2, · · · , m. D

Proof.

W e can give an intuitive explanation of why frames are important in signal transmission. A more detailed argument is given in [105]. Let us assume that we want to transmit the signal f belonging to a vector space V from a transmitter A t o a receiver n. I f both A and R have knowledge of a frame { k }/:'= 1 for V, this can be done if A transmits the frame coefficients { ( , s - 1 k) }/:'=1 ; based on knowledge of these numbers, the receiver n can reconstruct the signal fusing the frame decomposition. Now assume that n receives a noisy signal, meaning a perturbation { ( , s - 1 k) + ck}/:'= 1 of the correct frame coefficients. Based on the received coefficients, R will claim that the transmitted signal was m

L

(( , s - 1 k) + ck)

k=1

k

=

m

LU,

k=1

s - 1 fk)

k

+

m

L ck k

=f +

k=1

m

L ckfk;

k=1

this differs from the correct signal f by the noise : ~ = ckfk· I f { k}/:'= 1 is overcomplete, the pre-frame operator T{ck}/:'= 1 = : ~ = ckfk has a non trivial kernel, implying that parts of the noise contribution might add up t o zero and cancel. This will never happen if { k}/:'= 1 is an orthonormal basis In that case ckfkll 2 = : ~ = so each noise contribution : ~ = will make the reconstruction worse. W e have already seen that, for given f E V, the frame coefficients { ( , 1 k) }/:'=1 have minimal £2-norm among all sequences {ck}k : 1 for which f = : ~ = ckfk· W e can also choose to minimize the norm in other spaces than £2 ; we now show the existence of coefficients minimizing the £1 -norm.

s-

 

8

1. Frames in Finite-dimensional Inner Product Spaces

Theorem 1.1.7 Let { f k } ~ be a frame for a finite-dimensional vector space V . Given f E V , there exist coefficients {dk}k= 1 E em such that

f

=

dk/k, and

m

{ ; ldkl Proof.

Fix

f

= nf

{

m

{;hi : f

=[ ; c d k m

}

.

(1.12)

E V. I t is clear that we can choose a set of coefficients

to we can ckfk; let r :it= isE;;'= {ck}k=1 {ck}k= 1 such E;;'=1 want 1-norm hi· Since minimize also1 clear that we now the fthat f =of the coefficients, restrict our search for a minimizer to sequences {dk}k= 1 belonging to the compact set

E

M := { { d k } ~

em

: ldkl $

r, k = 1, . . . ,m}.

Now the result follows from the fact that the set

{{

d k } ~

EM I f =

is compact and that the function¢: continuous.

k f k }

em -+

IR, ¢{ddr= 1 : = E;;'= 1 ldkl is

0

There are some important differences between Theorem 1.1.5 and Theo rem 1.1. 7. In Theorem 1.1.5 we find the sequence minimizing the f 2 -norm of the coefficients in the expansion o f f explicitly; it is unique, and i t depends linearly on f. On the other hand, Theorem 1.1.7 only gives the existence of an f 1-minimizer, and it might not be unique (Exercise 1.4). Even if the minimizer is unique, it might not depend linearly on f (Exercise 1.5). An algorithm to find an f 1-minimizer {dk}k= 1 can be found in [64]. As we have seen in Proposition 1.1.2, every finite set of vectors {/k}k= 1 is a frame for its span. If span{fk}k= 1 -:f. V, the frame decomposition associated with {/k}k= 1 gives a convenient expression for the orthogonal projection onto s p n { k } ~ The or e m 1.1.8 Let {fk}k= {fk}k=1 1 be a frame for a subspace W o f the vector space V . Then the orthogonal projection o f V onto W is given by m

Pf

= Lu,s- 1 /k)fk.

(1.13)

k=1

Proof.

I t is enough to prove that if we define P by (1.13), then

Pf

=f

for f E W and P f

= 0 for

f E W .L.

The first equation follows by Theorem 1.1.5, and the second by the fact because s i s a bijection on w. 0 that the range of s - 1 equals

w

 

1.1 Some basic facts about frames

9

Example 1.1.9 Let {ekH=l be an orthonor or thonormal mal basis for for a two-dimensional

vector space V with inner product. Let

Then

{ f k } ~

l

is a frame for V . Using the definition of the frame operator, 3

Sf= L(f,fk)fk, k=l

we obtain that Se1

Se2

=

=

e1

+ e1

-(e1 -

+ e1 + e2 = 3el. e2) + e1 + e2 = 2e2.

- e2

Thus

Therefore the canonical dual frame is

Via Theorem 1.1.5, the representation of given by

f

f

E V in terms of the frame is

3

= 'LU, s-l k) k k=l

0

Let us for a moment consider an orthonormal basis {ek}k=l for V. I t is clear that by adding a finite collection of vectors to { ek} k=I we obtain a frame for V. Also, if we perturb the vectors { ek} k=l slightly, we still have a basis, but in general not an orthonormal basis. More precisely, if {gk}k=l is a family of vectors in V and

then also {gk}k=l is a basis for V. In fact, given a scalar sequence {ck}k=P the opposite triangle inequality followed by Cauchy-Schwarz' inequality

 

10

1. Frames in Finite-dimensional Inner Product Spaces

gives that

I ,c•g•ll

>

llt,c•eoll-llt,c,(g,- e,)ll

(t,1"1f'- (t,lle,-g,ll'f (t,lc·lf'

>

(1 - R)

(t, c•l

r

This shows that {gk}J:= 1 is linearly independent, and since dimV = n, we conclude that {gk}k=l is a basis. W e return to more general perturbation results for frames in Chapter 15. 0

1.2

Frame Fram e bounds bou nds and frame algorithms

The speed of convergence in numerical procedures involving a strictly posi tive definite definite matri mat rix x depends heavil heavily y on the condition number of the matrix,

which is defined as the ratio between the largest eigenvalue and the small est eigenvalue. In case of the frame operator, these eigenvalues correspond to the optimal frame bounds: The or e m 1.2.1 Let

be a frame for V . Then the following holds:

} ~

{i) {i) The opti optimal mal low lower er frame boun bound d is th thee small smallest est eigenvalue for S , and the optimal upper frame bound is the largest eigenvalue. (ii) Let {.\k}k=l denote the eigenvalues for S ; each eigenvalue appears in the list corresponding to its algebraic multiplicity. Then n

m

LAk = L:11hll

k=l

2

·

k=l

{iii) Assume that V has dimension n. I f {fk}k=l is tight and llfkll for all k, then the frame bound is A = m f n .

=1

Assume that {h}k'= 1 is a frame for V . Since the frame operator S : V - + Vis self-adjoint, Theorem A.2.1 shows that V has an orthonormal basis consisting of eigenvectors {ek}k=l for S. Denote the corresponding eigenvalues by {Ak}f:= 1 . Given f E V , we can write f = L ~ = ( J , e k ) e k . Then Proof.

Sf=

n

n

k=l

k=l

L:U, ek)Sek = L .\k(J, ek)ek,

 

go r i t hm s 1.2   F r a m e   bo u n ds an d  fram e   a l gor 1.2

a nd   m

n

\kl(ff , e k ) l 2 · L l ( f , f k ) l 2 =  ( S f , f ) =  L . \kl( k= l

k= k=ll

11  11 

f ore T h e r e fore m

minllfllll 2 Aminllf

:S  :S 

xllflll Ama axllfl k W   :S Am L   l(f l(f,f ,fk

2

·

k=l k= l

So Amin is a lower lo wer fra fr a m e b o u n d ,  ,   a n d Am e   b o u nd nd . T h a t Amaax is a n u p p e r f r a m e  e ctor t h ey a r e  th e o p t i m a l fram e b o u nds fo llows by tak i n g f t o be  an   e i g e n v ector x)· max)· vely Ama spectively o n d i n g  to  Amin ( r e specti c o r r e s p on we   h a v e For th e p r o o f o f (ii), we n 

n

n

.\ k l L , \ k  =  L .\k

k= l

k =l

e k ll 2   =  

L (S ek.ek )  k= l  m

n

LLI(ek,M I 2 · k= l  1 = 1

for   s is for    k} is an   o r t h o n o r m a l b a sis e s  and u sing t hat { ek} i n g th e  s u m s  I n t e r ch c h a n g in o ns im ply t h a t  th e (iii), th e  a s s u m p t i ons Fo r th e  p r o o f   of (iii), ( ii). For g ives (ii). V fin fi n ally gives ists of  of  th e f rame b o u n d  A r e p e a t e d  n   t i m es; consists \ k } ~ = l cons ues ei genvalues s e t o f eigenval D (ii).  follows from   (ii).  result   follows t h u s the result  f ra m e   fo r V . T h e n the c o n d i t i o n nu m ber C o r o l l a r y 1 . 2 . 2 L e t { f k }k=l be a  fr im a l u p p e r fra m e to   the th e r atio   be tw e e n th e o p t im f o r   t h e  f ra m e   op e r a t o r is   equ a l to  b o u n d an d  th e   o p t i m a l lo w e r f r a m e b o u n d .  

icientss coefficient  off th e  coeff ledge o n  knowledge n t f E   V b a s e d o n know elemen f i n d an   eleme I f w e w a n t t o fi 1.1 .5: e m 1.1.5: T h e o r em { ( f,fk f,fk)) } k = l   w e c a n   use Th m 

J=

L:u, h ) s - l

k

u, h ) } k = l · u,

=  s - lr {  

k= l 

u l we need t o  i n vert t h e   usefu a   t o  b e   usef his fo r m u l a  r   for tthis er, in o r d e r  owever, H owev fram e o p e r a t o r ,   which  which  c a n b e c o m p l i cated c ated if i f the d i m e nsion n sion o f V is  is   large. large. io n s of f. A   is t o u se an   a l g o r i t h m to  o b t a i n a p p r o x i m a t io A n o t h e r o p t i o n is th m : gorith t h m is know n as t h e  f r a m e a l gori ical a l g o r i th classical class k}k=ll   be a   f ra m e f o r   V   w ith f r a m e   bo u n d s   A , B . L e m m a  1 .2.3 L e t { f k}k= } ~ in V b y G i v e n   f E   V , d e f ine th e s e q u e n c e {

9o =   0,

2  9k 9k   = 9 k - l + A +  B S ( f - 9 k - d , k   2: 1.

Th en

9kll   :S I I - 9kll

A )k  )k 

B ( B  + A  

11 11 11·  11·

 

12 Proof.

1. Frames in Finite-dimensional Inner Product Spaces

Let I denote the identity operator on V. Using (1.7),

(1 . 14)

so via the frame condition, {(

I- A

Similarly,

2

B S

2

2A A+B

~ {

I

f , f } ~ l l f l l -

B- A - B +A

IIIII

2

2

IIIII =

B - A B+A

2

IIIII·

2

A + B S ) f , f).

The two inequalities and (A.8) together give that

III- A Bs Bslll

Using the definition of

f -

~ ~

{ g k } ~

9k

=

f -

=

(1- A

9k-1 - A

2

+ B S(f

- 9k- d

8 s)u-9k-d,

and by repeating the argument,

f - 9k

=

(1- -A +2Bs)k

f - go).

Thus, applying (A.6) and (A.7),

11(1- A Bs)'(f-90)11

ilf-g•ll

< <

III- A:A)k8W (B B+A

I I - 9oll

IIIII· 0

In particular, the vectors 9k in (1.14) converge to f as k oo. The algorithm depends on the knowledge of some frame bounds, and the guar anteed ante ed speed spe ed of conver convergence gence also depends on them. them . I f B is much larger than A (either because only bad estimates for the optimal bounds are known, or because the frame is far from being tight) the convergence might be slow.

 

1.2 Frame bounds and frame algorithms

13

I t is natural to apply some of the known acceleration algorithms from lin ear algebra to obtain faster convergence. Grochenig showed in [152] how to apply the Chebyshev method and the conjugate gradient method. W e begin with the Chebyshev method:

{fk}k : 1

The or e m 1.2.4 Let

and let

be a frame for V with frame boun bounds ds A , B ,

B-A

a :=

P := B + A ' Given f E V, define the sequence { A k } ~ by

= 0,

go

and Ak

=

{ g k } ~

= A

in

2

+ B Sf,

V and corresponding numbers

.A1

= 2,

2,

o r k ~

1-

91

.JB-v0f. ..fB + v0f."

, TAk-1

9k

= Ak (9k-1 -

gk-2

+ A + B S ( J - 9 k - d ) + gk-2·

Then

Th e Chebyshev algorithm guarantees The guaran tees a faster convergenc convergencee than the frame frame algorithm when B is much larger than A. Knowledge of some frame bounds is also needed in order to apply the Chebyshev algorithm. In contrast, the conjugate gradient algorithm described below works without knowledge of the frame bounds: only when we want to estimate the error I I - gkll do we need them. Following Grochenig, we formulate the result using the norm

lllflll = (J,Sf) 112 , f

E V.

W e leave i t to the reader t o check that Ill · Ill is in fact a norm on V. Remember also that all norms on a finite-dimensional vector space are equivalent; that is, there exist constants c1, c2 > 0 such that

This means that an error estimate in the norm into a n error estimate in the usual norm. The or e m 1.2.5 Let

the vectors

{ g k } ~

Udr=1

{ r k } ~

Ill · Ill can be transferred

be a frame for V . Let f E V \ {0} and define { P k } ~ _ by and numbers { A k } ~

9o = 0, ro =Po = S J, P-1 = 0

 

14

1.

Frames  Frames  i n F i n i t e - d i m e n s i o n a l In n er Pr o duct S p a c e s

a n d , f o r  r   k  2::: 0,

(r k , P k)

(Pk, S p k)' 9k

+  AkPk,

r k - )..kS )..kSPk, Pk, S k

( S p k , S p k) k) (Spk> S p k- 1 ) ( p k ,S , S P k ) P k - (Pk-1 (Pk-1,SPk-1 ,SPk-1)P )P k - 1 ·  

Then 9k - t f as k   - t oo. I f  w e let A den o t e t h e s m a l l e s t e i g e n va va lu e f o r S a n d B   the large large s t eigenvalu eigenval u e   a n d l e t a   = ~ ~ ~ t he speed o f c o n v e r g e nc nc e  

be  e s t i m a te c a n be  t e d   by

klll Ill - 9klll

2ak :S 1 + a 2 k

111 111·  111· 

I n the e x p r e ssion for for   Pk Pk+ + 1 , the l a s t   t e r m is i n t e r p r e t e d as zero for k

1.3

F ra m e s in

= 0.

en

T he   natural e x a m ples of fin fi n i t e-dimens e-dimensio io n al vector  vector   s paces are

lRn = { ( c1

, c2 , ... , Cn) I c;

E JR, i =   1 . . .  , n}

en

) 2 ) . . ) Cn) I C;

E C, i = 1 ) . . . ) n } ;

and = { ( c1

th e latter is e q u i p p e d w it i t h th e i n n e r  r   p ro d u c t n

({ck}k= ({c k}k=1' 1' {dk} {dk}k= k= 1) =

I::> kdk

k=1

a n d th e a s s o c ia ia t e d n o r m n

ll{ck}k=111 =  

lck lckl 2· L k=1   k=1

T h i s   correspon correspon d s t o the d e f initions in i n JRn, JRn,  e x c e p t that c o m pl p l e x  c o n j u g at at i o n a nd  m o dulus is not needed needed   in  in   t he real real   c ase. W e w ill  ill   describe  describe  t h e t h e o r y   f or bases  bases   and frames frames   in  in   en) b u t  e a sy modifi modif i c a tions give giv e th e c o r r e sp sponding result s i n ] R n .  .   If,  forr example, f,  fo examp le, { f k } k=1 is a frame  frame   for for en en))  th en the 2m   v e c tors tors consisti  consistin n g o f the r e a l p a r t s , r e sp s p e c tively th e i m a g i n a r y  p arts, of t h e f r a m e   vectors  vectors  w ill be a fr f r a m e for JRn JRn ( Exercise 1 .6 ); in pa rtic ular, if the vectors   { vectors } ~ h a v e   real coor coor d i nates, th e y c o n s t i t u te t e a   frame for for JRn. On   t h e o t h e r  r   h a n d a fram fra m e   for JRn is a u t o m a t ic i c a l ly a fram e for en ( E x e rcise 1 .7 ) .

 

1 .3 Fr a m e s i n   C "

15

8k   w h ere 8k nsists ts o f t he v e c tors {   k } ~ = l forr  en c o nsis T he c a n o n i c a l   b a si s   fo s e 0. W e w ill is th e v e c t o r in   en h a v i n g   1  a t t h e k -th e n t r y a n d   o t he r w i se asis.. this   basis a t i o n i n this en   with   th e i r r e p r e s e n t at ctorss i n en c o n s e q u e n t l y   i d en t i f y v e ctor valen n t   c o nd ition s  for lineaa r   a l ge b r a w e  kn o w   m an y   e qu i vale ntary y line F r o m  elem e ntar imporr tant lis t the m ost impo en.. L e t   us list asis   fo forr en  vectorr s   t o c o n s t i t u t e a   basis a s e t o f  vecto

c h a r a c t e r i z atio n s : ectorss   i n i d e r n v ector eorem m   1  3   1 C o n s id T h eore trix   a n n x n ma trix

en   a n d   w r ite   t h em   as en

c o l u m n s in

le nt: uivale lo wing   are e q uiva follo T h e n t h e fol basis f o r s t i t u t e a   basis tors)   c o n st {i} T h e   c o lu m n s i n  A   ( i .e., th e g i v e n v e c tors) basis for s t i t u t e a   basis ro w s i n A c o n st {ii} {i i}   T he row

en. en.

en. en.

- z e ro. is   n o n -z i n a n t o f  A is { i ii) T h e d e t e r m in (iv) A  is i n verti vertib b le.

en. en.. m a ppin g   o n en

n e s a n   injec ti v e m a ppin g   o n defin (v) A   defi definee s a s u rjec t i v e { vi) A defin

inearr l y ind e p e n d e nt. ar e l inea (vii) The c o l u m n s i n A   are al   to n.   eq ual {viii)) A h a s ran k   equ {viii

as   the d i m e n s i o n o f   its defin n e d as i x E is defi R e c a l l   t h a t t he  r a n k   o f a m a t r ix ed i n t o turne r a n g e R e .   W e al s o r e m i n d   the r e a d e r   t h a t a n y b a s is c a n   b e  turn i z a tion   an o r t h o n o r m a l b a sis b y a p p lyin g t h e   G r a m - S c h m i d t o r t h o g o n a l iz p ro cedu re. ently   a t we c o nseq u ently en. . aNtor itex   th a t i o n s w ith r e p r e s e n t at r m t h e ien en   --+ em w ith th v   : en em  . i n   e n a n d em b a ses in anonii cal ba re sp ect to   the c anon re-frr a m e op e ra to r T m a p s   em en,,  t h e   pre-f forr en is  a fr a m e fo I n   c as e   { fk }k=l is  em   en   and em in  en en'' and i t s   m at r i x w i t h r e s p e c t   t o  t he c a n o n ical b a s e s   in  o n t o   en

W e n ow   tu r n t o a d is c u s sion sion   of fram fra m e s f or

a tors opera y   oper i d e n t i f y 

is

I 

( 1 .1 5 )

h

I 

colum m ns. ctorss ]k   a s colu i. i.ee ., t h e n x   m   m atrix h a v i n g the v e ctor

 

16

l   In ne r  Pr o d u ct S p a c e s - d i m e n s i o n a l  1.  Fr a m es in F i n i t e -d

ily essarily s pace, we n e c essar i onal space, i m e n s ional ors can ca n a t  m o s t s p a n an   m - d im vectors S ince m   vect  T   h a s a t l e a s t trix  fo r en, i.e., t h e  m a trix fra me for {fk}k k = l is a frame h a v e  e  m   2:: n w h e n {fk} rows. s   as rows. a s m a n y c o l u m n s 

en..  en

tors  fk c a n be th e v e c tors T h e n   the m 1 3 2 L e t  { f k } k=l be a f r a m e f o r eorem T h eore i t u tin g   natess o f s o m e v e c t o r s gk in  em c o n s t it rst n c o o r d i nate

nsidee r e d as th e   f i c o nsid

em.

is a ti g h t fra m e , t h e n th e v e c t o r s k  a re th e   f i rs t I f  { k } ~ n   c oo r dina te s o f  s o m e   v ec to r s gk i n em c o n s t i t u t i n g a n o r t h o g o n a l b as is fo r si s   f o r  a   b asis

em .

 f rame for rary  P r o o f   L e t { fk } k=l b e an   a rb it rary the m a p p i n g

en.. T he n m en

2::

n . C onsid e r

en-+  em em,,  F x  = { ( x , /k)}k ' = 1 · F :  en-+ forr  F w ith ratorr  T a n d th e  m a tr ix fo F is the a d j o i n t of t he p r e-fram e o pe rato matrii x w h e r e th e k - t h   row is ba ses is  the m   x n   matr i cal bases r e s p e c t t o   th e c a n o n ical fk , i.e., e  of fk, e x c o n j u g a t e of th e c o m p l ex

F=

h 

(

fm  

0

l 

=en, =en

it l  f k } k = l  11Fxlll 2 = :Z ::::;'= 1 I(x, fk ) l 2 . S ince s p a n { fk I f F x =  0, th e n 0 =  11Fx nd r efore   e x t e nd ping. W e c a n t h e refore ective   m a p ping. so   F is an   in j ective w s th a t x =   0, so  follow follo m ple, still le t t i n g { b k }k = l b e   forr  e x a mple, em:: fo bije ction F o f em o n t o em F t o a bijection fo r the o r t h o g o n a l bas is for em, let { ¢ k } k = n + l   be  a b a sis for t he  c a nonica l basis

¢k> k =   n t of R p   in em a nd e x t e n d F   by t he d e f initio n F b k   : = ¢k> c o m p l e m e nt fi rst n h ose first i x , w hose for   P is a n m   x m m a t r ix 2,  ... , m . T h e  ma t rix for n   + 1, n + 2, F:   fr o m F: colum n s a r e t he  colum ns fro

of  th e ro ws eq uals s  s p a n em. T he r a nk of  Since  P is  Since  is  surjec tive, t he c o l u m n s s m,   a nd t he y a r e als o th e ro w s in P   sp a n  em, ,   so als t he  r an k  o f th e c o l u m n s ,  bas is for em. st i t u t e a basis s, t h e y   c o n st .   T h u s, n d e n t .  line arly in d e p e nd {bk}}k'= 1   still n d A a n d  {bk frame   b o u nd for en w i t h frame r a m e for I f {fk {f k }k'= 1 is a t i g h t f ra s hows t h a t 1 .1.4 shows i t i o n 1.1.4 P r o p o s it basi s for d e n o t e s t h e  c a nonica l basis

en,

A b 'j,l, j , l = l ,  ... , n .   f o r T T * , so this in th e  m at r i x r e p r e s e n t a t i o n fo t , bj) is t h e j , l -th e n t r y in ( T T * b ' t, ( 1.15) for in   the m at r ix  r e p r e s e n t a t i o n (1.15) sho w s t h a t t h e n   ro ws in  c a l culati on show we   vectors in em. B y a d d i n g m   - n rows we as   vectors co nsider ed as  t h o g o n al, consider T a r e o r th

 

1.3 Frames in C"

17

can extend the matrix for T to an m x m matrix in which the rows are orthogonal. Therefore the columns columns are orthogonal. 0 Geometrically, Theorem 1.3.2 means that if Udr=l is a frame for C", there exist vectors {hk}r=l in em-n such that the columns in the matrix

(1.16)

+) m

I

constitute a basis for em. For a given m x n matrix A the following proposition gives a condition for the rows constituting a frame for C". Proposition 1.3.3 For an m x n matrix

Au A= (

A ~ the following are equivalent: (i) There There exists a constant cons tant A

>0

such that

n

A L ickl 2

IIA{ck}k'=III 2

k=l

,

'v'{ck}k=l

EC" ·

(ii) The The columns columns in A constitute a basis for their span in

em.

(iii} The rows rows in A constitute a frame for C".

Denote the columns in A by g 1 , . . . , gn; they are vectors in By definition, (i) means that for all {ck}k=l E C", Proof.

em.

(1.17) which is equivalent to {gk}k=l being a basis for its span in em (use an argument such as in the proof of Proposition 1.1.2). On the other hand, denoting the rows in A by ft, ... .. . , fm, (i) can also be written as n

A L

i c k i k=l

n

~

k=l

0

which is equivalent t o (iii).

 

18 18  

ct S p a ces Product Innerr  Produ ona l   Inne t e - d i m e n s i onal in   F i ni te 1.  1.  F r a m es in

.3 .3 , c o n s ider  t h e m a trix A s a n i l l u s tr a t i o n o f P r o p o s itio n   1 .3.3

A=o n, 

i t  is cle a r t h a t  th e  ro w s (

) , (

) , (

forr ( (/l. )   c on st i tu t e a   f r am e   fo

T h e  c o l u m n s (

~

)

""is   foe t h e i c s pa n in C ', b u t o ns t it u te a   b""is

, (

) 

io n a l s u b s p a c e   o f C3 . i m e n s io the sp a n is o n l y   a tw o - d im e   of the p r o o f  o f P r o p o s i t i o n   1.3.3 w e h a v e   As a n  i m m e d i a t e c o nseq u e n c e  t he fo l lo w in g u s e f u l f a c t: lary   1   3 4 Le t A   be a n   m x n C o r o l lary 9 1 , . . . , g n a n d the r o w s b y

co n st i tu t e a f r a m e f o r

en  en 

m atrix .

note   t h e D e note

rs   { f k } b 1  0 , the v e c t o rs w i t h b ound s A, B   i f  an d o n l y i f

h , . .. , fm ·

G i v e n A ,  B  

>

lc•l',   V {   . } ~ = '•9•1•11'1' oO B ~   lc•l', '•9

A ~   h i '  oO

c o l u m n s   by

E C"

E x a m p l e   1 3 5 C o n s i d e r   th e  v ecto r s  

( 

in C 3

.

~  

.  x  .  )   .   0 

0 

)

(

(

0

) 

f i ) ( - f i  )  

(1 .1 8 )

ider   th e m a trix C o r r e s p o n d i n g   t o   t he s e v e c t ors w e  c o n s ider 0 0

A = 

0

fi

-fi 

If  I f   - I1 f I f 0

0  0

If If

that   t he c o lu m n s { g k } ~ T he  r e a d e r c a n   c h ec k   that a ll   ha v e l e n g t h  

ji.

l   a re o r t h o g o n a l in   CS

and

T herefore

efinee d   r s d efin  t h e v e c to rs that  1.3.4  w e c o n clud e  that fo r   a ll c 1 , c 2 , c 3 E  C. B y  C or o llar y 1.3.4 for   C3   w i t h   f ra m e   b o un d   T h e f r a m e r a m e for titutt e  a t i g h t f ra 8 ) co ns titu (1.18 b y (1.1 0  is n o r m aliz e d .

 

1.4 The discrete Fourier transform

19

For later use we state a special case of Corollary 1.3.4 (Exercise 1.8): Corollary 1.3.6 Let

equivalent: {i) A* A

= I,

A be an m x n matrix. Then the following are

the n x n identity matrix.

{ii) The column columnss g1 ,

... ,

gn in A constitute an orthonormal system in em.

{iii) The rows It, ... , f m in A constitute a tight frame for bound equal to 1.

1.4

en

with frame

The discrete Fourier transform

When working with frames and bases in en one has to be particularly careful with the meaning of the notation. For example, we have used /k and g k to denote vectors in en, while Ck in general is the k-th COOrdinate of a sequence {ck}r=l E en, i.e., ck is a scalar. In order to avoid confusion we will change the notation slightly in this section. The key to the new notation is the observation that to have a sequence in en is equivalent to having a function f : {1, . . . ,n}-+ C;

the j-th entry in the sequence corresponds to the j-th function value f(j). Our purpose is to consider a special orthonormal basis for en. Given f E en we denote the coordinates of f with respect to the canonical or thonormal basis {8k}r=l by { (j)}j=t· For k = 1, ... , n we define vectors e k E en by ') -

1

ek (J - V n e

21ri(j-l)

k;_-1

, J• -- 1, . . . , n,.

(1.19)

that is

1 e2,.ik;_-• e4,.ik;_-•

The or e m 1.4.1 The

orthonormal basis for

(1.20)

, k = 1 , ... n.

defined by (1.19) constitute an

vectors {ek}k=l

en .

Since {ek}k=l are n vectors in an n-dimensional vector space, it is enough to prove that they constitute an orthonormal system. I t is clear Proof.

 

20  20  

that

1.  F r a m es in  in   Finite - d i m en e n s i o n a l In n e r Prod Product uct S p aces

lllle ekll

= 1  for al alll k . N ow, g iven k =f.£ =f.£,,

(e k , ee ) = -1  L n n

e

. . k-l 2 rrt(J-1 )n

e

.

.

t

l

-2 rrt(J-1 )n

j=1 

U si n g th e f o r m ula ( 1 - x )(1

+ x +  · · ·  +  x n - 1 )

1 

=-   n

1

··

k- t

e 2 1 r t )n -

.

j=O  

=   1 - xn w i t h   x   =   e 2  

i k ; ; - t

we g e t 0

T h e b a s i s { ek ek}k= }k= 1 is c a l le d t h e   d i s c r e t e F o u r i e r tran transform sform    b a si s . Usin Using g  this   basis basis,, ever every y   s e qu e nce f E en h a s   a  r e pr e s e n t a tion  

W r i t t e n o u t   i n co ordin ordinaa tes, t h i s m e ans that

f (j)

=

n

t t 

f ( £ ) e - 2 r r i ( l - 1 )  k; k;;;-

1 e 2 r r i ( j - 1 ) k;;: 1

k = 1 l= 1  

t t 

f ( £ )e2r )e2rrr i( j-l)

k ;;:l

j =   1, . . .  , n.

k =1 l=1  

A p p lica licatt i o n s o f te n as a s k for  for  ti g h t fr a m e s beca u s e th e c u m b e rs om o m e inve inv e rsio rsion n o f th e fr a me opera operator tor is a void voidee d   in th is case ca se , see  see   ( 1. 9) . I t is  is  i n t e r es e s ting  that

o v e r c o m p l ete ti t i g h t f ra mes c a n b e o b t a i n e d i n   en en   b y pro j e c tin g  t h e  d iscre t e F o u rier t r a n s f o r m b a sis in i n a ny e m , m   > n , onto   en en::

P r o p o s itio ition n   1 4 2 Let m

> n

en  by  

a n d d e f i n e th e   v ec t o r s {f {fk k } k'= 1 i n  

k

T he n {  

} ~

to   on e ,   an d

=  1 ,2 ,  . . . , m .

i s a   t i gh t   o ve r c o m p l e t e f r a m e f o r  

llfkllll llfk

~ f o r   a ll k.

en

w i t h   f ra m e   b o u n d e q u a l

 

m sform r i e r tr a n sfor scretee F o u ri 1.4 1. 4   T he d i scret

for   onicaa l b a si s   for   j} j = 1 b e  the c a n onic P r o o f   L e t   {8 or   em   i.e., f o r m b as is f or iscree t e F o u r ier t r a n s fo d iscr

;tl (k;;;tl e 2 , . i ( n - 1 ) (k;;

en, en,

21  21 

k} k=l b e the a n d l e t {  ek} 

e 2 , .i(m - 1 )  (k;;;Il

I d e n tify i n g en   w i t h   a  s u bs p a c e  o  off em em,, t h e  o rt h o g o n a l   p ro j e c t i o n o f  e k o n t o 0  1.9.   ercise se 1.9. follow s f ro m E x erci w t h e   r e su l t follow now en is Pek =   /k; no noticc e   t h a t all t he v e c to r s fk in P r o p o s i t i o n 1 . 4.2 h a v e t o  noti mportt ant to I t is i mpor erefo o r e n o r m a li z e   t he m w h il e k e e p in g   t he s a m e n o r m . I f n e e d e d , w e c a n t h eref ngly.. ccord d i ngly a t i g h t f r a m e ; w e o n ly h av e t o a d j u s t t h e f r a m e b o u n d   accor n, C o r ol la r y  1   4 3 F o r a n y m   vector s . g   o f m n o r m a lize d   vector c o n sistin s istin g 

th ere

e xists

a tigh t fra m e

in  

en

ists   o f t h e scrett e   F o ur i e r t r a nsfo r m   b asi s i n  C4 c o n s ists E x a m ple 1 4   4   Th e  d i scre ctorss v e ctor 1

1

2  (

1) 

1 

,2

i (  1 )

~  

,  1

-1

1

~  1

,  2

( 1 )

-i ( 1  )

~  1

ectorr s   1.4.2, 2,   t h e   vecto V i a   P r o p o s itio n 1.4.

0 

c o n s t i t u t e   a t i g h t fram e   in ((/l.

asis   is  is  t h a t lete   fr a m e c o m p a r e d   to a   basis O n e   a d v a n t a g e   o f  an o v e rcom p lete t h e fram e   p ro p e r t y   m i g h t b e k e p t i f  a  an n e l e m e n t   is  is  re m o v e d . H o w ev er, e v e n for ainin n g   s et is n o   lo n g e r   a fr a m e ,   for fo r   fr a m e s   it c a n h a p p e n   t h a t the r e m aini le  g o nal t o t h e   rest o f  t he  f r a m e   ele e x a m p l e i f th e r e m o ved e l e m e n t is o r t h o go m e n ts. U n f o r t u nate l y   t h is c a n b e  th e  c ase n o   m a t t er h ow   l ar g e th e n u m b e r is  I f  w e h a v e e., no n o m a t t e r ho w   r e d u n d a n t   t h e  f ra m e  is  is,  i. i.e., o f f r a m e e l e m e n t s is,  fr a m e b o u n d   a n d t h e n o r m   o f the f r a m e e l e m e n t s   lo w er fra inforr m atio n   o n the low info re m o ve : how w   m a n y   el e m e n t s w e c a n   ( at l e a s t) rem forr ho riterii o n fo w e c a n   p r ov i d e a   criter er ositio o n 1   4 5 L et { h } k = l be a n o r m a lize d fra m e for e n   with   lo w er Prop ositi III   < A ,   {1, . . .  ,  m } w ith III e x s e t I C   {1, f o r a n y   i n d ex nd A >  1. T h en, fo f r a m e   b o u nd I. IJI. d   A - IJ en   w ith lo w e r b o u n d  fr a m e fo r en i s a fra ly { f k } k ~ t h e   f a m i ly

 

22 P roof .

1. F r a m e s in   Finite-di Finite-di m e n s i o n a l I nn e r Product S paces

G i ven

f

E C",

kEf

kEf 

Thus

krf_ff krf_

IU, hW

~

A

-111)1 -111)111 11 2 ·

D 

T h e o r e m   1.2.1 show show s that i f { f k } ~ then   is a t i g h t n o r m alized fram e , then app l i es if II IIII < I[t. C o n s i dering an   a rb itrary f r a m e P r o p o s i t i o n   1.4.5 appl one   ca n h o p e t o remove Udk"= 1 for C " , th e  m a x i m a l n u m b e r o f e llee ments one  ove w h i le  keeping t he f r a m e p r o p e r t y i s m - n . I f w e w a n t t o be a b l e to  r e m ov is a nor m - n   a rb i t ra ry e l e ments it is n o t e n o u g h to a s s u m e   that { f k } ~ n this . 3 .5 ; iin frame,, a s d e m o n s t r ate d by th e frame in E x a m p l e 1 .3 malized  tight frame (1.18 ) d o no t   co n stitute vecto r s in (1.18) th e t h r e e   first vecto e x a m p l e m - n = 2, but the e m oval of v e c tors, the a fram e for C 3 . C o ncerning t he stability   a g a i n s t r em itrary  e l e m e n t s well:: m - n ar b itrary 1.4.2 4.2 b e h a v e well o s i tion 1. f r a m e s o b t a i n e d  i n P r o p os e m o ved: c a n b e r em 6 C o n s i d e r the fra m e { f k } ~ o n 1.4. 1.4.6 P r opositi opositio

for C " d e fine d i n P r o p o 

l e a s t n elem elemen en ts o f this   f r a m e fo r m s sitio n 1 . 4.2. A n y s u b s e t c o n t a i n i n g a t le sition a f r a m e for C   .

Proof. C o nsider an a rbitrary s u b s et { k 1 , k 2 ,   . . . , kn}

1. Then there 0 and orthonormal bases {uk}k=l for R E and

{vk}k= 1 for RE· such that (1.24) Proof. Observe that E* E is a self-adjoint n x n matrix; by Theorem A.2.1 this implies that there exists an orthonormal basis {vk}k=l for en consisting of eigenvectors for E* E. Let {Ak}k=l denote the corresponding eigenvalues. Note that for each k, >.k

The rank of

= >..kllvkll 2 = (E*Evk,vk) = IIEvkW

0.

E is given by r = dimRE = dimRE·;

since n ~

= NE·, we have R E · = RE· E = span{E* Evk}k=l = span{..\kvk}k=l·

(1.25)

Thus, the rank is equal t o the numb er of non-ze non-zero ro ei eigenv genvalu alues, es, cou counted nted with multiplicity. We can assume that the eigenvectors {vk} k=l are ordered such that {vk}k=l corresponds t o the non-zero eigenvalues. Then (1.25) shows that {vk}k=l is a n orthonormal basis for RE·. Note that f o r k > r, we have

IIEvkll 2 = (E*Evk,vk) = 0, i.e.,

Evk = 0, k > r.

(1.26)

Defining Uk : =

1

~ E v k

v >.k

k = 1, . . . ,r,

we therefore obtain that {uk}k=l spans R E i and it is a n orthonormal basis for R E because for all k, l = 1, . . . , r we have

 

26

1.

F r a m e s   in Fin Fin i t e - d i m e n s i o n a l In ner P roduct S p a c es es

T h us, t h e c o n d itio n s in L emma 1.5. 1.5.3 3   are fulfille d   with ak =

v->::,,  k = v->::

1, ... ,r.

0 

L e mma 1.5.3   le a d s to th the e   singular va value lue decompo decomp o sition of E :  

T h eorem eorem   1 5 4 E v e r y m d e c o mp os o s ition

x n

matrix E

with

rank

r

>

1

has

a

E -u -u((D

o ) v ·  , 0

0

( 1.27)

where U i s a u n i t a r y m x m   ma t r i x , V  is a unita r y n x n m a t r i x , a n d

(

~

)

is an m   x n block m a t r i x i n w hich D is  is   an r x   r d iagonal

matrix w ith positive  positive   e n t r i e s a 1 ,

...

,

a r in th thee diagonal.

Proof W e u s e the p r oof of Lemm Lemma a 1.5.3. L et  { vdr= vdr=ll be  the o rt h o n ormal b a s i s f or o r en c o n s i d e r e d t h e r e , or dered s u c h  t h a t { v k}k= k}k=ll is a n o rthonormal rtho normal   b a s is f o r R E · . Let V b e t h e n x n   matrix h a v i n g the v e c t o r s {v {vdr= dr=ll a s c o l u m n s . Extend Extend the  the orthon orthono o rmal b a s is i s   {uk}J.:= 1 f o r   RE to an o an  orthonorm rthonorma al b a s is { } ~ for em a n d  le t U be  t h e m X m m a trix h a v i n g   these v e c t o r s a s c o l u m n s . F i n a l l y , le t D b e  t h e   r x r d i a g o n a l m a t rix h aving aving  a1, . . . , G'r in t h e d i a g o n a l. Via ( 1.24) and ( 1 .26),

EV 

=   ( a1 u1

arur 

u (  ~   ~

0

0 )

. 0

M u l t i p l y i n g with V* f r o m t h e r ig h t g ive iv e s t h e  r e s u lt lt. 

The n um b ers a 1 , . . . , G ' r ar e   c a l l e d singular   values fo r E ; t h e pr o o f of Lemma  1 . 5 . 3 s h o w s t h a t t h e y are t he square ro ots o f the p o s itiv e e ig e n v a lu e s Lemma

fo r E * E. C o rollary 1 5 5 With the th e n o t a t i o n i n T h e o r e m   1.5.4, the p s e u d o - i n v er erse o f E is given by  by  

E t-

w here (

~

)

ma tr i x having having   1 / a 1 ,

v ( n-1 0

o ) u· ,

(1.28)

is is   an n x   m b lock matri matrix x in which ...

,

n- 1

is the r   x r

1 /ar in the diagonal. d iagonal.

 

u e   d e co m p o s i t i on valu l a r val -inverr s e s an d  the s i n g u la 1.5 P seud o -inve

Proof 

27

sfiess t h e .28)   s a ti sfie ined   by (1 .28) W e  c h e ck t hat the m a t r i x   E t d e f ined

.27),, s t , via (1 .27) l(ii).. F i r st r e q u irem e nts i n   P r o p osit i o n 1 . 5 . l(ii)

=

U  ( U (

~

~ I 

V*V (

0) U* ,

n-1  n-1 0 

0  0

jointt is elf-aa d join oint.. Th e  p r o o f  t h a t E t E is s elflf-ad d j oint is  se self-a w h ic h sh o w s t h a t E E t is  or   E E t ,   presss i o n for  s i m i l a r . F u r t h e r m o r e , u s i n g t h e d e ri v e d e x pres

EEt E

= 

U (

= 

E . 

) U *U   (  

~

) V*

0

that   E t E E t = Et. erify y that S im il a rl y , o n e   c a n  verif

en   w i t h   pr e  or   en } ~ is a f r a m e f or  us   re t urn t o  the s e tting   w h e r e   { L e t us  fficien en ts Th e  c a l c u l a t i o n o f t h e f r a m e c o e ffici f r a m e o p e r a t o r   T : em -+ a m o u n t s   to fi n d in g r t :  

en..  en

'= 1   be a f r a m e for {fk}k }k'= T heor e m 1 5 6   L e t   {fk T   a n d f r a m e   o p er a t o r S . Th en

en,

w i t h   p r e- f r a m e o p e r a t o r

(1.29) (1.2 9)  

en.

E x p r e s s e d in   te r m s o f th e p r e - f r a m e o pe r at or T , t h e   Proof L e t f E followss esultt n ow   follow = f . Th e  r esul ~ c kfk m e a n s t h a t   {   k } ~ e q u a t i o n f  = 0 1.5.2.. .5 a n d   T he o r e m   1.5.2 .1.5 b y   co m b i n in g   T h e o r e m 1 .1 6 is t hat w h en {fk}k'= 1   is a   fr a m e for 1.5. 5.6 i o n   o f T h e o r e m   1. O n e i n t e r p r e t a t io e  off t h e is   o b t a i n e d   by p l a c in g  the c o m p le x c o n j u g a t e o or   r t  is t he  m a trix f or in  the c a n o n i c a l d u a l  f r am e   { S - 1 f d k '= 1   a s r ow s   i n a n  m x n   m a t r i x : v e c to rs in 

en,

1

  11-ll . 

-S - 1 S - fz-

rt

=

( - S - 1  f m -

.29 9) m e a n s  t h a t (1.2 I n   o p e r a t o r t e r m s ,   (1

r t = T * ( T T*)- 1 , o-inv v e rse o f   a n   pseud d o-in or   t h e pseu eneraa l l y for  a form u l a t h a t  is k n o w n   to  h old g ener jectii v e o p e r a t o r T .   rary s u r ject arbitrary arbit co effi ob t ain coeffi ay   to obt aturall  w ay  giv ves es   a n atura sitio o n   gi Th e  s i n g u l a r v a l u e d e c o m p o siti verco o m p le te   '= 1 b e   a n  o verc {fk}k }k'= 2:::;;= ::;;= 1 c kfk· L e t   {fk }k'= 1   s u c h t hat f   = 2: {ck}k'= c i e n t s {ck

 

28  28  

ces ite-dii m e n s i o na l I n n e r Pr oduct S pa ce 1. F r a m e s   in Fi n ite-d

en..  en

is   s u r j e c t i ve, i t s   r a n k   e qu a l s n , and ince   T is en   a nd  l e t f E   S ince fo r en f r a m e for ositii o n   o f T is singu u lar v a lue d e c o m p osit th e sing T  = U ( D

0  ) V * .

is  now a n n x m m a trix ; N o t e   th a t s in c e T i s  a n n  x m   m a t r ix, ( D 0   ) is  is  an m   x   m   m atrix . G iv e n   a ny ( m - n ) x   n i x ,   a n d   V is  U   is a n n  x n m a t r ix m at r ix F,   we h a v e

TV ( D ;

1 

)

U *  f 

=

U ( D

0  ) V*V  ( D ;

1 

)

U* f

U IU*f

=   f.

ents   effici cients usee t he  co effi T h i s m e a n s   t h a t we c a n us

oice   .1.5   the ch oice for fo r   th e reco n s t r u c t i o n o f  f . A s n o t e d   a lr e a d y in   T h e o r e m   1.1.5 ents   is effici ci ents sensee t h a t th e   e2 - n o r m   of the co effi F   = 0 is o p t i m a l i n   th e  sens eferaa b le. The ices   m i g h t b e p r efer r   pu r p o s e s o t h e r c ho ices imizee d , b ut for o t h e r  m in imiz m a t rix

~ t ) U *

v

d i n v er erse ofT . lized is  is  f r eq u e n t l y   c a lle d   a g e n e r a lize

1 .6

F i n ite- d i m e n s i o n a l  f u n c t i o n   s pa c e s  

ite-dim im e n s io n a l   vecto r fram am e s in   i n fin ite-d d e a l w i t h   fr Th e  r e s t o f t h e b o o k w ill de lik e   L 2 ( - 1 T , 1r) a n d   ction n s p a c e s like tionss in f u n ctio nstru u c tion aces,, with   conc r e t e c o nstr s p aces fo r the m o m e n t   2,   a n d for iven   in C h a p t e r  2, ition n will b e  g iven L 2 (1 (1R); R); t h e i r e x a c t d e f in itio or   w hic h siderr L 2 ( I), ~   lR s im p l y as t h e   s et o f f u n c ti o n s for  we c o n side

~   l f ( x ) j 2 dx <  oo. porta a nt to  to  noti noticc e   th a t in e v ery re al-l al-lif if e   a pp l icat icatii o n w h e r e t h ese I t is i m port fine   to f i n i t e - d i m e n s i o n a l s p a c e s a p p e a r , o n e   will a t s o m e poin t h a v e to co n fine we   c o n c lu d e t h i s  c h a p ter w i t h a   sh o r t d e s c r i p tion   s u b s p a c es. F o r t h i s   r e as o n we spacee s . ction n spac fu n ctio of fra fr a m e s in   f i ni te -dim e nsio n al fun G i v e n a, b E lR w i t h   a <   b, l e t C [ a, b] d e note   the s e t o f c o n t i n u o u s nctio o n s f : [a, b] --t C. W e e q uip C[a, b] w i t h   the s u p r e m u m s - n o r m ,   f u ncti

oo =   s u p llfll llflloo

x E [a,b]

f(x)ii . i f(x)

ation n T h e o r e m sa y s t h a t  e v e r y f   E C [ a, b] c a n   roxim m atio Th e W e i e r s t r a s s ' Ap p roxi b e   a p p r o x i m a t e d a r bitr a r i l y w ell b y a p o ly n o m ia l:

 

1.6 Finite-dimensional function spaces T heorem 1.6.1 Let f E

P(x) = L ~

O

C(a, b]. Given

f

ckxk such that

II/- Plloo

>

29

0, there exists a polynomial

~ f

I t is essential for the conclusion that [a, b] is a finite and closed interval (Exercise 1.12). Also, we note that the order of the approximating poly nomial depends as well on the chosen f as the given function f and the

interval [a, b]. The polynomials {l,x,x 2 , . . . }

=

x k } ~

are linearly independent and

do spanspace a finite-dimensional subspace of C[a, b]. But for a given n EN, vector thenot

v : = span{1, x, . . . 'xn} is a finite-dimensional subspace of C[a, b] with the polynomials { xk}k=O as basis. norm m I f we equip V with the 11·11 0 0 -norm, we do not have the benefit of a nor arising from a n inner product. But all norms on a finite-dimensional vector space are equivalent (see page 13), and V can also be equipped with the norm

11 11 =

(

) 1/2

b

11f(xWdx

arising from the inner product (1.30)

( , g ) = 1 b f(x)g(x)dx.

Via the Gram-Schmidt orthogonalization procedure one can construct a n orthonorm orth onormal al basis fo forr (V, 11·11) (Exercise 1.14). In classical Fourier analysis one expands functions in £ 2 (0, 1) in terms of the complex exponential functions { e2 1rikx h E Z · In Chapter 7 we will obtain more general results with {e 2 1rikx hez replaced by { ei>.kx hez for some real sequence {Ak} kEZ satisfying certain density conditions. Let us for the mome moment nt consider a finite colle collection ction of exponential expone ntial functions {ei>.kx} k=l, where {Ak}k=l is a sequence of real numbers. Unless {Ak}k=l contains repetitions, such a family of exponentials is always linearly independent: L emma 1.6.2 Let {Ak}k=l be a sequence o f real numbers, and assume that .Xk =f. Aj for k =f. j . Let I lR be an arbitrary non-empty interval, and consider the complex exponentials {ei>.kx} k=l as functions on I . Then the

functions {ei>.kx}k=l are linearly independent.

I t is enough to prove that the functions { ei>.kx h E Z are linearly independent as functions on any bounded interval]a, b[, where a, b E IR, Proof.

 

30

a

< b.

1. Frames in Finite-dimensional Inner Product Spaces

Assume that for some coefficients {ck}

=

L ckei>.kx = 0, Vx E]a, b[. n

k=1

When x runs through the interval ] a2b, b;a [, the variable x + through ]a, b[; i t follows that '

n

k=1

' c k e i > . k ( x + ~ )

=0, V E ]a -b b - a [ X

2

'

2

.

runs

~ d k e

·>.

kx

= 0,

k=1

a -b b-a Vx E ] 2 - , - 2- [ .

By differentiating differentiating this equation j times, j . .. dk(i>-.k)Je' kx k=1

Putting x

= 0,

= 0, 1, · · · , we

a-b b-a

Vx E ] 2- - , - - [ , j 2

obtain that

= 0, 1, · · · .

= 0 and writing the corresponding equations for j = 0, ... , n - 1

as a matrix equation gives

)..n-1 2

)..n-1 n

The system matrix is a Vandermonde matrix with determinant n

ll=

II

> - . k - A j ) ~ O ;

k,j=1,k#j

therefore d1 = Thus {ei>.kx} =

d2

= · · · = d n = 0, which implies that c1 = · · · = are linearly independent. independ ent.

Cn

= 0. D

In words, Lemma 1.6.2 means that complex exponentials do not give A j for natural examples of frames in finite-dimensional spaces: if Ak ~

k ~ j , then the complex exponentials { e i > . k x } ~ = form a basis for their span in L 2 (I) for any interval I of finite length, and not an overcomplete system. W e can not n ot obtain overcom overcomplet pletenes enesss by adding extra exponentials (except by repeating repeat ing som somee of th thee >-.-values) - this will just enlarge the space. In Exercise 1.15 the similar problem for sines and cosines is considered.

 

1.6 Finite-dimensional function spaces

As an important special case we now consider the case where >..k A function f which is a finite linear combination of the type N2

f(x)

=L

c k e 21rikx

for some

Ck

E C, Nl> N2 E Z , N2?: N1

31

= 2rrk. (1.31)

k=N1

is called a trigonometric poly polynomi nomial. al. Trigonomet Trigonometric ric polynomials correspond correspon d to partial sums in Fourier series, a topic to which we return in Section 3. 7. A trigonometric polynomial f can also be written as a linear combination of functions sin(2rrkx), cos(2rrkx), in general with complex coefficients. I t that if f is real-valued and the coefficients Ck in will beare useful (1.31) real,later thentof note is a linear combination offunc of functio tions ns cos(2rrkx cos(2rrkx)) alone: alone:

L emma 1.6.3 Assume that the trigonometric polynomial f i n

real-valued and that the coefficients f(x)

Ck

E It Then

N2

=

L

Ck

cos(2rrkx).

(1.31) is

(1.32)

k=N1

W e le leav avee the short proof to the reader. Note that we need the assumption that Ck E JR: for example, the function f(x)

= 21i e n. x -

1 . i e n x 2

= sin(x),

is real-valued, but does not have the form (1.31). Again for later use we mention that a positive-valued positive-valued trigonometric trigonome tric poly poly nomial has a square root (in the sense of (1.35) below), which is again a trigonometric trigonome tric polynomia polynomial: l: L emma 1.6.4 Let f be a positive-valued trigonometric polynomial o f the form N

f(x)

= L Ck cos(2rrkx),

Ck

E JR.

(1.33)

E JR,

(1.34)

k=O

Then there exists a trigonometric polynomial N

g(x)

= L d k e 21rikx

with

dk

k=O

such that

lg(xW

= f(x),

(1.35)

Vx E JR.

A constructive proof can be found in (106]. Note that by definition, the function g in (1.34) is complex-valued, unless f is constant; that the com plex terms in g do not cancel follows from Exercise 1.15. Actually, despite

 

32

1. Frames in Finite-dimensional Inner Product Spaces

the fact that

f

is assumed t o be positive, there might not exist a positive trigonometric polynomial g satisfying (1.35). The complex exponentials do not belong t o L 2 (IR), but by multiplying them with a function g E L 2 (JR) we obtain a class of functions in L 2 (1R). In Chapters 8-10 we will work with systems offunctions in L 2 (1R) of the form {EmbTna9}m,nEZ : = {e 2 11'imbxg(x-

na)}m,nEZi

here g is a given function in L 2 (JR), and the parameters a, bare positive real numbers. number s. Such a family of functions is called a Gabor system. I t was proved by Linnell (214] in 1997 that if g ::1 0, then an arbitrary finite subfamily 2

2

{e 11'imbxg(x-na)}(m,n)EF• :F Z , is linearly independent. At the moment it is not known what happenscif we replace the numbers {(na, mb)}m,nEZ

by arbitrary distinct points in JR2 • To be more precise, Heil, Ramanathan and Topiwala [170] formulated the following conjecture in 1995: Conjecture: Given any finite collection of distinct points {(JLk, Ak)}keF in IR2 and a function g ::j; 0, the Gabor system { e 2 ,.i>.kx g(x - JLk) he.r is

linearly independent.

Considerable effort has been invested in the conjecture, but it is still open. W e return t o this conjecture in its right context on page 229. Also, in Section 10.3 we will construct frames in en having the Gabor-structure. Wavelets is another important class of functions in L 2 (1R); we consider of them detail in Chapters 11-14. A wavelet system consists functions of the in form

'l/Jj,k(x) = 2 j / 2 '1j;(2jx- k), j , k E Z ,

where 'ljJ E L 2 (IR) is a given function. Linearly dependent wavelet systems exist. For example, by letting 'ljJ : = X(o,l(• one has 'l/Jo,o

=

1 J2('l/J1,o

+ l/J1,I).

I f a finite wavelet system, { l / J j , k } I J J , I k i ~ N for some N E N, happens to be linearly independent, one could ask for the minimal number m N of independent sets i t can be split into. One could expect m N t o grow with N; however, in case 'ljJ has compact support and l'l/JI > 0 on some interval of positive length, it is proved in (83] that one can find a number m E N such that { l / J j , k } l j l , l k i ~ N can be split into m linearly independent sets, regardless r egardless of how large N is. I t is not known whether the result holds if 'ljJ is not assumed to have compact support.

1. 7 1.1

Exercises Show that every frame {h}f=1 for a finite-dimensional vector space V contains a subset which is a basis for V.

 

1. 7 E xercises xercises  

33

1 2 C an a f r am e  e   in a finite-d finite- d imensiona imensionall s pace conta cont a in infinitely infinitel y   m a n y

elements? 1.3 L e t   {f kh E I b e   a frame for for  a finite-di finite-di m ensional v ector space  space  V and as s um e  that llh ll h ll is bounded   below. Prov e that I is f inite. (w.l.o (w.l. o .g. you m ay as s um e  e   that V = lRn lRn a n d that 11/k 11/kllll = 1, V k; expl ai n   why if  if  you w ant to  use this fac fa c t ) 1 . 4 C o n s truct a fram fra m e { f k } for Cl for w hich there   exists f E C l s u c h t h a t the c oefficients { dk}r=l in T h e o r e m 1.1 1 .1 .7 are not u nique. 1 . 5 L et { e1, e2} b e the c a n o nical ortho n or m a l ba bass is   for

Cl

an d  consider

the f r a m e {fk}L 1 = { e1, e1, e2,  e2, e1 + e2}.

(i) F ind the c o e fficients w i t h m i ni m al  al   €2 - n o r m a m ong all seq sequ u ences { c k } ~ = l

for which e1 = L ~   l

c kf k ·

(ii) (i i)   F i n d the c oefficients { ci 1 ) H = 1 a n d  { c i2 ) H = 1 w hich minim mini m ize the €1 -norm in t he r epr es e n t at i on of e 1 a n d e 2 , re s pectively. (iii) (i ii) Clearly Clearly,, e1 + e 2 = ~ + c i2 ))fki bu t is { c i1 ) + c i2 ) H = 1 m inimizing   t he €1 - nor m   among all  all  sequences r e pr es ent i ng  ng   e1 + e2? 1 6 Assume t h a t { } ~ is a frame fo forr en. P r ove ov e that that  the 2 m vect or orss consisting consistin g   o f the r e a l   par t s , r esp esp ectively t h e  i m agi nar y   pa rts, o f t he   frame v ectors con s titute a fram fra m e for JRn. 1

Show t h a t a frame ffo o r lRn is als also o   a frame fo forr  en en..

1 . 8   P r ove C or ol la l a ry 1.3.6. 1 9 L et {  

b e a   frame for V   w i t h bou n d s A, B an d let P d e n ot e the orth o gonal pr o j e ction of V ont o a s ub s pace W. P r o v e t h a t { P   k } r= l is a   frame for W   with fram fra m e b o u n d s A , B. } ~

1 1 0   L e t   {fk}r=l b e   a normali normalizz ed t i ght f ra r a me. Prove t hat the f r a m e b o u n d A   is a t leas t   1, a n d that that   A = 1 if an d only if { fk}r=l is a n   o r t h o n o r m a l basis. 1 11 L e t {fk}r= l be a frame  frame   for ann-di m e n s i o n a l v ector space  space   V , and let B d e n o t e t he opt im i m a l u p p e r b o und. P r ove ov e   that that   m 

B ~

L

k=l

llfkll 2

~   nB.

 

34

in   F i ni te t e - d i m en e n sion a l   I n n er er   Produ Product ct S p aces 1.  1.  F r a m es in 

cl o se d   and b o u n d e d   i n te r val ils if i f t h e clo fails 1 1 2 P r o v e  t h a t T h e o r e m   1.6.1 fa nterv v a l o r   an u n b o u n d e d i n t e r v a l. b]   is  is  r e p l a c e d   by a n  o p e n   inter [a, b]

1, x , . . . , x n } a r e   1   1 3   P r o v e  t h a t for a n y n E N, t h e p o ly n o m i a l s {  1, li n e a rly i n d e p e n d e n t in C (O, 1) . as   f un c t i o n s o n   th e  i n t e r v a l 1 1 4   C o n s i der the p o l y n o m i als { 1, x , x 2 } as [0, 1], 1], a n d l e t V = s p a n { 1 , x , x 2 } .  E q u i p V w i t h th e i n n e r p r o d u c t fo r V . fi n d a n o r t h o n o r m a l b a s i s for .30)   and fin (1 .30) 1 1 5   L e t { A k} k}k k =l b e a sequ sequee n c e o f   r e al n u m b e rs.

x}  k= 1   a r e   l i ne a rly i n d e p e n d e n t i n C  ( - 1 , 1) if ( ) P r o v e t h a t {c o s Ak x}

forr k =f j . l-\k I =f l-\j l-\j I fo a n d o n l y if l-\k if   e p e n d e n t in   C( - 1 , 1) if  early y i n d ep n-\kx x } k = l   ar e l i n earl (ii) P r o v e th a t { s i n-\k l-\jll f o r k  = f j .   l-\kll  =f l-\j n-zerr o   an d  l-\k if  all Ak  a r e n o n-ze and o n ly if 

k} k =l, {Jlk Jlk}} k = l a r e  the (iii) iii ) U n d e r w h ich c o n d i t i o n s o n   s e qu e n c e s { Ak}  ction ns f u n ctio os-\k k x } k = l U  { s i n   { c os-\

k X } ~

early y inde p e n d e n t in   C ( - 1 , 1 )? l i n earl (iv) (i v) R e plac placee th e  inte interr val] val]-- 1 , 1[ b y a n a r b i t r a r y  n on - e m p t y i n t e r v a l   (iii).. (i),(i (i i ) a n d (iii) ralizee (i), a n d   g en e raliz tive   t ri g o n o m e t r i c p o l y n o m i a l f ( x ) = 1 + c o s ( x). 1 1 6   C o n s i der the p o s i tive ials   F i nd  b y   d ir e c t c a l c u l a t i o n a ll trig o n o m e t r i c p o l y n o m ials

g ( x )   = d o + d 1 eix, d o , d 1 E for w h ich lg (x)l 2 = f ( x ) .

 

2 I n f in ite - d im e n s io n a l V e c to r  S p a c e s a n d   Se q u e n c e s  

A f ter t h e i n t r o d u c t i o n   t o   f r a m e s in   f ini in i t e-di e-dim m ensi ensio o n a l v e c t o r  r   sp a c e s   i n C h apte apterr   1, t h e r e s t o f th e b o o k   will d eal w ith e x p a n s i o n s   in  in   inf in f i nite d i m e n s i o n a l   v ecto r spac spacee s . H e r e g r e a t c a r e   is n e e d e d :   we n e e d t o   r e p l a c e finite fini te s e q u e n c e s {fk}k {fk}k== ==ll b y infi infin n i te s e q uenc uencee s {fk}); {f k});"== "== 1 , an d s u d d e n l y t he   q u e s t i o n o f c o n v e rgen rgencc e p r o p e r t i e s   b ec o m e s   a  c e n t ral is sue. sue.   T he   v ecto r s p a c e   itsel itsel f   m ig h t a l s o   c a us e   p r ob l e m s , e.g. e.g.,, i n th e   s e n s e   that C a u c h y s e q u e n ces m i g h t n o t be   c o n v e r g e n t . W e  e x p e c t th e   r e a d e r to have have   a b a s i c k n o w le d g e a b o u t t h e s e p r o blem s   and th e w a y t o circ u m v e n t t h e m ,   b u t f or c o m p l e t e ness ness   w e r e p e a t th e c e n tral t h e m e s in S e c tion tionss 2. 1-2 1- 2 . 2 . I n S e c t i o n s 2 . 3 - 2 . 5 w e d i s c uss t h e   H i l b e r t   sp a c e  e   L 2 (IR (IR)) c o n s i stin g o f t he   s q u a r e i n t e  g r a b l e f u n c t i o n s   o n IR an d   t h r e e c la sses sses   of o p e r a t o rs r s here o n , a s w ell  ell   a s  the F o u r i e r t r a n s f o r m . T h e m a t e r i a l in t h o s e s e c tion tionss is no t n e e d e d fo r th e   st ud y   o f  f  frra m e s a nd b ases ases   on a bs tract H i l b e r t s p a c e s i n Ch ap t er s 3- 6,   b u t

i t f o r m s t h e  basi basiss for a ll   t h e c o n s t r u ctio n s in  in   C h a p t e r s 7 - 1 4 .

2.1

S e quen ces

A   c e n t r a l t h e m e i n this   b o o k  k   is t o find find   co n d i t i o n s   on a   se q u e n c e {fk} {f k} i n a v e c t o r   sp a c e  e   X   s u c h t h a t   e v e r y f   E   X h a s a r e p r e s e n t a t i o n   as a s u p e r p o s i tion   of the v e c t o r s h .  I n   m o s t   space pacess a p p e a r i n g i n f u n ctio n a l analy anal y s is, th t h i s c a n not b e   d on e   w it h   a fin fin i te seq se q u e n c e { f k } . W e a r e t h e r e f o r e fo rced rced   t o  w o r k w i t h   i n fi n ite se s e q u e n c e s, say, sa y,   {fk}j;" {fk}j;"== == 1 , a n d the  the  r e p r e s e n t a t i o n   o f f   in t e r m s o f   { } ~ w ill b e via an   i n f i n ite se ries. ries.   Fo r t h i s r e a s o n t h e

 

36

2. I n f i n i t e - d i m e n s i o n a l V e c t o r   S p a c e s and S e q u e n c e s

starting p o i n t m u s t b e a d i scuss scussion ion of conv convergenc ergencee of infinite infinite   series. series. We  We   collect coll ect t h e basi basicc d e finition finitionss here her e t o g e t h e r w i t h s om o m e c onvent onventions. ions.   T hr oughout oughout  th e sectio section we   let X   b e a   n o r m ed n we ed v e ctor sp ace, w i t h n o r m  

d e n o t e d b y   11·11- W e s a y that a sequence seq uence {xdz {xdze' e'= = 1 in X (i) co converg nvergee s t o x E X   if

l l x - xkll xkll-+ -+ 0 fork - +  oo; (ii) is  is  a C a u c h y se s e q u e n c e if f or each each  

l l x k - x1ll ~

f

>  0 t h e r e   exist exist s

N E N s u c h that

w h e never n ever k, l 2: N .

A   c o n v ergen ergentt sequence seque nce is auto m a t i c a l l y a  a   C a uc h y s equen e quence, ce, but th e   o p  posite   is n ot true in g eneral. Th e r e  are, howe however, ver, no r m e d v e c t o r spaces spa ces in which whic h a sequen sequence ce is  is  convergent convergent   if and

o n l y if it is is   a C a u c h y   s e quence; a s p ace X w i t h  t hi s   property is called a Ba n ach s pace. All spaces spa ces conside red in th i s b o o k   a r e  Banach s p a ces. Imitatii n g t h e f i n ite-dim ensio nal se tting Imitat tting,, we w a n t to study s e quence quencess { k } ~ in X w i t h t h e property that e a c h f   E X h a s a r e p r e s e n ta tation f = ~ c k ] k for som e c o efficien efficients ts Ck Ck   E C. I n o r d e r to d o so, we have   to  explain   e x ac t l y what w e  m e a n by conver convergenc gencee of an infin ite series, seri es, a nd t here here  are, in fact, fact,  a t leas t th r e e differ different ent o ptions. ptions.   F i r st st , the notat notatii o n  { h }ze = 1 i n d icates that w e h a v e chosen cho sen som e o r d e r i n g  g   o f t h e  vect vectors ors h ,

h ' h, h, . .. , h , h + l , . . . . W e say that that  a n  i n f i n i t e se s e ri e s

~ =

if

c k h is c o n vergent v ergent   w i t h  h  sum f E X  

W he n  t h i s c o n d i ti t i o n is s atisf atisfied ied we write   00

f  = L c k h ·  

(2.1)

k=1 k=1

T h u s , th e defin definition ition   of a conve convergent rgent infinit in finitee series serie s cor corrr e s p o n d s e x a c t l y to our d e finition finition   o f a  convergent convergent   sequenc sequenc e w ith X n = ckfk· 1. A h ove w e insis insisted ted on a fixed fix ed o rdering  o f th e  sequ sequence ence { h } I t  is very   importa import a nt to notic noticee t h a t conv ergenc ergencee p r o p e r t i es e s of ck]k not o n l y d e p e n d s o n the sequenc sequencee { f dze'= 1 and t he co effici efficients ents { ck}ze'= 1 , but also also   o n the o rdering rdering.. E v en if } ~ is a   s e quence in in t h e s i m p lest possibl possiblee B anach space space,, n a m e l y IR, IR, it ca n ha ppen that that   f k is c onverg ent, but that  f " ( k )   is d i v erge ergent nt for a  a   c e rt a i n pe rm u tation a o f   the natural natura l n u m b e r s .   Th i s o b s e rvatio r vatio n leads  leads   t o a second secon d d e finition   o f converg ence ence.. I f f " ( k )   is c o n v ergen e rgentt for all a ll pe rmutation rmutationss a, we s a y t hat f k is u n c o n d i t i o n a l l y c o n v e rgent. rg ent. I n t h a t case case,, the l i mit is the s a m e r egardle s s o f the o r d e r o f s u m m a t i o n .

 

2.1   Se q u e n c e s 2.1

37  

ie sz b a s e s i t will b e c o m e   c l ea r   defin n e d f r a m e s a n d R iesz soon   as w e  h a v e defi A s soon

ergen n t  e x pa n s i o n s . F o r onalll y  c o nv erge  lead  t o u n c o n d i t i onal that t h e y a u t o m a t ical l y  lead rges   convee rges eriess conv iven   serie t h i s r e a son w e n e v e r n e e d to  p ro v e b y h a nd t h a t  a   given tailee d   a n al y sis o f   fo r a m ore d e tail 210]] a n d   (260] for unco n ditio n ally . W e re f er t o  (210 ol l ow i ng   l e m m a . ence   and t h e p r o o f o f  the f oll feren n t   t y pe s   o f c o n v e rg ence the d if fere be a   se quen que n c e in a Ba n ac h s p a c e X , an d l e t L e m m a   2 1  1 Le t { f k } ~ f E   X . Th en th e   f ollow i n g a re e q u i v alen t: (ii)

L::%"= 1 f k

c o n v e r g e s u n c o n d i t i o n a l l y to   f  E X .  

{ i i) F o r   ev e r y   €   >  0 t h e r e e x i s t s a f i n i t e set F s u c h t ha t  

n i t e s e t s   I C N c o n t ai n i n g F . fin f o r   all fi

seri ri es infin n i te se F i n a l l y , a n infi

L::::% %: 1 fk  is

ol u t ely c o n v e r g e n t if s a i d t o  b e a bs olu

00 

llfkllll <  0 0 . :L  llfk :L

k =1

ence   o f A b s o l u t e c o n v e r g ence

:%:: 1   fk  fk  L: L::%

lies   t h a t   t h e im p lies

eriess s erie

co n v e rg e s

u n c o ndit nditii o nall nally y ( E x e r c ise 2 . 3), b ut  t h e o p p o s ite d o es n o t h o ld   i n in fini finite te  two o t y p e s   o f c o n v e r finite te - d im e n s io n a l s p a c e s t h e   tw sionaa l   sp ac e s. I n fini d i m e n sion nticaa l . g e n c e   ar e i d e ntic  f or e a c h   X   ( c o u n t a b l e o r n o t ) is s a i d   to b e   d e ns e in   X i f  fo A   su b s e t   Z f E X   a nd e a c h   € >  0 t h e r e e x i s ts g   E Z   s u c h t h a t  

I I - 911

~

€. 

this   m e a n s t h a t e l e m e n t s i n X c a n   b e  a p p r o x i m a t e d a r b i t r a r i l y ords,, this I n w ords by   ele m e n ts i n Z . w ell by  d e n ote the v e c t o r in X   we l e t s p n { f k } ~ iven   se q u e n c e { f k } ~ F o r a   given v e c t ors f k . T h e d e f i n i tion    c on s i s t i n g o f all f i ni t e  l i n e a r c o m b i n a tion s o f ve s p a c e  co rgencc e   sh ow s t h a t if e a c h f   E   X   h a s a r e p r e s e n t a t i o n o f t h e  t ype o f c o n v e rgen in   no r m   by a n w e ll in  r a r i l y we E X  c a n b e   a p p r o x i m a t e d a r b i t ra 2.1), then  i.e.,   i.e., n {f f k } ~ p t   i n  es ach e( 2.1) l e m ,e nthen (2 .2) (2.2) is s a i d   t o  b e c o m p l e t e o r h a v i n g   t h e p r o p e r t y   (2.2) se q u e n c e { f k } ~ A   seq to t al. W e n o t e   that t h e r e   e xist n o r m e d   s p ac e s w h e re n o   s e qu e n c e   { f k } ~ is c o m p l e t e . A n o r m e d   v e ct o r s p a c e in   wh ic h a   c ou n t a b l e  a nd  d e n s e f a m i ly   is  said t o b e se p a ra b le . e x i s ts is  W h e n w e s p e a k a b out a f i n i t e s e q u e n c e ,   we m e a n a   se q u e n c e w h e r e   a t entrii e s a r e n o n - z e ro. itely   m a n y entr m o s t f in itely

 

38

2. Infinite-dimensional Vector Spaces and Sequences

2.2 2. 2

Bana Ba nach ch space spacess and Hilbert spaces spaces

All normed vector spaces considered in this book are Banach spaces, and very often convergence of a sequence will be verified by checking that it is a Cauchy sequence. An important class of Banach spaces is the £P-spaces, 1 oo. p L 0 0 ( R) is the space of essentiall essentially y bounded bou nded measurable functions f : IR-+ C, equipped with the supremums-norm. For 1 p < oo, LP(IR) is the space of functions f for which 1/IP is integrable with respect t o the Lebesgue measure: LP(IR) : = { f : IR-+ C I f is measurable and ; _ : lf(x)IPdx

The norm on LP (IR) is

< oo}

.

II/II=

roo ) 1/p ( 1-oo lf(x)IPdx

To be more precise, £P(JR) consists of equivalence classes of functions which are equal almost everywhere, and for which a representative (and hence all) for the equivalence class satisfies the integrability condition. However, we adopt the standard terminology and speak about functions in LP(IR). A vector space X with an inner product (·, ·} can be equipped with the norm

llxll

: = J( x,x}, x EX ,

and Cauchy-Schwarz' inequality states that

l(x,y}l

llxiiiiYII,

Vx,y EX .

W e will always choose the inner product linear in the first entry. A vector space with inner product, which is a Banach space with respect t o the induced norm, is called a Hilbert space. W e reserve the letter 11. for these spaces. The standard examples are the spaces L 2 (1R) and t 2 (N) discussed in the next section.

L 2 (IR) is defined as the space of complex-valued functions, defined on lR, which are square integrable with respect to Lebesgue measure: L 2 (1R) : = { f : 1R-+ C I f is measurable and / _ : lf(xWdx

< oo}.

 

L 2 (1R) is a Hilbert space with respect t o the inner product

( ,g)=/_: f(x)g(x)dx,

J,g E

L 2 (1R).

The spaces L 2 (0), where 0 is a n open subset of 1R are defined similarly. According t o the general definition, a sequence of functions {9k} : , 1 in L 2 (0) converges t o g E L 2 (0) if

119- 9kll

=

(

In

~

lg( x) - 9k(xWdx

~

0

s k ~ oo.

Convergence in L 2 is very different from pointwise convergence. As a positive result we have Ri Ries esz' z' Su Subsequ bsequence ence Theorem Theorem::

The or e m 2.3.1 Let

0

~

IR be an open set, and let {gk} be a sequence

in L 2 (0) which converges to g E L 2 (0). Then {gk} has a subsequence {9nk }f:: 1 such that g(x)

for almost every

X

=

lim 9nk (x) ~ o o

E 0.

The result holds no matter how we choose the representatives for the

equivalence classes. This is typical for for th this is book, where w e rarel rarely y dea deall with a specifi spec ificc repr represen esentativ tativee ffor or a give given n cla class. ss. T The here re are are,, howeve however, r, a few important exceptions. When we speak about a continuous function, it is clear that we have chosen a specific representative, and the same is the case when we discuss Lebesgue points. By definition, a point y E IR is a Lebesgue point for a function f if

.Y+ • l f ( y ) -

1 lim-

f(x)ldx

y- •

f

• ~

= 0.

I f f is continuous in y, then y is a Lebesgue point (Exercise 2.1). More generally, one can prove that if f E L 1 (IR), then almost every y E IR is a Lebesgue point. from the definition that different representatives for the same I t is clear equivalence class will have different Lebesgue points: for example, every y E IR is a Lebesgue point for the function f = 0; changing the definition of f in a single point y will not change the equivalence class, but y will no longer be a Lebesgue point. See Exercise 2.1 for some related observations. In L 2 (1R), Cauchy-Schwarz' inequality states that for all J,g E L 2 (1R),

rX)

1-oo lf(x)g(x)ldx

~

(1-oo roo ) lf(x)l dx 2

1/2 (

) 1-oo lg(xWdx roo

1/2

 

40

2. Infinite-dimensional Vector Spaces and Sequences

The discrete analogue of L 2 (JR) is £2 ( / ) , the space of square summable scalar sequences with a countable index set I :

f2 (/)

:=

{{xk}kEI

C

I L lxkl 2 < oo} kEI

·

f 2 ( / ) is a Hilbert space with respect to the inner product ({xk}, {yk}) =

L

kEI

XkYk;

in this case Cauchy-Schwarz' inequality gives that

I

LXk Yk l kEI

2

L kEI

lxkl 2 L 1Ykl 2 , {xk}kEI, {yk}kEI kEl

E f 2 (J).

2.4

The Four Fourier ier transform transfo rm

L:

For f E L 1 (JR), the Fourie Fourierr transform

j('y)

:=

j

is defined by

(x)e-21fix"'fdx,

"'E

JR.

Frequently we will also denote the Fourier transform of f by F f. I f (£ 1 n L 2 )(JR) is equipped with the L 2 (JR)-norm, the Fourier transform is an isometry from ( £ 1 n L 2 )(JR) into L 2 (JR). I f f E L 2 (JR) and { h } f : 1 is a sequence of functions in ( £ 1 n L 2 )(JR) which converges t o f in £ 2 -sense, then the sequence {A}f: 1 is also convergent in L 2 (JR), with a limit which is independent of the choice of {fk}f: 1 . Defining

J=

lim

k-+oo

Jk

we can extend the Fourier transform to a unitary mapping of £ 2 (JR) onto L 2 (JR). W e will use the same notation to denote this extension. In particular we have Plancherel's equation

( ] , g ) = ( ,g), Vf,g E L 2 (JR), and

llfll = llfll·

(2.3)

J

I f f E L 1 (JR), then j is continuous. I f the function f as well as belong to £1 (JR), the inversion formula formula describes how to come back to f from the

function values j('y):

L:

The or e m 2.4.1 Assume that f , f

f(x)

=

E L 1 (JR). Then

('y)e 2trix"'fd"f, a.e. x E

(2.4)

JR.

The pointwise formula (2.4) holds a t least for all Lebesgue points for

cf. [10).

f,

 

2.5 Operators on L 2 (R)

2.5

41

Opera Op erator torss on L 2 (JR)

In this Section we consider three classes of operators on L 2 {1R) which will play a key role in our analysis of Gabor frames and wavelets. Their definitions are as follows: Translation by a E lR, Ta: L 2 (JR) -+ L 2 (JR), (Taf)(x) Modulation by b E IR, Eb: L 2 (1R) -+ L 2 (1R), (Ebf)(x) Dilation by a

' I 0,

Da:

L 2 (1R) -+ L 2 (1R), ( D af) ( x)

=

= f(x-

=

a);

e 2 ,.ibx f(x);

} o f ( ~ ) .

vial

a

(2.5) (2.6) (2.7)

A comment comment abo ut notation: we will usually skip the brackets and simply write Taf( x) , and similarly for the other operators. Frequently we will also let Eb denote the function x t-+ e2 ,.ibx. W e collect some of the most important properties for the operators in (2.5)-(2. 7):

Lemma 2.5.1 The translation operators satisfy the following:

(i) Ta is unitary for all a E JR. (ii) For each f E L 2 (1R), y t-+ T y f is continuous from lR to L 2 (1R). Similar statements hold for Eb, b E lR and Da, a

Proof.

' I 0.

Let us prove that the operators Ta are unitary. Since E L 2 (1R),

(Taf,g) = ( , T - a g } , V f , g

we see that T ; = T - a. On the other hand, Ta is clearly a n invertible operator with T; ; 1 T_a, so we conclude that T;; T; ; 1 T;. To prove the continuity of the mapping y t-+ T y f we first assume that f is continuous and has compact support, say, contained in the bounded interval [c, d]. For notational convenience we prove the continuity in Yo = 0. he function First, Firs t, fo forr y E ] -

=

=

H

¢(x)

= T y f ( x ) - Ty

0

f(x)

= f(x-

y ) - f(x)

has support in the interval [ -   + c, d + ]. Since we can for any given f > 0 find 8 > 0 such that

f

is uniformly continuous,

for all x E lR whenever JyJ with this choice of 8 we thus obtain that l f ( x - y ) - f(x)l

f

(I:::

<

if(x-

y)-

8;

f(x)i'dx) 'i'

Vd-c+l.

This proves the continuity in the considered special case. The cas casee of an arbitrary arbit rary functi function on f E L 2 (JR) follows by an approximation argument, using

 

42

2. Infinite-dimensional Vector Spaces and Sequences

that the continuous functions with compact support are dense in £   ~ ) (Exercise 2.4). The proofs of the statements for Eb and Da are left t o the

reader rea der (Exercise 2.5). 2.5).

0

Chapters 8-14 will deal with Gabor systems and wavelet systems in £   ~ ) ; both classes consist of functions in £   ~ ) which are defined by compositions compos itions of some some of the operators Ta, Eb and Da. For this reason the following commutator relations are important: e-21riba EbTaf(x)

TaEbf(x)

=

= e2,.ib(x-a) f ( x -

X 1 Jjajf(

n D a f (x)

=

DaTb;af(x)

DaEbf(x)

=

_1_e211"ixbfa f(:._) J j aj a

b ~ ) ,

= EJLDaf(x). a

a),

(2.8) (2.9) (2.10)

In wavelet analysis the dilation operator D 1; 2 plays a special role, and we simply write D f(x)

:=

21 / 2 f(2x).

With this notation, the commutator relation (2.9) in particular implies that

(2.11)

W e will often use the Fourier transformation in connection with Gabor systems and wavelet systems. In this context we need the commutator relations

2. 6

Exercises

2.1 Here we ask the reader to prove some results concerning Lebesgue points. (i) Assume that f : a Lebesgue point.

(ii) Prove that x (iii) Let

f =

- +

C is continuous. Prove that every y E

~ i s

= 0 is not a Lebesgue point for the function X[O,l]·

XQ· Prove that every y f/_

Q is a Lebesgue point, and

that the rational numbers are not Lebesgue points.

of real numbers for which 2 : : ~ 2.2 Find a sequence { k } ~ convergent, but not unconditionally convergent.

ak

is

 

2.6 Exercises

43

2.3 Let { k}f: 1 be a sequence in a Banach space. Prove that absolute

convergence converge nce of

2::::, 1 fk

implies unconditional convergence.

2.4 Complete the proof of Lemma 2.5.1 by showing the continuity of y t-t Ty/ for f E L 2 (JR). 2.5 Prove the statements about Eb and Da in Lemma 2.5.1. 2.6 Prove the commutator relations (2.12).

 

3 Bases

Bases play a prominent role in the analysis of vector spaces, as well in the finite-dimensional as in the infinite-dimensional case. The idea is the same in both cases, namely to consider a family of elements such that all vectors in the considered space can be expressed in a unique way way as a linear combination combinati on of these elements. elements. In th thee infinite-dimensional infinite-dimensional case the situation is complicated: we are forced to work with infinite series, and different concepts of a basis are possible, depending on how we want the series to converge. For example, are we asking for for the th e series series to t o converge with respect to a fixed order of the elements (conditional convergence) or do we want it to converge regardless of how the elements are ordered (unconditional convergence)? W e define the relevant types of bases in general Banach spaces in Section 3.1, but besides this we mainly consider Hilbert spaces. In Section 3.4 we discuss the most important properties of orthonormal bases in Hilbert spaces; we expect th thee reader to have some some basic knowled knowledge ge thiss subject subject.. A slight (but (bu t useful) useful) modification leads t o the definition about thi of Rie sz bases, base s, which whic h arethe tre t reat ated ed in detail in Section Secti on 3.6. 3.6. which O Orth rthono onorma l bases andRiesz Riesz bases satisfy so-called Bessel inequality, is rmal the key to the observation that they deliver unconditionally convergent expansions and can be ordered in an arbitrary way. Sequences satisfying the Bessel inequality are therefore discussed already in Section 3.2. Concret Con cretee examples of bases in function spaces are given in Sections 3. 7 and 3.8, where the basic theory for Fourier series is revisited (again this subject is expected to be known) and Gabor bases as well as wavelet bases for L 2 (1R) are introduced. These sections form the background for Chapters 7-14.

 

46

3. Bases

In the entire chapter, X denotes a Banach space, and 1£ is a Hilbert space with the inner product ( , ·) linear in the first entry. W e will assume that the spaces are separable and infinite-dimensional, and we leave the modifications in the finite-dimensional case t o the reader.

3.1

Bases in Banach spaces

The most fundamental concept of a basis was introduced by Schauder [253] in 1927. It takes place in a Banach space X , and captures the basic idea of having a family of vectors with the property that each f E X has a unique expansion in terms of the given vectors. All bases considered in this book are Schauder bases. Before giving the formal definition, we emphasize once more that a sequence {ek}f=1 in X is an ordered set, i.e., {

k } ~

= {e1,e2, ... }.

Definition 3.1.1 Let X be a Banach space. A sequence o f vectors { k } ~ belonging to X is a (Schauder) basis for X if, for each f E X , there exist unique scalar coefficients {ck(f)}f= 1 such that

f =L 00

k=1

(3.1)

ck(f)ek.

Sometimes we refer to (3.1) as the expansion o f f in the basis { k } ~ Equation (3.1) merely means that the series f = L:r= 1 ck(f)ek converges with respect to th thee chosen chosen order of the th e elem elements ents.. I f the series (3.1) converges unconditionally for each f E X , we say that {ek} is an unconditional is an unconditional basis if and only if basis. One can prove that { k } ~ { eu(k) is a basis for every permutation a of N, cf. [260]. In other words, if {ek} is a basis which is not unconditional, there exists a permutation a for which {eu(k) is not a basis. I t is known that every Banach space which has a basis also has a conditional basis, cf. [233]. Besides the existence of an expansion of each f E X , Definition 3.1.1 asks for uniqueness. This is usually obtained by requiring { ek}r: 1 to be independent independ ent in an approp a ppropriat riatee sens sense. e. In infinite-dimensional infinite-dimensional Banach Bana ch spaces, spaces, different concepts of independence exist: Definition 3.1.2 Let {

k } ~

be a sequence in X . We say that

(i) { k } ~ is linearly independent i f every finite subset o f { linearly independent;

k } ~

is

(ii} { k } ~ is w-independent i f whenever the series L:r= 1 ckfk is con vergent and equal to zero for some scalar coefficients {ck}f=P then necessarily Ck 0 for all k E N.

=

 

3.1 Bases in Banach spaces

{iii) {fk}f: 1 is minimal i f

fJ ¢ span{ k}k;ej, Vj

47

EN .

The relationship between the definitions is as follows: Le mma 3.1.3 Let {fk}k-'= 1 be a sequence i n X . Then the following holds:

{i) I f {fk}k-'= 1 is minimal, then {fk}k-'= 1 is w-independent. (ii) I f { k}k-'= 1 is w-independent, then {fk}f: 1 is linearly independent. The opposite implications i n (i) and (ii) are not valid. Proof.

For the proof of (i), assume that {fk}k-'= 1 is not w-independent.

Cj j , such that Choose scalar coefficients {ck} f : 1 with f= 0 that for some ~ ckfk = 0; then fJ = L k # j =f; fk, implying / j E span{fkh#j· That is, {fk}k-'= 1 is not minimal. The statement (ii) is obvious, and the fact that the opposite implications are not valid is demonstrated by examples in Exercise 3.4. 0

A Banach Bana ch space having a basis is necessarily separable. separabl e. Most of o f the th e known separable Banach spaces have a basis; the first example of a separable Banach space not having a basis was constructed by Enfl.o [122] in 1972. I t is clear that a basis for X is complete and consists of non non-zero -zero vectors. Adding an extra condition leads to a characterization of bases: The or e m 3.1.4 A complete family o f non-zero vectors {ek}f: i n X is 1 t for al a basis for X i f and only i f there exists a cons constant tant K such tha that alll m , n E N with m n,

(3.2) for all scalar-valued sequences {ck}f: 1.

Suppose that {ek}f: 1 is a basis. Then each Ckek, and expansion f = ~ Proof.

f

E X has a unique

Note that if lllflll = 0, then : : ~ = ckekll = 0 for all n E N; it follows that ck = 0 for all k E N, and f = 0. One can check (Exercise 3.1) that Ill · Ill satisfies the other conditions for a norm on X , and that X is a Banach space with respect to this norm. By definition of Ill · Ill, we have II/II lllflll, V f E X , meaning that the identity operator is a continuous and injective mapping of (X, Ill ·Ill) onto (X, II · II). By Theorem A.5.2 i t follows that this operator has a continuous inverse, i.e., that there exists a constant K > 0 such that lllflll K llfll for all f E X . In particular, fixing

 

48

Basess  3. Base

3.2).. F o r tain   ( 3.2) Ckek,, we o b tain ering g f =   ~ = l   Ckek trary   n E N a n d   co n s id erin a n ar b i trary of   no n - z e r o il y   {  k } ~ catio o n , a s s u m e t h a t a   co m p l ete fa m ily th e o t h e r i m p l i cati (3.2). ). Let A   d e n o t e th e v e c t o r  s p ac e c o n s istin g  o f all f E X sfiess (3.2 v e c t o r s s a ti sfie fficie ie n ts {   k } ~ fo r s o m e c o e ffic ckek   for ckek w h i c h   c an b e e x p a n d e d a s   f =   : ~ assum m e d t o b e c o m p l e t e we is assu sincee { e k } ~ irst   we p r o v e t h a t  A = X ; sinc F irst is  cl o se d . L e t densee in X , so i t is e n o u g h   t o p ro v e t h a t  A   is  kn o w   t h a t A is dens oo . C   A su c h t h a t f j -+ f a s   j   - + oo. f  E X , a n d c h o o se a s e q u e n c e { j ~

ij )  } ~ ficien n t s { c ij) ek for or   a p p r o p r i a t e co ef ficie fJ  = l   ~ W r i t e fJ  or   all j , l E N t h a t   e a c h i   E N  a n d a ll n ~   m   ~ i , we h a v e for

lclj) - c)'lllllll•·ll •·ll

<

K 

 0, c h o o se N

K

(11 -ftll+llfz-~ciL)ekll)·

2

E   N  s u c h   t h a t

~

II ~   I I - IJ IJII

for j  ~

N . 

follow low s f ro m t h e   a b ov e   e st i m a t e   t h a t   oo,   it fol i n g  n -+   oo, B y l e t t in ( 3 .4 )

(3.3), ), a n d , v i a th e  i n term e d i a t e   s t e p (3.3

F o r e a c h  i E N , the s e q u e n c e by   (3.4 ) , say, is c o n v e r g e n t by  oo   in   ( 3 .4 ) a n d   (3.5 ) , we o b t a i n  that letti n g l - + oo a s l - +  oo. B y   letti l c ~ j

c i l l l e ill ~

E

Ci

3.6)   ( 3.6)

fo forr a ll i E N , j ~   N ,

and

~ ~ ~

c ~ ) -

C k ) ekll

~

E

fo forr all m E N , j

~  

N.

(3 .7 )

 

in   B a n a c h s p a c e s 3.1 B a ses in 

49

al l j E N, N ow , for g iv en m   E N   an d all

II - ~ c , e • l l II II ·HI,   III,- ~ c ~   e · I H ~   c ~ ) - c·HI, J,lll J,l

  0 w e c a n   ch o o se   N E N so t h a t  (3.7 we   c a n ::;  E; a f t e r   t h at, we JII ::; or   j > N we o b t a i n t h a t I I - IJII l a r g e v a l u e for o b t a i n t hat

II

/ j - 2:::::;;'= 1

ek ek  

II ::;

E 

icien n tl y l a r g e . osing g m E   N s u f f icie b y c h o osin

largee . W e  c o n c l u d e t hat ently y larg suffici ci entl forr m   suffi T hu s I I   - 2:::::;;'= 1 c k e k l l ::; 3€ fo is a b a s i s To   pr o v e t h a t  { e k } ~ esiree d . To as   desir i.e.,, f E   A   as c k e k , i.e. f =  ll  k E N. or   a ll  c k ek = 0, th e n Ck =   0 for  show   t h a t  i f we o n ly n e e d t o  show ck e k = 0 , t h e n   for e a c h i E   N a nd   fo llow lo w s f ro m ( 3 .2): if : : : : : T h i s   a g ai n   fol all n   2:: i ,

-+   oo. oo.   fr o m h ere we o b t a i n th e r e s u l t b y   l ettin g n -+ 

0

fo r a n   asis   co n s t a n t , w h ich for si n g   t h e   b asis often n   f o rm u l a t e d   u sin .1.4   is ofte T h e o r e m   3.1.4 ar b itrary itrary   se q u e n c e {  e k }

{ I I ~

K : =sup  

is d e fi ned by   c k e kll

: m ::; n ,

I I ~   c k e k ll

= 1}.

(3 .8)

t  that c a n b e   us e d   alless t   c o n s t a n t  early y t h e s m alle is   c l earl is  is  a b a s is, t h i s is initee , th e n  { e k } ~ ot he r h a n d , i f the b a sis c o n s t a n t   is in f init 3.2).. O n the othe in   (3.2) defin ned k}f= = 1 th e  b a s i s c o n s t a n t is defi sequee n c e { ek}f initee sequ asis.   Fo r a f init is n ot a b asis. ::;; N . a s a b o v e , w i t h   t h e  a d d i t i o n t h a t  w e  c o n s id e r n   ::

If 

e k } ~

c a n   b e  a b a s i s Th e  b a s is   c o n s t a n t K te l ls w h e t h e r t h e   s eq u e n c e {  e k }   spectt t o t h e c h o s e n o r der o f   th e  e le m e n ts. W e  n o t e   t h a t a s i m i l a r w ith r e spec sequee nce 60]: ]: a c o m p l e t e sequ [260 ition n a l b a s e s e x i s ts, c f. [2 c t e riza t i o n o f u n c o n d itio c h a r a ct tionaa l  b a si s  if a n d o n l y ents  is is   a n u n c o n d i tion n-zerr o  el em ents  off n o n-ze sistin n g  o c o n sisti { ek} 

if i t s   u n c o n d i t i o n a l ba sis c o n s t a n t   sup

kekll   =   1 L:::C C kekll ckee kl klll :   II L::: 2:::0·k :0·kck {112:: {11

an d

a k   =   ±1 ,

'v'k}   'v'k}

inite. e. is f init in ientss { c k   / } ~ fi c ient ek}k"= = 1 i t is c l e a r   t h a t t he  c o ef fic G iv e n   a b a sis {ek}k" early y o n f . Th e m a p p i n g s   f - + c k ( f ) a r e c a l l e d c o e ffic i e n t (3 .1)   d e pe n d l i n earl As   a   c o ns e q u e n c e o f T h e o r e m 3 .1 .4 t h e y a r e   c o nt i n u o u s : f u n c t io n a l s . As

 

3. B a s e s

50

asso ciated   to a basis oeffic ient fu n c tio n a ls {   k } ~ C o r o l la r y 3   1 5 The ccoeffic me n t s in red as  as   e l e me nsidered s   be c o nside a n th u s  f o r X are co n tin u o u s, a n d c an { ek }   alll  the du d u a l X * . I f t here e xists a c o n s t a n t C > 0   s u c h   t h a t llekll 2: C for al u nded. a re un i f o r m ly b o unded. k E N , t h e n the no r ms o f { k } ~

ther e .   G iv e n .1.4   a n d the n o t a t i o n intr o d u c e d there W e u se   T h e o r e m   3.1.4 2:  j , forr  an y j   E N a nd a ll n 2:  k . T h e n ,   fo c k(f)e k(f)ek ~ f   E X , w r ite f =

Proof 

that    L e t t i n g n   --+ oo we o b t a i n that 0

elow . is  is  n o t n o r m - b o u n d e d   b elow here   { e k } ~ for   t h e  ca se   w here rcisee 3 .2 for Se e E x e rcis i n X * a r e s a i d   to b e   i n X a n d   a  s eq u e n c e { g k } ~ sequee n c e { k } ~ A   sequ

thogo n a l if biorthogo bior

. =8

(fj)   g k (fj)

.

·= {  1

k,j k,j .

0

=  j , f. j .

k k

( 3 .9 )  

sis fo r X . T h e n {   k } ~ is a ba basis

C o r o ll ary 3   1 6   S u p p ose th a t { k } ~ ffi cient f u n c t io io n a l s { t h e   c o e ffi

if if

and 

c o n stitu te a b iortho iorthogonal gonal s y s t e m .

} ~

(Exerc rc ise 3 . 3). F o r c o m p l e t e n e s s we W e l ea v e t h e p r o o f to   the r e a d e r (Exe

ction n a ls; t h e y a r e nt f u n ctio efficie ient bout   t h e c o effic low w ing in g   r es u lts a bout ollo m e n t i o n   the f ol 79].   [279]. [210],   [2 p r o v e d i n   e.g., [210], T h e o r e m   3 1 7   Le t   { e k } ~

be a ba sis for   X

an d l e t {

k } ~

be th e  

n  ls.   T h e n  tio n a ls.  fficient   fu n c tio ciated   c o e fficient asso ciated losed sp a n in X * , a n d i t s assoc iated is for for   its cclosed basis (i) {  ck}k"= 1 i s a bas as   e l e m e n t s in  x · · . sidere d   as  (considere dk"=ll   (con em   is { e dk"= system onal syst orthogonal b i orthog exive, t h e n { reflexive, (ii) I f X is refl

3 .2

B essel se q u e n c es

k } ~

basiss fo r X * . is a basi

in H i l b e r t

spaces

(s ee o u r co n lbertt s p a c e s   (se T he r e s t o f t h i s   c h a p t e r c o n c e r n s s e q u e n c e s   in H i lber al l  s eq u e n c e s b y   th e   ntion n s st a ted o n p a g e 4 6). F o r c o n v en ie n ce w e i n d e x   all v e ntio re s u lts a c t u a l l y   that   a ll res seee that so o n se ion.   W e s h a ll soo na t u ral n u m b e r s i n t h i s se c t ion. ntabll e   i n de x   s et s .   rbitra a ry c o u ntab h o ld   w ith a rbitr

 

3.2 Bessel sequences in Hilbert spaces Le mma 3.2.1 Let { k}f: 1 be a sequence i n 1£, and suppose that is convergent for all {ck}f: 1 E f 2 (N). Then

51 ckfk

00

T : f 2 (N) - t 1£, T{ck}f=1 : = "L,.ck/k

(3.10)

k=l

defines a bounded linear operator. The adjoint operator is given by

(3.11) Furthermore, 00

L k=l Proof.

1{ , hW

IITII 2 11/11 2 , 'Vf E 1£.

Consider the sequence of bounded linear operators n

Tn: f 2 (N) - t 1£, Tn{ck}f:,l : =

k=l

Ckfk·

(3.12)

Clearly T n - t T pointwise as n - t oo, s o T is bounded by Theorem A.5.1. In order to find the expression forT*, let f E 1£, {ck}f: 1 E f 2 (N). Then

={ ,L 00

{ , T{ck}f:l)H

k=l

00

ckfk)H

= L,u, /k)ck.

(3.13)

k=l

W e mention two ways to find T* f from here. {ck}f:, 1 E f 2 (N) 1) The convergence of the series E':= 1{ , /k)ck for all {ck}f:, implies that { { , /k) } f : 1 E f 2 (N); see for example [174], page 145. Thus we can write

=

{ , T{ck}f:, 1)H

({ { , /k)}, {ck})e2(N)

and conclude that T* f

= { { , /k)}f:I·

2 Alternatively, when T : f 2 (N) - t 1l is bounded we already know that T* is a bounded operator from 1 l to £2 (N). Therefore the k-th coordinate function is bounded from 1 l to C; by Riesz' representation theorem, T* therefore has the form T* I =

{ , gk)}f:1

for some {gk}':= 1 in 1£. By definition ofT*, (3.13) now shows that 00

00

"L,.{f,gk)ck = "L,.{f,/k)ck, 'V{ck}f:, 1 E f 2 (N), f E 1£. k=l k=l I t follows from here that 9k

= fk.

 

52

3. Bases

The adjoint of a bounded operator T is itsel itselff bounded, bounded, and Under the assumption in Lemma 3.2.1, we therefore have

liT* f l l 2

IITII 2 llfll 2,

IITII

= liT* II·

V f E 1l,

0

which leads to (3.12).

Sequences { f k } ~ for which an inequality of the type (3.12) holds will play a crucial role in the sequel. i n 1l is called a Bessel sequence i f A sequence { f k } ~ there exists a constant B > 0 such that

Definition 3.2.2

00

Ll(f,h)l2 k=1

B llfll 2, V f E 1{.

(3.14)

Every number B satisfying (3.14) is called a Bessel bound for {fk}f= 1. Theorem 3.2.3 Let { h } ~

be a sequence i n 1l. Then { h } ~ sequence with Bessel bound B i f and only i f

is a Bessel

00

T: {

Lckfk

k } ~

k=1

is a well-defined bounded operator from f 2(N) into 1l and

IITII

~

VB.

Proof. First assume that { h } ~

is a Bessel sequence with Bessel bound B . Let {ck}f= {ck}f=1 1 E f 2(N). First we want to show that { c k } ~ is well defined, i.e., that ~ ckfk is convergent. Consider n, m E N, n > m . Then

llt.c·f·- t.c·J.II

~

c.J·II

II.I~ = <

sup

11911=1

I(

t

ckfk,g)l

k=m+l

n

L

sup

11911=1 k = m + l

t

<

(

<

.jjj

k=m+l

lck(fk,g)l

lckl 2)

112

t

sup ( l(fk,g)l 2) 11911= 1 k=m+1

Cf+, hi'),,,

112

Since { k } ~ E f 2(N), we know that {L:Z= 1 ckl 2} : = 1 is a Cauchy se quence in C. The above calculation now shows that {l::Z= 1ckfk} : = 1 is a

 

3.2 Bessel sequences in Hilbert spaces

53

Cauchy sequence in 1£, and therefore convergent. Thus T{ck}r: 1 is well defined. Clearly Tis linea linear; r; since si nce liT {ck}f= 1 ll = sup\\g\\= 1 I(T{ ck}f= 1 , g)l, a calculation as above shows that Tis bounded and that IITII :::; ,fii. For the opposite implication, suppose that T is well-defined and that IITII :::; ,fii. Then (3.12) shows that {h }f = 1 is a Bessel sequence with Bessel bound B . D Lemma 3.2.1 shows that if we only need to know that {fk}r: 1 is a Bessel sequence sequ ence and the value for the Bessel bound is irrelevant, we can just check that the operator T is well defined: is a sequence in 1£ and ~ c k h is convergent for all {ck}r: 1 E t' 2 (N), then {fk}f= 1 is a Bessel sequence.

Corollary 3.2.4 I f {fk}r: 1

The Besse Bessell condition (3.14) remains the t he same, regardless of how how the ele ments { h } r : 1 are numbered. This leads to a very important consequence of Theorem 3.2.3: is a Bessel sequence in 1£, then converges unconditionally for all {ck} r : E t'2 (N).

Corollary 3.2.5 I f {fk}r: 1

~

ckh

1

Thus a reordering of the elements in {fk}r: 1 will not affect the series 2::%"= 1 c k h when {ck}f= {ck}f= 1 is reordered the same way: the series will converge towards the same element as before. For this reason we can choose an arbitrary indexing of the elements in the Bessel sequence; in particular i t is not a restriction that we present all results with the natural numbers as index set. As we will see in the sequel, all orthonormal bases, Riesz bases, and frames are Bessel sequences. I t is enough t o check the Bessel condition (3.14) on a dense subset of 1£: Lemma 3.2.6 Suppose that { k}f= 1 is a sequence o f elements i n 1£ and

that there exists a constant B

>0

L l(f,

such that

00

k=1

k)l 2

:::;

B llfll 2

for all f i n a dense subset V of1l. Then {fk}f= 1 is a Bessel sequence with bound B .

W e have to prove that the Bessel condition is satisfied for all elements in 1£. Let g E 1£, and suppose by contradiction that Proof.

00

L

l(g, k)l 2

> B 11911 2 •

k=l

Then there exists a finite set F C N such that

L

l(g,fk)l 2

> B 11911 2 ·

kEF

 

54

3. Base Basess

l i e s t h a t t he r e e x i s t s h E   V s u c h t h a t   se in 1 { , th is  i m p li Sinc e V i s   d e n se

hll 2 , ,fkW kW   >  B llllh (h,f (h kEF

e   t hat c t i o n . W e c o n c l u d e  b u t th is is a   c o n t r a d i ct 00

k=1

:; fkW::; l(g,,fkW: l(g

B 

llgW,

0

Vg E 1 { .

 spirit . same  S ee E x ercis e   3. 7  f or a r e sult i n th e  same

3 .3   B a s e s a n d b i o r t h o g o n a l   s ys t e m s  

n 1 l

3 .1. T h e f i r s t e f ined i n S e c t i o n 3.1. f   the c o n c e p t s d ef W e n o w r e t urn to  s o m e o f  ffices   to   ll y hold holdss in B an ac h spac es, b u t for ou r p ur p os e i t s u ffices ctuall le m m a a ctua se q u e n c e { k } ~ 1£;   t h u s ,   if a se e 1 { . N o t e t h a t 1 { *   =   1£; t   s p a c e  c o n s i d er a H i l b e r t  ce is a se q u e n ce {gk}   "= 1 , th en  a l s o {gk} t h o g o n a l s e q u e n c e   {gk} ;:;:"= i n 1 {   has a   b i o r th in H .

be be   a s e q u e n c e in  1 { .   T he n

L e m m a 3   3 1 L et {   k } ~ (i)  {   k } ~

h a s   a bi o rtho g o n a l   s eq u e n c e { g k } ~

i f a n d   o nl y i f {

k } ~

is m i n i m a l . ( i i ) I f a   b io r thog o n a l s e q u e n c e   f o r { e x ists, ists,   i t is  is   u n i q u e l y d e t e r  k } ~ te i n H . k '= 1   is c o m pl e te {fk}k m ined i f  a nd   o n l y i f  {fk}

 { g k } ~ stem  ha s  a bio r t h o g o n a l sy stem

S u p p o s e t hat {   k } ~ Proof fo r   a ny g iven j  E N, 

=  1   a n d

(IJ,gj)

(/k,gj)

=  0

fork

=f.

T h en ,

j. 

a l. Fo r   the o t h e r   is m i n i m al. T h e r e f o r e i J   f/: s p a n { h h j , i. e ., {   k } ~ is  m i ni m a l , a nd l e t (i),  a s s u m e t h a t { k } ~ i m p l i c a t i o n i n (i), 

Ho   : =  s Ho

p

n {

k } ~

on   of 1 i o n to ection proj ecti o gona gonall   proj ortho E N, l e t   P j d en o te   the orth G iv en j followss Ho.. T h e n   it follow  on Ho rator  ntity y op e rator   j , a n d l e t   0 b e  the i de ntit {fk}k  span{fk}k span

j)IJ   =f.  t h a t ( / o - P j)IJ f.  0, a nd

(fj, ( Io - P j)fj)

=  (P j i J   + U o -

F o r   k   =f. j   clea rly ( /k , (

P ;) IJ , ( I o - P ; ) IJ )

=  II II(( Io - P j)fjll 2   =f.

0.

definii n g P j) f j ) = 0 B y  defin

0 -

9] : =

( I o - P j)fj 

j)IJII IJII 2 '  II( II ( Io - Pj)

j EN 

"= 1  is a b i o r t h o g o n a l sy s te m fo r   { h } k= 1 . {gkk} ;:;:"= w e o b t ain t h a t  {g

 

3.3 Bases and biorthogonal systems in 1i

55

For the proof of (ii), assume that {fk}f:: 1 has a biorthogonal system {gk} I f {/k}f::1 is not complete we can replace {gk} by {gk + hk}f:: 1 for some hk E 1/.c} \ {0} and hereby obtain a new biorthogonal system for { k}f:: 1. On the other hand, we leave it to the reader to verify that if {fk}f:: 1 is complete, then the biorthogonality condition can a t most be satisfied for one family {gk} 0 Theorem 3.3.2 Assume that {ek}f:: 1 is a basis for the Hilbert space 11.. Then there exists a unique family

{gk}

i n 11. for which

00

f {

} ~

L(f,gk)ek, = k=1

'