Concise Introduction to Wavelets

CONCISE INTRODUCTION TO WAVELETS

Ing. Ren´e Puchinger, 2013

Contents

Contents Preface 1 Introduction 1.1 The space Lp . . . . . . . . . . . . . . 1.2 Fourier transforms . . . . . . . . . . . 1.3 Frames and bases . . . . . . . . . . . . 1.4 The continuous wavelet transform . . 1.5 Discretization of the wavelet transform

i

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

1 1 2 3 6 7

2 Multiresolution analysis 2.1 The definition, scaling functions . . . . . . . . . . . . . . . . . 2.2 Wavelet functions . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Vanishing moments and the order of multiresolution analysis 2.4 Special instances of multiresolution . . . . . . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

10 10 11 12 13

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

3 Construction of orthogonal wavelets 3.1 Further properties of a scaling function . . . . . . . . 3.2 Wavelets induced by a scaling function . . . . . . . . 3.3 Vanishing moments and regularity . . . . . . . . . . 3.4 Construction of compactly supported wavelets . . . . 3.5 Determining φ and ψ from hn . . . . . . . . . . . . . 3.6 Numerical computation of the scaling filter hn . . . . 3.7 The Discrete Wavelet Transform - Mallat algorithm 3.8 Plots of several scaling functions and wavelets . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

17 18 22 24 27 33 35 37 39

4 Construction of biorthogonal wavelets 4.1 Exact reconstruction filters . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Construction of symmetric biorthogonal wavelets with compact support 4.3 Alternative construction - the lifting scheme . . . . . . . . . . . . . . . . 4.4 Lifted wavelet transform . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Interpretation of lifting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Lifting in terms of z-tranform . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Factorization of a polyphase matrix into lifting steps . . . . . . . . . . .

. . . . . . .

. . . . . . .

43 44 46 55 59 61 63 66

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

Chapter 0. Preface

i

Preface This text is based on my research in which I was interested during my former master studies. It may be a valuable resource for any MSc students interested in the theory of wavelets. The whole text is presented as a compilation, it contains research and ideas of respected mathematicians and authors. Bibliography is provided at the end of this document. Ren´e Puchinger.

Chapter 0. Preface

ii

Chapter 1. Introduction

1

Chapter 1

Introduction In this chapter we introduce the basic mathematical tools which are exploited throughout the text. For a more complete treatise we refer to [3, 7, 10, 11, 12, 17, 18, 23].

1.1

The space Lp

For p ∈ N, we define

{ Lp =

∫ } p f : R 7→ C |f (x)| dx < +∞ ,

where the integration is with respect to the whole real line. For each p, Lp is a Banach space with the corresponding norm (∫ )1 p p ∥f ∥p = |f (x)| dx . Of particular importance are L1 , the space of absolutely integrable functions, and L2 , the space of functions of finite energy. The space L2 is a Hilbert space with the inner product ∫ ⟨f, g⟩ = f (x)g(x) dx, where g(x) denotes the complex conjugate of g(x). We will also work with a more general space, L2 = (X , S, µ), where X ⊂ Rn , S is a σ-algebra and µ a corresponding measure. The inner product is then defined as ∫ f g dµ. ⟨f, g⟩ = X

A similar discussion applies to sequences (of complex numbers), denoted by c := (cj )j∈J , where J is a countable index set, which can be finite or infinite. If we define { } ∑ p ℓp (J ) = (cj )j∈J |cn | < +∞ , n∈J

Chapter 1. Introduction

2

for p ∈ N, then ℓp (J ) is a Banach space with the corresponding norm  ∥c∥p = 

∑

1

p

|cj |

p

j∈J

and in particular, ℓ2 (J ) is a Hilbert space with the inner product ∑ ⟨c, d⟩ = cn dn . n

In case J is infinite, we will write just ℓp instead of ℓp (J ).

1.2

Fourier transforms

For f ∈ L1 , the Fourier transform is defined and denoted by ∫ 1 f (x)e−iξx dx. F (ξ) := (Ff )(ξ) = √ 2π

(1.1)

Given F = Ff ∈ L1 , the function f can be reconstructed by the inverse formula as ∫ 1 f (x) = √ F (ξ)eiξx dξ. (1.2) 2π The reason for this particular normalization is the fact that the Fourier transform is then an isometry (see below). The convolution of two functions f, h ∈ L1 is defined by ∫ g(x) = (f ∗ h)(x) = f (t)h(x − t) dt. In terms of the Fourier transform this can be stated as G(ξ) = F (ξ)H(ξ), which is the content of the convolution theorem. A direct consequence of the convolution theorem is the Parseval formula: for f, h ∈ L1 ∩ L2 , we have ⟨Ff, Fh⟩ = ⟨f, h⟩ , (1.3) which means that the Fourier transform preserves inner product. The same applies to the L2 -norm: ∥Ff ∥2 = ∥f ∥2 , (1.4) for f ∈ L1 ∩ L2 . This results immediately from the previous formula, setting h := f , and is refered to as the Plancherel formula. The Fourier transform is thus an isometric operator. When dealing with functions of the Hilbert space L2 , the formula (1.1) cannot be used because it applies only to absolutely integrable functions. However, by a simple limiting

Chapter 1. Introduction

3

process, it can be extended to functions of finite energy, assuming the transform and the Plancherel formula for more specific family of functions (which are also dense in L2 ) is provided. To illustrate this process, take f ∈ L2 . Since the space of smooth functions with compact support (denoted by C0∞ ) is dense in L2 , there exists a Cauchy sequence fn of functions in C0∞ so that limn→+∞ ∥f − fn ∥ = 0. The Fourier transform is well defined for all fn ∈ C0∞ , is in L2 and will be denoted by Fn . By the Plancherel formula (which works also for functions of C0∞ ), Fn is also a Cauchy sequence. Since Fn ∈ L2 and L2 is a Hilbert space, Fn has a (strong) L2 -limit F = lim Fn , n→+∞

which is in L2 and will be called the Fourier transform of f . Such defined extension of the Fourier transform enjoys all the properties of the transform for C0∞ functions [10]. For convenience, we will use the same notation as in case of absolutely integrable functions. The Discrete Time Fourier Transform of (xn )n∈Z ∈ ℓ1 is defined by ∑ X(ξ) = xn e−iξn . (1.5) n∈Z

The inverse transform is given by xn =

1 2π

∫

2π

X(ξ)eiξn dξ,

(1.6)

0

under the assumption X(ξ) ∈ L1 . Using similar arguments as above, the transform can be extended to ℓ2 . With this normalization, the analogue of the Parseval and Plancherel formulas take the form ∫ 2π ∑ (1.7) xn y n , X(ξ)Y (ξ) dξ = 2π 0

and

n∈Z

∫

2π

|X(ξ)|2 dξ = 2π

0

∑

|xn |2 ,

(1.8)

n∈Z

respectively.

1.3

Frames and bases

Frames were introduced in [9] in connection with a class of nonharmonic Fourier series. They provide a general yet stable framework for various function representations. An example of such a representation is ∑ f= ⟨f, ej ⟩ ej , ∀f ∈ L2 , (1.9) j∈J

which applies if the system (ej )j∈J constitutes an orthonormal basis of L2 . An orthonormal basis is in fact a special case of a frame, as pointed out later.

Chapter 1. Introduction

4

An interested reader can find an in-depth discussion of frames, also in connection with wavelets, in [3, 7, 12]. A system (ej )j∈J of functions in L2 is called a frame if for some α, β > 0 and for all f ∈ L2 , ∑ α∥f ∥2 ≤ |⟨f, ej ⟩|2 ≤ β∥f ∥2 . (1.10) j∈J

A frame is said to be exact if, after removing any element from it, the inequality (1.10) no longer applies. In case α = β, the frame is called a tight frame and the definition reads ∥f ∥2 =

1∑ |⟨f, ej ⟩|2 , α

(1.11)

j∈J

which after some manipulations with inner products yields ⟨f, g⟩ =

1∑ ⟨f, ej ⟩ ⟨ej , g⟩ , α j∈J

where g is any function in L2 . Since the inner product is linear in the first argument, it is indeed possible to write ⟩ ⟨ 1∑ ⟨f, ej ⟩ ej , g . ⟨f, g⟩ = α j∈J

With a slight abuse of notation one can rewrite this formula as f=

1∑ ⟨f, ej ⟩ ej . α

(1.12)

j∈J

In fact, one must always be able to recognize that such a representation is merely formal and that only after taking the inner product with any g ∈ L2 it leads to the right formula. It is usual to say that such a formula holds “in the weak sense” [7]. We will use this notation whenever possible. If (ej )j∈J is a tight frame with α = β = 1 and is such that ∥ej ∥ = 1 for all j ∈ J , then the equation (1.11) is precisely the Parseval identity and since ∑ ∑ |⟨ek , ej ⟩|2 , ∥ek ∥2 = |⟨ek , ej ⟩|2 = ∥ek ∥4 + j∈J

j∈J ,j̸=k

it follows that (ej )j∈J is nothing more than an orthonormal basis. In following we see the derivation of a general expansion for arbitrary frames similar to (1.9) or (1.12). If (ej )j∈J is a frame, we define a linear map T : L2 7→ ℓ2 (J) by (T f )j = ⟨f, ej ⟩ , ∀j ∈ J .

(1.13)

By (1.10), T is bounded by a strictly positive constant β. Moreover, its adjoint T ∗ can be computed as ∑ T ∗c = cj ej , (1.14) j∈J

Chapter 1. Introduction

5

in the weak sense. 1) If we define a linear operator S : L2 7→ L2 by S = T ∗ T , then ∑ ⟨Sf, f ⟩ = ∥T f ∥2 = |⟨f, ej ⟩|2 . j∈J

We will call S a frame operator. It can be verified that S is bounded, self-adjoint and invertible. The frame condition (1.10) now can be stated in an operator form (I denotes the identity operator) as αI ≤ S ≤ βI, which implies that S is bounded below by α > 0 and therefore is invertible and S −1 is bounded above by α1 , see [7]. If we now define e˜j = S −1 ej , then it can be checked that the system (˜ ej )j∈J is a frame as well [7], satisfying ∑ 1 1 ∥f ∥2 ≤ |⟨f, e˜j ⟩|2 ≤ ∥f ∥2 , β α j∈J

or, in the operator form, 1 1 I ≤ S −1 ≤ I. β α The system (˜ ej )j∈J is called the dual frame, because the dual frame of (˜ ej )j∈J is again (ej )j∈J . We will also denote T˜ the linear map associated with the dual frame as in (1.13): (T˜f )j = ⟨f, e˜j ⟩ , ∀j ∈ J , and the dual frame operator S˜ is defined as S˜ = T˜∗ T˜. Since S is self-adjoint, we have ⟨ ⟩ ⟨ ⟩ (T S −1 f )j = S −1 f, ej = f, S −1 ej = ⟨f, e˜j ⟩ = (T˜f )j , or,

T˜ = T S −1 ,

which implies and

T˜∗ T = (T S −1 )∗ T = S −1 T ∗ T = S −1 S = I T ∗ T˜ = T ∗ T S −1 = SS −1 = I.

Inserting (1.13) and (1.14) in the last two equalities results in ∑ ∑ f= ⟨f, ej ⟩ e˜j = ⟨f, e˜j ⟩ ej , j∈J

j∈J

in the weak sense. 1)

This follows from

⟨T ∗ c, f ⟩ = ⟨c, T f ⟩ =

∑ j∈J

cj ⟨f, ej ⟩ =

∑ j∈J

cj ⟨ej , f ⟩ .

(1.15)

Chapter 1. Introduction

6

The reconstruction formula (1.15) also implies that any frame (ej )j∈J , ej ∈ L2 generates the space L2 , i.e. the closure of the linear span of (ej )j∈J equals L2 . This means that, given a frame (ej )j∈J , ej ∈ L2 , one can express any f ∈ L2 as f=

∑

cj ej

j∈J

If we also premise that this representation is unique, the frame becomes a basis with the additional condition (1.10). Such obtained basis is called a Riesz basis. An interesting fact is that every Riesz basis is an exact frame (defined above) and conversely, every exact frame is a Riesz basis. One can find the proof in [23]. Assuming that the representation (1.15) is unique (i.e., the frame (ej )j∈J is a Riesz basis) and substituting f := ek , we deduce an interesting biorthogonal relation ⟨ek , e˜j ⟩ = δk,j , where e˜j = S −1 ej and S is the frame operator defined above. The systems (ej )j∈J and (˜ ej )j∈J are therefore said to be mutually biorthogonal systems. According to [23], a system (ej )j∈J in a separable Hilbert space is a Riesz basis if and only if it possesses a biorthogonal system (˜ ej )j∈J which is likewise a Riesz basis.

1.4

The continuous wavelet transform

In this section we will briefly review the (historically) first example of a wavelet transform, neglecting some technical details. We will examine how the continuous transform is defined and what properties must be satisfied to have an inverse transform. For a complete reference, consult e.g. [4, 7, 17]. In subsequent chapters we will investigate a more sophisticated concept which leads to an elegant description of the same problem, but it will then be enlightening to see the connection with the continuous transform introduced here. To define a wavelet one must first specify the function space where the analysis takes place. In this introductory chapter we will consider only the Hilbert space L2 of functions defined on the whole real line (to avoid boundary problems). A wavelet is then be defined as any function ψ ∈ L2 , satisfying the admissibility condition ∫ |Ψ(ξ)|2 dξ = Cψ , with 0 < Cψ < +∞, (1.16) |ξ| where Ψ denotes the Fourier transform of ψ and Cψ is a constant depending only on ψ. This condition is needed in order to have an inverse wavelet transform (defined below). We will see that additional requirements are needed for practical analysis. Practical examples of wavelets will be given later. In many practical situations ψ will also be (absolutely) integrable and thus its Fourier transform Ψ will be continuous which together with (1.16) implies Ψ(0) = 0, or equivalently, ∫ ψ(x) dx = 0,

Chapter 1. Introduction

7

which means that in all practical situations the function ψ must oscillate at least a little. This is the motivation behind the term “wavelet”. As we already indicated, the continuous wavelet transform is defined as CW Tψ (f, a, b) = ⟨f, ψa,b ⟩ , where ψa,b (x) =

1 ψ |a|1/2

(

x−b a

(1.17)

) , ∀a ∈ R \ {0}, b ∈ R

is a family of functions obtained from ψ by translations and dilations. The reason for this particular normalization is the fact that the norm of all ψa,b for all values of a and b is preserved. One can also write CW Tψ (f, a, b) =

1 |a|1/2

(

∫ f (x)ψ

x−b a

) dx,

(1.18)

where the complex conjugation can be omitted if the analyzing wavelet is real. The operator CW Tψ (·, a, b) is, of course, linear with respect to the first variable. To compute the inverse transform one can use the following observation [7]: ∫

+∞∫ +∞

−∞ −∞

1 √ CW Tψ (f, a, b)CW Tψ (g, a, b) da db = 2πCψ ⟨f, g⟩ , a

(1.19)

which can be derived by substituting (1.18) to the left side of (1.19) and using the isometry of the Fourier transform and the admissibility condition (1.16). From (1.19) it already follows the inverse wavelet transform, holding in the weak sense: ∫ +∞∫ +∞ 1 1 √ CW Tψ (f, a, b)ψa,b da db. f= (1.20) 2πCψ −∞ −∞ a The continuous wavelet transform is usually depicted in so-called scalogram, where the x-axis represents the time dilation parameter b and the y-axis the scale parameter a. One example of such scalogram is given in Figure 1.1.

1.5

Discretization of the wavelet transform

The strength of the continuous wavelet transform lies in better time-frequency resolution as opposed to the Short Time Fourier Transform. However, it turns out that, under modest requirements on the analyzing wavelet, the continuous representation tends to be superfluous in order to represent functions. We will discuss this briefly; a full exposition is given in [7] or [4]. The idea is to restrict the “continuous” variables a and b in (1.17) to proper discrete values so that for all a we have ∪ supp(ψa,b ) = R. b

Chapter 1. Introduction

8

Figure 1.1: Signal ‘chirp’ and its scalogram

1 0.5 0 −0.5 −1

0

100

200

300

400

500

scale

time 58 55 52 49 46 43 40 37 34 31 28 25 22 19 16 13 10 7 4 1 50

100

150

200

250 time

300

350

400

450

500

−j This can be accomplished by putting a = a−j 0 , b = b0 a0 k, with a0 , b0 ∈ R, a0 ̸= 1 and j, k ∈ Z, leading to j

ψj,k (x) = a02 ψ(aj0 x − kb0 ). The wavelet transform in its discrete form is then evaluated by means of the wavelet coefficients ⟨f, ψj,k ⟩. Let us examine how such a discretized wavelet transform actually leads to a stable function representation. A function f can be represented by means of the wavelet coefficients {⟨f, ψj,k ⟩} if for some different function g ̸= f the wavelet coefficients {⟨g, ψj,k ⟩} differ from {⟨f, ψj,k ⟩}. This is equivalent to saying that if for all j, k ∈ Z, ⟨f, ψj,k ⟩ = 0 then f is zero. Stability is generally defined as the property of having bounded output for some bounded input. In our situation, a representation is said to be stable if for ∑ bounded ℓ2 -norm of the sequence ⟨f, ψj,k ⟩ the L2 -norm of f is likewise bounded, i.e., if j,k |⟨f, ψj,k ⟩|2 ≤ ε, then ∥f ∥2 ≤ δ. Assuming that the representation is stable, then by putting g=

the property

∑ j,k

√

ε· ( ∑

f 2 j,k |⟨f, ψj,k ⟩|

)1/2 ,

|⟨g, ψj,k ⟩|2 ≤ ε is clearly satisfied and therefore ∑

ε∥f ∥2 = ∥g∥2 ≤ δ, 2 |⟨f, ψ ⟩| j,k j,k

Chapter 1. Introduction

9

implying the lower bound estimate in the definition of a frame: ∑ α∥f ∥2 ≤ |⟨f, ψj,k ⟩|2 ,

(1.21)

j,k

where, in this particular case, we have α = δε . ∑ The converse implication is even more obvious - if j,k |⟨f, ψj,k ⟩|2 is bounded then, due to (1.21), ∥f ∥2 is also bounded. It turns out that for all practical purposes (decay of ψ in frequency and in the independent variable) the second frame estimate, ∑ |⟨f, ψj,k ⟩|2 ≤ β∥f ∥2 , j,k

is automatically satisfied (see below) and hence for a stable function reconstruction the discretized wavelet family {ψj,k } needs to constitute a frame. The following proposition is borrowed from the Daubechies book, giving also estimates on the frame bounds α, β. The proof is challenging and is not reproduced here. Proposition 1.1 (Daubechies, [7]). If ψ and a0 are such that ∑ 2 |Ψ(am inf 0 ξ)| > 0, |ξ|∈[1,a0 ]

sup

m∈Z

∑

|ξ|∈[1,a0 ] m∈Z

2 |Ψ(am 0 ξ)| < +∞,

(1.22)

∑ m −(1+ε) , and if θ(s) = supξ∈R m∈Z |Ψ(am 0 ξ)||Ψ(a0 ξ + s)| decays at least as fast as (1 + |s|) thr with ε > 0, then there exists b0 > 0 such that the the ψj,k constitute a frame for all choices thr b0 < bthr 0 . For b0 < b0 , the following expressions are frame bounds for the ψj,k :   [ ( ) ( )] 1   ∑ ∑ 2 2π 2π 2π 2 α= |Ψ(am θ k θ − k , inf 0 ξ)| −   b0 |ξ|∈[1,a0 ] b0 b0 m∈Z

 ∑ 2π  2 β= sup |Ψ(am 0 ξ)| + b0 |ξ|∈[1,a0 ] m∈Z

k∈Z,k̸=0

 1 ∑ [ ( 2π ) ( 2π )] 2  θ . k θ − k  b0 b0

k∈Z,k̸=0

The conditions on θ and (1.22) are satisfied if, e.g., |Ψ(ξ)| ≤ C|ξ|µ (1 + |ξ|)−ν with µ > 0, µ > ν + 1. We have seen that the wavelet transforms discussed so far lead to efficient function representations. Still, there is no indication of the actual construction of any wavelets such that the family ψj,k constitutes a frame. In some applications, one may also impose additional requirements on ψ such as smoothness, compact support, or it may even be desired for the ψj,k to constitute an orthonormal basis. The aim of the subsequent chapters is to present a concise exposition of several methods dealing with construction of wavelets with desired properties. The underlying structure for these methods is so called multiresolution analysis to which the first chapter is devoted.

Chapter 2. Multiresolution analysis

10

Chapter 2

Multiresolution analysis The concept of multiresolution is considered to be vital in the theory of wavelets. It originates from S. Mallat [16] and since then it has been used as a tool for various wavelet constructions. The definition presented here is due to W. Sweldens [21], for the sake of generality it provides. After discussing the basic concept, we will briefly review three important cases, which will be further elaborated in subsequent chapters.

2.1

The definition, scaling functions

For this moment we will stick with a more general setting involving a general function space L2 = (X , S, µ), see the introduction. For the time being, in terms of complexity this amounts to the same setting as with our primary space L2 . In the subsequent two chapters we will, however, find a necessity to abstract away from this general concept, but later, in Chapter 4, we will return to this general setting. A (primal) multiresolution analysis1) is defined as a sequence of closed subspaces Vj ⊂ L2 , j ∈ J ⊂ Z, satisfying 1. For all j ∈ J , Vj ⊂ Vj+1 , ∪ 2. j∈J Vj = L2 , ∩ 3. If J = Z, then also j∈J Vj = {0}, 4. For all j ∈ J , there exists a family of (primal) scaling functions {φj,k | k ∈ Kj } constituting a Riesz basis of Vj , where Kj is a set of indices satisfying Kj ⊂ Kj+1 . We assume that either J = Z or J = Nj0 , where Nj0 = {j0 , j0 + 1, j0 + 2, . . .} for some j0 ∈ Z. Recalling the introductory material, we know that for a family of scaling functions {φj,k | k ∈ Kj } constituting a Riesz basis there exists a family {φ˜j,k | k ∈ Kj }, likewise constituting a Riesz basis of some V˜j . The latter family is referred to as dual scaling 1)

Instead of “analysis”, one may also use the terms “representation” or “approximation”. We will occasionally refer to the same thing simply as “multiresolution”

Chapter 2. Multiresolution analysis

11

functions and the sequence V˜j is naturally defined as a dual multiresolution analysis. Moreover, there is a biorthogonal relation between φj,k and φ˜j,k : ⟨φj,k1 , φ˜j,k2 ⟩ = δk1 ,k2 , for all k1 , k2 ∈ Kj and fixed j ∈ J . A basic property of multiresolution analysis is a two-scale relation, also referred to as a refinement or dilation equation: ∑ φj,k = hj,k,l φj+1,l . (2.1) l∈Kj+1

It follows from the fact that φj,k ∈ Vj ⊂ Vj+1 and {φj+1,l | l ∈ Kj+1 } is a Riesz basis of Vj+1 .

2.2

Wavelet functions

One can utilize scaling functions we have just introduced, to approximate any function f in our space L2 , in the sense that lim ∥f − Pj f ∥ = 0,

j→+∞

∑ where Pj = k∈Kj ⟨·, φ˜j,k ⟩ φj,k denotes the projection from L2 onto Vj and the norm is induced by the metric defined on X . To capture the missing details owing to these approximations, Sweldens defined the space Wj as a complement of Vj in Vj+1 , intended as Vj+1 = Vj ⊕ Wj , where the sum is direct. This is a slight extension of the original Mallat definition, which we will see later. If for all j ∈ J , Wj has a Riesz basis denoted by {ψj,m | m ∈ Mj }, where Mj = Kj+1 \ Kj and V˜j ⊥ Wj , we call {ψj,m | m ∈ Mj } a family of wavelet functions, or just wavelets. From this definition one may deduce several facts. First off, for each j ∈ J there exists ˜ j denotes the space they a dual set of wavelet functions, call them {ψ˜j,m | m ∈ Mj }. Let W generate. Analogously to the case of scaling functions there is a relation between primal and dual wavelet functions ⟩ ⟨ ψj,m1 , ψ˜j,m2 = δm1 ,m2 , ∀m1 , m2 ∈ Mj . The equality Vj+1 = Vj ⊕ Wj can be iterated. In case there is a coarsest level j0 , we obtain j ⊕ Wl , Vj+1 = Vj0 ⊕ l=j0

whilst the second multiresolution property ensures that L2 = Vj0 ⊕

+∞ ⊕ l=j0

Wl .

(2.2)

Chapter 2. Multiresolution analysis

12

If J = Z, then due to the third property it is possible to write j ⊕

Vj+1 =

Wl ,

l=−∞

L2 =

+∞ ⊕

Wl .

l=−∞

It can be suspected that in case of wavelets there should be likewise some relation between scales. Naturally, since ψj,m ∈ Wj ⊂ Vj+1 , it follows that ∑

ψj,m =

gj,m,l φj+1,l .

(2.3)

l∈Kj+1

2.3

Vanishing moments and the order of multiresolution analysis

The dual wavelet functions ψ˜j,m are said to have N vanishing moments, if for all r ∈ N0 , r < N and j ∈ J , m ∈ Mj , ∫ Pr ψ˜j,m dµ = 0, X

where Pr ∈ C ∞ are linearly independent2) functions defined on X and P0 ≡ 1. The fact that the dual wavelets ψ˜j,m have N vanishing moments can also ⟨ be expressed ⟩ ˜ ˜ in terms of the corresponding scaling functions. Since Wj ⊥ Vj , we have ψj,m , φj,k = 0, for all possible indices, implying ⟨

∑

⟩ (r) cj,k φj,k , ψ˜j,m

= 0,

k∈Kj (r)

where cj,k ̸= 0 are arbitrarily chosen. But we also have ⟩ ⟨ P r , ψ˜j,m = 0, ∀r ∈ N0 , r < N. Combining these two equations together yields Pr =

∑

(r)

cj,k φj,k , ∀r ∈ N0 , r < N,

k∈Kj

at least in the weak sense. Hence the scaling functions reproduce up to N linearly independent, C ∞ functions. 2)

In the sense that “the restrictions of a finite number of these functions to any ε-ball are linearly independent.” [21]

Chapter 2. Multiresolution analysis

13

Conversely, if the scaling functions φj,k reproduce up to N linearly independent, C ∞ functions, then ∫ ⟩ ⟨ ⟩ ∑ (r) ⟨ Pr ψ˜j,m dµ = P r , ψ˜j,m = cj,k φj,k , ψ˜j,m , ∀r ∈ N0 , r < N, X

k∈Kj

˜ j ⊥ Vj , the right side of the equation evaluates to zero. but since W All in all, if ψ˜j,m have N vanishing moments or if φj,k reproduce N linearly independent, ∞ C functions, we say that the order of the multiresolution analysis is N . The same applies to the dual multiresolution analysis. The (primal) wavelets ψj,m are ˜ vanishing moments if said to have N ∫ ˜, Pr ψj,m dµ = 0, ∀r ∈ N0 , r < N X

or, by means of scaling functions, ∑ (r) ˜. P˜ r = c˜j,k φ˜j,k , ∀r ∈ N0 , r < N k∈Kj

˜. In this case we say that the order of the dual multiresolution analysis is N

2.4

Special instances of multiresolution

Now that we have established a fairly general framework, we will examine specific properties of multiresolution, assuming some additional conditions may be specified.

Multiresolution leading to (classical) orthogonal wavelets Herein we impose these essential constraints on a multiresolution analysis {Vj | j ∈ J }: 1. The underlying space where the analysis is done, is the space L2 of functions of finite energy and the index set J is all Z. 2. The scaling functions φj,k are obtained from one single function φ by φj,k = 2j/2 φ(2j · −k),

(2.4)

with j, k ∈ Z. 3. The space Wj which is now defined as the orthogonal complement of Vj in Vj+1 is required to possess an orthonormal basis. If this is the case, we will call the wavelets orthogonal. Without loss of generality, we may suppose that the scaling functions φj,k = 2j/2 φ(2j · −k) constitute an orthonormal basis of Vj , for if this was not the case, then obtaining a new, orthonormal basis φ# j,k is only a matter of writing    j   2 (Fφ)(ξ) 2 −1  j  √ F φ# = (∑ ) 1  (2 · −k), j,k 2π  2 2 |(Fφ)(ξ + 2lπ)| l∈Z

Chapter 2. Multiresolution analysis

14

where F and F −1 denote the forward and inverse Fourier transform operators, respectively. We will tackle the proof in the next chapter which is devoted specially to this kind of multiresolution. Note that, since any function f ∈ Vj can now be expressed as ∑ f= ⟨f, φj,k ⟩ φj,k , k∈Z

the dual multiresolution analysis V˜j coincides with the primal, Vj . In the following chapter, we will also discuss the third constraint in more depth, for it is not immediately obvious that the space spanned by wavelets automatically has an orthonormal basis. However, it will emerge that this is always possible, provided we have a proper multiresolution analysis with the first two additional requirements mentioned above. Moreover, it will turn out that the resulting wavelets likewise enjoy the property of being obtained from one fixed function, in the sense that ψj,k = 2j/2 ψ(2j · −k).

(2.5)

It is of no surprise that the two-scale relation takes somewhat more specific form. Since φ ∈ V0 ⊂ V1 and due to (2.4), it is possible to write ∑√ 2hk φ(2 · −k), hk ∈ ℓ2 (2.6) φ= k∈Z

or, in terms of the Fourier transform, ξ 1 ∑ Φ(ξ) = √ hk e−i 2 k Φ 2 k∈Z

This can be simplified to 1 Φ(ξ) = √ H 2

( ) ξ . 2

( ) ( ) ξ ξ Φ , 2 2

(2.7)

∑ where H(ξ) = k∈Z hk e−iξk is the Discrete Time Fourier Transform of the sequence hk . We will refer to the sequence hk (or H(ξ), interchangeably) as a filter; later it will become obvious why. The sequence hk defined above is in ℓ2 , for we have hk = ⟨φ, φ1,k ⟩ and, by the Parseval identity, ∑ |hk |2 = 1. k∈Z

As a consequence of the Dominated Convergence Theorem [18, 11], the interchange of the sum and the integral when taking the Fourier transform of (2.6), is possible. This possibility appears commonly throughout the text; additional comments may be found in [22]. A vital property of such multiresolution is that of scale, meaning that f ∈ Vj is equivalent to f (2 ·) ∈ Vj+1 . This is, again, a direct consequence of (2.4), since f ∈ Vj is expressible as ∑ ∑ f= cj,k φj,k = cj,k 2j/2 φ(2j · −k), k∈Z

k∈Z

Chapter 2. Multiresolution analysis but then also f (2 ·) =

∑

c′j,k 2

15

j+1 2

φ(2j+1 · −k) =

∑

c′j,k φj+1,k ,

k∈Z

k∈Z

where cj,k = ⟨f, φj,k ⟩ and c′j,k = 2− 2 ⟨f, φj,k ⟩. Hence f (2 ·) ∈ Vj+1 . Equally obvious is the property of invariance under dyadic integer translation, namely, f ∈ Vj is equivalent to f (· − 2−j m), with m ∈ Z. Since we are working with the space L2 , the property of vanishing moments may be simplified to polynomial reproduction. In other words, one may take polynomials of degree r as the linearly independent, C ∞ functions we denoted as Pr . We then say that the wavelet ψ has N vanishing moments if ∫ xr ψ(x) dx = 0, ∀r ∈ N0 , r < N. 1

With respect to the ∑ scaling function, this means that any polynomial of degree less than N is expressible as k∈Z ck φ(· − k). Later we will see how the number of vanishing moments is connected with regularity of wavelets. As we have already indicated, the details of this type of multiresolution analysis are further elaborated in the following chapter, along with a possible construction of compactly supported wavelets which are of particular importance.

Multiresolution leading to (classical) biorthogonal wavelets This setting is fairly similar to that of orthogonal wavelets. The main difference is that now we work in a fully biorthogonal manner, which has several significant consequences. We will consider that the space Vj spanned by scaling functions has merely a Riesz basis φj,k . The space spanned by the dual scaling functions φ˜j,k is again denoted by V˜j . The space Wj is defined as a complement of Vj in Vj+1 , in the sense that the sum Vj+1 = Vj ⊕ Wj is direct, with Wj ⊥ V˜j . Hence we respect this multiresolution property in its most general form. What remains the same as in the previous case, is the fact that all the work is accomplished in L2 , under the requirement φj,k = 2j/2 φ(2j · −k),

φ˜j,k = 2j/2 φ(2 ˜ j · −k).

This means that there are two scaling equations, φ=

∑√

2hk φ(2 · −k),

k∈Z

or,

ξ 1 ∑ hk e−i 2 k Φ Φ(ξ) = √ 2 k∈Z

φ˜ =

∑√

˜ k φ(2 2h ˜ · −k),

(2.8)

k∈Z

( ) ξ , 2

1 ∑ ˜ −i ξ k ˜ ˜ hk e 2 Φ Φ(ξ) =√ 2 k∈Z

( ) ξ . 2

The property of scale and the notion of vanishing moments remain the same. We will study biorthogonal wavelets in more depth in Chapter 4.

Chapter 2. Multiresolution analysis

16

Multiresolution leading to second generation wavelets In this case, we establish no further restrictions on the underlying multiresolution. The space L2 = (X , S, µ) is much too general to utilize the Fourier transform as a construction tool. Besides, the scaling functions are not necessarily translated and dilated versions of one fixed function, so the Fourier transform would be practically of no effect. Instead, we take advantage of a promising technique known as lifting, whose power lies also in the fact that it provides a consistent way of building the classical wavelets we briefly discussed above, as well. Lifting may even shed new light on the process and purpose of wavelet analysis, for the principle may seem somewhat more “digestible” than that of classical treatise. Lifting is introduced in section 4.3.

Chapter 3. Construction of orthogonal wavelets

17

Chapter 3

Construction of orthogonal wavelets In this chapter we consider a classical multiresolution setting as proposed by S. Mallat. From the vast range of literature, this survey is mainly based on [6, 7, 13, 15, 17]. For clarity, we begin by summarizing the requirements we have already imposed on a multiresolution analysis (leading to classical wavelets). It consists of closed subspaces Vj ⊂ L2 , satisfying 1. Vj ⊂ Vj+1 , ∀j ∈ Z 2. 3.

∪ j∈J

∩ j∈J

Vj = L2 , Vj = {0},

4. For all j ∈ J , there is a family of scaling functions {φj,k | k ∈ Z} constituting a Riesz basis of Vj . The scaling functions φj,k are obtained from one single function φ by φj,k = 2j/2 φ(2j · −k). We have seen that this definition necessarily implies the property of scale, that is f ∈ Vj ⇔ f (2 ·) ∈ Vj+1 . In many situations, however, it will prove useful to involve merely the family φ0,k = φ(· − k) as a Riesz basis of V0 under the hypothesis of the property of scale. More precisely, if φ0,k = φ(· − k) is a Riesz basis of V0 , and if f ∈ Vj is equivalent to f (2 ·) ∈ Vj+1 , then φj,k = 2j/2 φ(2j · −k) is a Riesz basis of Vj . The reader can convince himself of this fact simply by using the definition of a Riesz basis. In the following few sections we discuss further properties of a scaling function, followed by arguments leading to a simple evaluation of the corresponding wavelet. After discussing a criterion for good approximation properties of the desired wavelet we turn our attention to an actual possibility of wavelet construction. This, as we will see, reduces to finding an appropriate scaling function. We conclude by deriving a fast algorithm of the Discrete Wavelet Transform, which fits neatly into the concept of multiresolution.

Chapter 3. Construction of orthogonal wavelets

3.1

18

Further properties of a scaling function

We begin by deriving two auxiliary lemmas for assessing the property of orthonormality and that of Riesz basis in terms of the Fourier transform. Lemma 3.1. Let f ∈ L2 . The system {f (· − n) | n ∈ Z} is orthonormal if and only if ∑

|F (ξ + 2kπ)|2 =

k∈Z

1 , a.e. 2π

Proof. The system {f (· − n) | n ∈ Z} is orthonormal if and only if ⟨f (· − k), f (· − l)⟩ = δk,l or, equivalently, ⟨f, f (· − l)⟩ = δ0,l . Due to isometry of the Fourier Transform, ∫ ⟨f, f (· − l)⟩ = ⟨F, Ff (· − l)⟩ = F (ξ)F (ξ)eiξl dξ ∫ = ∫

|F (ξ)| e

2 iξl

dξ =

∑∫

2(k+1)π

|F (ξ)|2 eiξl dξ

k∈Z 2kπ 2π

= 0

∑

∫

|F (ξ + 2kπ)| e e

2 iξl 2πikl

2π

eiξl

dξ = 0

k∈Z

∑

|F (ξ + 2kπ)|2 dξ.

k∈Z

The last expression has a form of an inverse Fourier Transform, implying ∑ Discrete Time 1 2 that ⟨f, f (· − n)⟩ = δ0,n is equivalent to k∈Z |F (ξ + 2kπ)| = 2π .  Lemma 3.2. Let S = {φ(· − n) | n ∈ Z} be a set of functions generating the space V0 . The set S is a Riesz basis of V0 if and only if ∑ A≤ |Φ(ξ + 2kπ)|2 ≤ B, a.e., (3.1) k∈Z

for some strictly positive A, B. ∑ Proof. Let f ∈ V0 with the expansion f (x) = n λn φ(· − n), λn not necessarily unique. Taking the Fourier transform of both sides gives F (ξ) = Λ(ξ)Φ(ξ), where Λ(ξ) is 2π-periodic. It follows ∫ ∫ 2 2 ∥f ∥ = |F (ξ)| dξ = |Λ(ξ)|2 |Φ(ξ)|2 dξ

=

∑∫

2(k+1)π

∫

2π

|Λ(ξ)| |Φ(ξ)| dξ = 2

2

0

k∈Z 2kπ

∫ = 0

2π

|Λ(ξ)|2

∑ k∈Z

∑

|Λ(ξ + 2kπ)|2 |Φ(ξ + 2kπ)|2 dξ

k∈Z

|Φ(ξ + 2kπ)|2 dξ.

(3.2)

Chapter 3. Construction of orthogonal wavelets

19

If Φ satisfies (3.1) then ∫

2π

∥f ∥ ≤ B 2

∫ |Λ(ξ)| dξ 2

∥f ∥ ≥ A

and

0

|Λ(ξ)|2 dξ,

0

that is to say, B

2π

2

−1

∫ ∥f ∥ ≤

2π

2

|Λ(ξ)|2 dξ ≤ A−1 ∥f ∥2 .

(3.3)

0

From the discrete version of the Parseval formula it follows that ∑ (2πB)−1 ∥f ∥2 ≤ |λn |2 ≤ (2πA)−1 ∥f ∥2 . n

∑ It suffices to show linear independence. If f = n λn φ(· − n) = 0 then by (3.3), Λ(ξ) is zero almost everywhere and hence λn = 0, ∀n ∈ Z. The set S is therefore a Riesz basis of V0 . We show the converse implication by contradiction. Let S be a Riesz basis of V0 , implying ∑ ∀f ∈ V0 , f (x) = λn φ(x − n), n

with the Riesz basis property (3.3). For any such f , (3.2) is also satisfied. If for either A > 0 or B > 0 the condition (3.1) was not true, then in case of the lower bound A, it is possible to construct a non-zero 2π periodic function Λ(ξ) defined for such ξ˜ for which ∑ 2 ˜ n |Φ(ξ + 2kπ)| < A. But due to (3.2) we deduce a contradiction with the Riesz basis property (3.3).  It is then simple to orthonormalize any Riesz basis {φ(· − n) | n ∈ Z} by putting Φ(ξ) 1 Φ# (ξ) = √ (∑ ) , 2 1/2 2π |Φ(ξ + 2kπ)| k∈Z

a.e.

By lemma 3.2, the denominator is assured to be strictly positive almost everywhere and thus Φ# (ξ) is well defined almost everywhere. Since 1 Φ(ξ) Φ# (ξ) = √ (∑ = Λ(ξ)Φ(ξ), ) 2 1/2 2π k∈Z |Φ(ξ + 2kπ)| with 2π-periodic Λ(ξ), we have φ# =

∑

λn φ(· − n),

n∈Z

therefore the system φ# (· − n) generates V0 . Orthonormality follows from lemma 3.1. The following necessary condition will prove to be fundamental in our development. It states how the orthonormality of {φ(· − n) | n ∈ Z} reflects on the filter H(ξ) appearing in the two-scale relation.

Chapter 3. Construction of orthogonal wavelets

20

Theorem 3.3. Let {φ(· − n) | n ∈ Z} be an orthonormal system in V0 . Then |H(ξ)|2 + |H(ξ + π)|2 = 2, a.e., where H(ξ) is given by the two-scale relation, 1 Φ(ξ) = √ H 2

( ) ( ) ξ ξ Φ . 2 2

(3.4)

Proof. Orthonormality implies ∑

|Φ(ζ + 2kπ)|2 =

k∈Z

1 , a.e. 2π

Applying (3.4) yields ) 2 ( ) 2 ∑ 1 ( ζ H Φ ζ + kπ = 1 , a.e., + kπ 2 2 2 2π k∈Z

or, substituting ξ = ζ/2, ∑1 1 = |H(ξ + kπ)|2 |Φ(ξ + kπ)|2 2π 2 k∈Z

=

] 1 ∑[ |H(ξ + 2kπ)|2 |Φ(ξ + 2kπ)|2 + |H(ξ + 2kπ + π)|2 |Φ(ξ + 2kπ + π)|2 2 k∈Z

[ ] ∑ ∑ 1 2 2 2 2 = |H(ξ)| |Φ(ξ + 2kπ)| + |H(ξ + π)| |Φ(ξ + 2kπ + π)| 2 k∈Z

k∈Z

=

1 (|H(ξ)2 + |H(ξ + π)|2 ). 4π

We have taken advantage of splitting the sum in even an odd indexes (since the terms in the sum are absolutely integrable).  The lemma below states that a reasonable scaling function φ satisfies |Φ(0)| = which will prove to be useful in later development.

√1 , 2π

Lemma 3.4. Let {φ(· − n) | n ∈ Z} be an orthonormal system of functions such that Φ is 2π-periodic and continuous in 0. Let H(ξ), given by (2.7), be 2π-periodic and continuous and let H(ξ) satisfies |H(ξ)|2 + |H(ξ + π)|2 = 2. Then necessarily ∫ 1 |Φ(0)| = φ(x) dx = √ . 2π

Chapter 3. Construction of orthogonal wavelets

21

Proof. Evaluating the equation |H(ξ)|2 + |H(ξ + π)|2 = 2 in ξ = 0 yields |H(0)| ≤

√

√

2.

√ We show that |H(0)| = 2 by contradiction. If |H(0)| < 2, then √12 |H(ζ)| < 1 for ζ small enough. By (2.7), for all ξ ∈ R, ( ) ( ) ( ) ( ) ( ) ξ 1 ξ 1 ξ ξ ξ Φ(ξ) = √ Φ H = √ Φ = ... = H H 2 2 2 2 2 2 2 2 2 ( 2) ( ) ( ) ( ) 1 ξ ξ ξ = √ Φ H ...H , m m m 2 2 2 ( 2) for m ≥ 1. Taking a limit m → +∞, we would derive ∀ξ Φ(ξ) = 0, which would mean that φ is identically zero (up √to a zero measure). This case was excluded in the definition. Knowing that |H(0)| = 2 and that H is 2π-periodic, we have √ |H(2kπ)| = 2. Since 1 |Φ(2q+1 2kπ)| = √ |Φ(2q 2kπ)| |H(2q 2kπ)| = |Φ(2q 2kπ)|, ∀q ≥ 0, k ∈ Z, 2 we deduce |Φ(2q+1 2kπ)| = |Φ(2kπ)|. For k ̸= 0, the Riemann-Lebesgue lemma (lim|ξ|→+∞ Φ(ξ) = 0) ensures Φ(2kπ) = 0. By lemma 3.1, ∑ ∑ 1 = |Φ(ξ + 2kπ)|2 = |Φ(ξ + 2kπ)|2 + |Φ(ξ)|2 . 2π k∈Z

k∈Z, k̸=0

Specially taking ξ = 0, we obtain |Φ(0)| =



√1 . 2π

We conclude this section by discussing a possible construction of a multiresolution analysis, starting from a suitable choice of a scaling function φ, not from the sequence Vj itself. If φ satisfies the two-scale relation ∑ φ= hn φ(2 · −n), (3.5) n∈Z

with hn ∈ ℓ2 and the Riesz basis property ∑ A≤ |Φ(ξ + 2kπ)|2 ≤ B, a.e.,

A, B > 0,

(3.6)

Vj = clos span{φj,k = 2j/2 φ(2j x − k) | k ∈ Z}.

(3.7)

k∈Z

we may define The property Vj ⊂ Vj+1 is a result of (3.5), and together with (3.6) it follows that φj,k ∪ is a Riesz basis of V . It comes to question whether the conditions j j∈J Vj = L2 and ∩ j∈J Vj = {0} are satisfied. This is addressed by the following proposition. The proof in [13] is elegant, albeit somewhat technical, so we decided to left it out.

Chapter 3. Construction of orthogonal wavelets

22

Proposition∩3.5. If φ is such that (3.5) and (3.6) hold, then the {Vj | j ∈ Z} defined by (3.7) satisfy j∈J Vj = {0}. Furthermore, if |Φ(ξ)| is continuous at 0 and if Φ(0) ̸= 0, ∪ then j∈J Vj = L2 .

3.2

Wavelets induced by a scaling function

In this section we will present a derivaion of an explicit formula for an orthogonal wavelet ψ ∈ W0 , meaning that the family {ψ0,m | m ∈ Z} is to constitute an orthonormal basis of W0 . As in case of a scaling function, the multiresolution framework ensures that the family {ψj,m = 2j/2 ψ(2j · −m) | m ∈ Z} then automatically constitutes an orthonormal basis of Wj . We begin by characterizing the space W0 . Since W0 is the orthogonal complement of V0 in V1 , the condition ∑ f ∈√W0 is equivalent to f ∈ V1 and f ⊥ V0 . The property f ∈ V1 is equivalent to f = k∈Z 2λn φ(2 · −k), or ( ) ( ) ξ ξ 1 Φ , F (ξ) = √ Λ 2 2 2 for 2π-periodic Λ ∈ L2 . On the other hand, the property f ⊥ V0 is equivalent to ⟨f, φ(· − k)⟩ = 0, ∀k ∈ Z, or ∫ 0=

F (ξ)Φ(ξ)e

iξk

dξ =

∑∫ l∈Z

∫

2π

eiξk

= 0

That is to say,

∑

∑

2π(l+1)

F (ξ)Φ(ξ)eiξk dξ

2πl

F (ξ + 2πl)Φ(ξ + 2πl) dξ.

l∈Z

F (ξ + 2πl)Φ(ξ + 2πl) = 0, a.e.

l∈Z

Substituting 1 F (ξ + 2πl) = √ Λ 2 and

1 Φ(ξ + 2πl) = √ H 2

yields

∑ l∈Z

( Λ

l∈Z

) ( ) ξ ξ + πl Φ + πl 2 2

(

) ( ) ξ ξ + πl Φ + πl 2 2

) ( ) ( ) 2 ξ ξ ξ + πl H + πl Φ + πl = 0, a.e., 2 2 2

which can be rewritten, putting ξ := ∑

(

ξ 2

and splitting the sum in even and odd indices, as

Λ(ξ)H(ξ) |Φ(ξ + 2πl)|2 +

∑ l∈Z

Λ(ξ + π)H(ξ + π) |Φ(ξ + 2πl + π)|2

Chapter 3. Construction of orthogonal wavelets

23

) 1 ( Λ(ξ)H(ξ) + Λ(ξ + π)H(ξ + π) = 0, a.e., (3.8) 2π where we utilized the orthonormality of φ(· − k). From the last equation we can determine Λ(ξ) more closely. For ξ such that H(ξ) = 0 we have Λ(ξ + π)H(ξ + π) = 0, but due to orthonormality of φ(· − k), we also have |H(ξ)|2 + |H(ξ + π)|2 = |H(ξ + π)|2 = 2, implying Λ(ξ + π) = 0, so the equation (3.8) applies if H(ξ) = 0. For ξ such that H(ξ) ̸= 0 we can write =

Λ(ξ) = −

Λ(ξ + π)H(ξ + π) = θ(ξ)H(ξ + π), a.e. H(ξ)

where θ(ξ) ∈( L)2 is 2π-periodic and satisfies θ(ξ) = −θ(ξ + π), a.e. If we now define ξ ν(ξ) = ei 2 θ 2ξ then ν(ξ) ∈ L2 is 2π-periodic and it follows that θ(ξ) = e−iξ ν(2ξ). Together, Λ(ξ) = e−iξ H(ξ + π)ν(2ξ). Combining various pieces together, we finally obtain ) ( ) ( ξ 1 −i ξ ξ 2 +π Φ ν(ξ), F (ξ) = √ e H 2 2 2 {

thus W0 =

} ( ) ( ) ξ 1 ξ ξ f ∈ L2 (Ff )(ξ) = √ e−i 2 H +π Φ ν(ξ) , 2 2 2

where ν ∈ L2 is 2π-periodic. We would like to characterize a ψ ∈ W0 so that ψ(· − m) constitute an orthonormal basis of W0 . Our assertion is that such ψ is given in terms of its Fourier transform by ( ) ( ) 1 −i ξ ξ ξ Ψ(ξ) = √ e 2 H +π Φ µ(ξ), (3.9) 2 2 2 where µ is 2π-periodic and |µ(ξ)| = 1, for almost all ξ. Specially, taking µ(ξ) ≡ 1, the formula can be rewritten as ∑ (3.10) ψ= (−1)n h1−n φ(2 · −n). n∈Z

We first prove that such a ψ leads to an orthonormal basis of W0 , in the sense that ψ(· − m) are orthonormal and generate W0 . Orthonormality follows from ( ) 2 ( ) 2 ∑ 1 ∑ ξ ξ 2 |Ψ(ξ + 2kπ)| = H + kπ + π Φ + kπ 2 2 2 k∈Z

1 = 2

k∈Z

( ( ) 2 ∑ ( ) 2 ( ) 2 ∑ ( ) 2 ) ξ ξ ξ ξ H Φ Φ + π + 2kπ + H + 2kπ + π 2 2 2 2 k∈Z

k∈Z

=

1 1 1 · ·2= . 2 2π 2π

Chapter 3. Construction of orthogonal wavelets

24

We already know that f ∈ W0 is equivalent to ξ 1 F (ξ) = √ e−i 2 H 2

or, ξ 1 F (ξ) = √ e−i 2 H 2

(

(

) ( ) ξ ξ +π Φ ν(ξ), 2 2

) ( ) ξ ξ ν(ξ) ν(ξ) +π Φ µ(ξ) = Ψ(ξ) = Γ(ξ)Ψ(ξ), 2 2 µ(ξ) µ(ξ)

with 2π-periodic Γ ∈ L2 . This can be rewritten as ∑ f= cm ψ(· − m), m∈Z

where cm ∈ ℓ2 . Hence ψ(· − m) generates W0 . It remains to check that there are no other orthogonal wavelets than that given by (3.9). Since ψ ∈ W0 , it must satisfy ξ 1 Ψ(ξ) = √ e−i 2 H 2

(

) ( ) ξ ξ µ(ξ), +π Φ 2 2

with 2π-periodic µ ∈ L2 . Orthonormality of ψ(· − m) implies ) 2 ( ) 2 ∑ ∑ (ξ 1 1 ξ 2 2 = |Ψ(ξ + 2kπ)| = |µ(ξ)| H + kπ + π Φ + kπ 2π 2 2 2 k∈Z

1 = |µ(ξ)|2 2

k∈Z

( ( ) 2 ∑ ( ) 2 ( ) 2 ∑ ( ) 2 ) ξ ξ ξ ξ Φ Φ H + π + 2kπ + H + 2kπ + π 2 2 2 2 k∈Z

k∈Z

( ( ) 2 ( ) 2 ) 1 1 1 ξ ξ H + π + H = |µ(ξ)|2 . = |µ(ξ)|2 2 2π 2 2 2π Therefore |µ(ξ)|2 = 1, which was to be proved.

3.3

Vanishing moments and regularity

In this section we address regularity questions regarding orthogonal wavelet bases. Specially, we will show that one can not achieve an orthogonal wavelet ψ to belong in C ∞ . Lemma 3.6. Let r ∈ N0 , ψ ∈ C r such that ψ is not identically constant, let ψ (l) be bounded for l ≤ r and let C0 |ψ(x)| ≤ , (3.11) (1 + |x|)r+1+ε for some ε > 0, C0 ≥ 0. Let {ψj,k | j, k ∈ Z} constitute and orthonormal basis in L2 . Then ψ has r + 1 vanishing moments, i.e., ∫ xl ψ(x) dx = 0, ∀l = 0, 1, . . . , r.

Chapter 3. Construction of orthogonal wavelets

25

Proof. The proof by induction on r is based on [13]. 1. Let r = 0. Since ψ is continuous, not identically constant and since the dyadic rationals {2−j k | j, k ∈ Z} are dense in R, there exist j0 , k0 ∈ Z such that ψ(2−j0 k0 ) ̸= 0. Orthogonality implies ∫ ψ(x)ψ(2j x − k) dx = 0, for all j ̸= 0, k ̸= 0. We choose j > max(j0 , 0) and k = 2j−j0 k0 . This yields ∫ ψ(x)ψ(2j (x − 2−j0 k0 )) dx = 0. By change of variables (t = 2j (x − 2−j0 k0 )), ∫ ψ(2−j t + 2−j0 k0 )ψ(t) dt = 0. Applying the Dominated Convergence Theorem for j → +∞, ∫ ψ(2−j0 k0 ) ψ(t) dt = 0, but ψ(2−j0 k0 ) ̸= 0, implying

∫ ψ(t) dt = 0.

2. Let us now assume that for all l = 0, 1, . . . , r − 1, ∫ xl ψ(x) dx = 0. ∫ We want to show that xr ψ(x) dx = 0. Let us therefore auxiliary functions ν0 , ν1 , . . . νr , so that ∫ x ν0 = ψ and ∀n = 1, . . . , r νn (x) = νn−1 (t) dt. −∞

Since |ν0 (x)| = |ψ(x)| ≤ it follows that |ν1 (x)| ≤

∫

x

−∞

∫ |ψ(t)| dt ≤ C0

x

−∞

C0 , (1 + |x|)r+1+ε

1 C1 dt = . (1 + |t|)r+1+ε (1 + |x|)r+ε

Applying this logic r-times and using the definition of νn , we end up with |νn (x)| ≤

Cn , (1 + |x|)r−n+1+ε

where n = 0, 1, . . . , r. Integrating by parts, ∫ ∫ +∞ νr (x) dx = [xνr (x)]−∞ − xνr−1 (x) dx.

(3.12)

Chapter 3. Construction of orthogonal wavelets Since 0 ≤ |xνr (x)| ≤

26

Cr |x| → 0 , when x → ±∞, (1 + |x|)1+ε

the first term equals zero. Integrating by parts r-times gives ∫ ∫ ∫ νr (x) dx = − xνr−1 (x) dx = α1 xνr−1 (x) dx = ∫

∫ 2

= α2

x νr−2 (x) dx = . . . = αr

xr ψ(x) dx,

for some constants αn . Therefore ∫ ∫ r x ψ(x) dx = 0 ⇔ νr (x) dx = 0.

(3.13)

We know that {2−j k | j, k ∈ Z} is dense in R, ψ is not a polynomial (otherwise it would not be bounded) and ψ (r) is continuous. Hence there exist j0 , k0 ∈ Z such that ψ (r) (2−j0 k0 ) ̸= 0. Orthogonality implies ∫ ψ(x)ψ(2j x − k) dx = 0. Taking again j > max(j0 , 0) and k = 2j−j0 k0 , ∫ ψ(x)ψ(2j (x − 2−j0 k0 ) dx = 0. Now let us consider the following expression: ∫ ψ (r) (x)νr (2j (x − 2−j0 k0 )) dx Using (3.12) and the fact that ψ (n) is bounded for n ≤ r, one can derive, integrating by parts, ∫ ∫ (r) j −j ψ (x)νr (2 (x − 2 0 k0 )) dx = K ψ(x)ψ(2j (x − 2−j0 k0 )) dx = 0. By change of variables, ∫

ψ (r) (2−j x + 2−j0 k0 )νr (x) dx = 0.

Applying again the Dominated Convergence Theorem and using the fact ψ (r) (2−j0 k0 ) ̸= 0, it follows ∫ νr (x) dx = 0. By (3.13), this concludes the proof. 

Chapter 3. Construction of orthogonal wavelets

27

Corollary 3.7. Let ψ ∈ L2 ∩ C ∞ and let ψ has compact support. Then {ψj,k | j, k ∈ Z} cannot constitute an orthonormal system. Proof. Since ψ has compact support, there certainly exists C0 so that (3.11) is satisfied. Let us pretend that {ψj,k | j, k ∈ Z} constitute an orthonormal system. By Theorem 3.6, ψ is then orthogonal to any polynomial, i.e. ∫ p(x)ψ(x) dx = 0, for every polynomial p. Due to the Weierstrass Approximation Theorem1) , for each ε > 0 there exists a polynomial p, such that sup |f (x) − p(x)| < ε, x∈K

where K denotes the support of ψ. We have ∫ ∫ ∫ 2 ∥ψ∥L2 = ⟨ψ, ψ⟩ = ψ(x)ψ(x) dx = ψ(x)ψ(x) dx − p(x)ψ(x) dx = ∫ = K

∫ [ψ(x) − p(x)]ψ(x) dx ≤

K

∫ |ψ(x) − p(x)| |ψ(x)| dx ≤ ε

K

|ψ(x)| dx

We see that the norm ∥ψ∥2L2 can be made arbitrarily small, which leads to contradiction with orthonormality (∥ψ∥2L2 must equal 1).  We are thus limited when designing compactly supported wavelets in the sense of smoothness - one can only construct compactly supported wavelets that belong to C N , for some fixed N .

3.4

Construction of compactly supported wavelets

In this section we will finally examine a method for construction of wavelets that are orthogonal and compactly supported, with a nice property of having prescribed number of vanishing moments. First off, by (2.6) it is clear that if φ has compact support then hn is finite, and by (3.10), ψ has likewise compact support. Orthogonality is reflected on the filter H, defined by (2.7), as |H(ξ)|2 + |H(ξ + π)|2 = 2, where we dropped the “a.e.” because H is now a trigonometric polynomial. The following lemma shows how the number of vanishing moments is reflected on the filter H. 1)

Let f be a function with compact support. Then for each ε > 0 there exists a polynomial p such that supx∈K |f (x) − p(x)| < ε, where K is the support of f .

Chapter 3. Construction of orthogonal wavelets

28

Lemma 3.8. Let φ be an orthogonal scaling function with compact support, ψ an associated wavelet defined by (3.10) such that φ, ψ ∈ C r . Then H(ξ) defined by (2.6) can be expressed as ( ) 1 + exp(−iξ) r+1 H(ξ) = P (ξ), 2 with P ∈ C r and 2π-periodic. Proof. Since φ has compact support then by the discussion above ψ has also compact support. Moreover, the compact support ensures that ψ (l) is bounded for 0 ≤ l ≤ r and that C |ψ(x)| ≤ , (1 + |x|)r+1+ε for some ε > 0. According to lemma 3.6 it follows ∫ xl ψ(x) dx = 0, ∀l = 0, 1, . . . , r. Due to a standard property of the Fourier transform, ∫ ] dj [ l j (l) l−j j ξ Ψ (ξ) = (−i) (−1) x f (x) exp(−iξx) dx, dxj we have

Ψ(l) (ξ) ξ=0 = 0, for all 0 ≤ l ≤ r.

(3.14)

Since hn is finite, it follows that H(ξ) is 2π-periodic and continuous. By lemma 3.4, Φ(0) is nonzero. Since 1 Ψ(ξ) = √ exp(−iξ/2)H(ξ/2 + π)Φ(ξ/2) 2 and since both Φ, Ψ ∈ C r , it follows that H(ξ) ∈ C r . Applying (3.14) gives H (l) (ξ) ξ=π = 0, ∀0 ≤ l ≤ r. This means that H has a zero of order at least r + 1 at ξ = π, implying ( H(ξ) =

1 + exp(−iξ) 2

)r+1 P (ξ),

with 2π-periodic P ∈ C r .



The strategy is then to find a trigonometric polynomial H such that the following two properties are satisfied: |H(ξ)|2 + |H(ξ + π)|2 = 2, (3.15) and

( H(ξ) =

1 + exp(−iξ) 2

This is addressed by the following two theorems.

)r+1 P (ξ).

(3.16)

Chapter 3. Construction of orthogonal wavelets

29

Theorem 3.9 (Daubechies). Let H be a trigonometric polynomial satisfying H(ξ) =

√

( 2

1 + exp(−iξ) 2

)N P (ξ),

(3.17)

where P is a trigonometric polynomial. We have |H(ξ)|2 + |H(ξ + π)|2 = 2,

(3.18)

if and only if L(ξ) := |P (ξ)|2 is of the form (

ξ L(ξ) = q sin 2

)

2

,

with q(y) = qN (y) + y N R(y), where qN (y) =

N −1 ( ∑ k=0

) N −1+k k y k

and R is a polynomial, antisymmetric with respect to 12 , such that q(y) ≥ 0 for all y ∈ [0, 1]. Proof. Inserting (3.17) in (3.18) yields |H(ξ)|2 + |H(ξ + π)|2 = 2, if and only if (

ξ cos 2

)N

( )N 2 ξ L(ξ) + sin L(ξ + π) = 1, 2

2

(3.19)

because ( |H(ξ)|2 = H(ξ)H(ξ) = 2 ( =2 and

1 + exp(−iξ) 2

1 + cos ξ 2

[ |H(ξ + π)| = 2 cos 2

( 2

)N

ξ+π 2

)N (

1 + exp(iξ) 2

(

ξ L(ξ) = 2 cos 2

|P (ξ)|2 =

)N

2

)]N

)N

L(ξ)

(3.20)

( )N 2 ξ L(ξ + π) = 2 sin L(ξ + π). 2

Since we assume that hn are real, it follows ∑ ∑ H(ξ) = hn exp(−iξn), H(ξ) = hk exp(iξk) n finite

and |H(ξ)|2 =

k finite

∑

∑

hn hk exp[iξ(k − n)].

n finite k finite

|H(ξ)|2

Moreover, is an even trigonometric polynomial, which can be verified by a simple change of variables n ˜ := k, k˜ := n. Taking (3.20) in account, we examine that L(ξ) = 2 |P (ξ)| is an even trigonometric polynomial and hence it can be expressed as L(ξ) =

Chapter 3. Construction of orthogonal wavelets

30

q0 (cos ξ), for some polynomial q0 . Using the identity cos ξ = 1 − 2 sin2 2ξ , L(ξ) can be expressed as ( ) 2 ξ L(ξ) = q sin , (3.21) 2 where q is some polynomial. Substituting this in (3.19) after some manipulation yields )N ( ) ( )N ( ) ( 2 ξ 2 ξ 2 ξ 2 ξ q sin + sin q 1 − sin = 1. 1 − sin 2 2 2 2 It is intuitive to substitute y = sin2

ξ 2

∈ [0, 1]. The condition (3.18) is then equivalent to

(1 − y)N q(y) + y N q(1 − y) = 1, ∀y ∈ [0, 1], (3.22) ( ) with q such that L(ξ) = q sin2 2ξ = q(y). It remains to determine q. By the Bezout theorem2) , there exist unique polynomials q1 , q2 with degree less than or equal to N − 1, such that (1 − y)N q1 (y) + y N q2 (y) = 1. (3.23) After a change of variables (y 7→ 1 − y), this gives (1 − y)N q2 (1 − y) + y N q1 (1 − y) = 1. Since q1 , q2 are unique, this means that q2 (y) = q1 (1 − y). Hence q1 is a solution to (3.22). The equation (3.23) can be rewritten as q1 (y) = (1 − y)−N [1 − y N q1 (1 − y)]. Since (1 − y)−N can be expanded in Taylor polynomial plus a residue as (1 − y)−N =

N −1 ( ∑ k=0

) N +k−1 k y + O(y N ), k

where O(y N ) carries the terms of power N or higher, the polynomial q1 then becomes q1 (y) =

N −1 ( ∑ k=0

) N +k−1 k y + O(y N ). k

But since q1 has degree deg(q1 ) ≤ N − 1, it follows q1 (y) =

N −1 ( ∑ k=0

) N +k−1 k y . k

(3.24)

Note that according to (3.21), q must ne nonnegative for y ∈ [0, 1], which is clearly satisfied. The polynomial q1 defined by (3.24) is thus the unique lowest degree solution of (3.22). We 2)

If p1 , p2 are two polynomials of degree n1 , n2 respectively, with no common zeros then there exist unique polynomials q1 , q2 of degree n2 − 1, n1 − 1 respectively, such that p1 (x)q1 (x) + p2 (x)q2 (x) = 1.

Chapter 3. Construction of orthogonal wavelets

31

will denote qN := q1 . To give a complete characterization of the equation (3.22), we need to obtain all solutions. If q is a solution to (3.22), then (subtracting the two equations for q and qN ), (1 − y)N [q(y) − qN (y)] + y N [q(1 − y) − qN (1 − y)] = 0. (3.25) If R(y) is a polynomial antisymmetric with respect to 12 , i.e. R(1 − y) = −R(y), then q(y) = qN (y) + y N R(y) is clearly a solution to (3.25), assuming that R is chosen such that q(y) ≥ 0, for y ∈ [0, 1]. This concludes the proof.  Now we have |H(ξ)|2 , but we need H(ξ) itself. For this purpose we have the following theorem. Theorem 3.10 (Riesz, Fej´er). Let p be a positive trigonometric polynomial of the form p(ξ) =

M ∑

αn cos(nξ),

where an ∈ R.

n=0

There exists a trigonometric polynomial q with real coefficients and of the same order as p, such that |q(ξ)|2 = p(ξ). Proof. We are looking for a trigonometric polynomial q of the form q(ξ) =

M ∑

βn exp(iξn),

n=0

where βn ∈ R. Using the identity ( ) ( ) n n n 2 n−2 cos(nξ) = cos ξ − sin ξ cos ξ+ sin4 ξ cosn−4 ξ − . . . , 2 4 it is indeed possible to express the trigonometric polynomial p as an ”ordinary” polynomial of the variable cos(ξ), i.e. p(ξ) = p˜(cos ξ), where p˜ is a polynomial with real coefficients and of degree M . Let v1 , . . . , vM be the roots of p˜ (not necessarily all different). Then we can write p˜(v) = C

M ∏

(v − vj ).

j=1

Let P (z) = Cz

M

M ( ∏ z + z −1 j=1

2

where z ∈ C, z = e−iξ and pj (z) =

) − vj

=C

M ( ∏ 1 j=1

1 2

1 − vj z + z 2 2 2

) =C

M ∏

pj (z),

j=1

− vj z + 12 z 2 . Clearly, the degree of P is 2M and

p(ξ) = eiM ξ P (e−iξ ),

(3.26)

Chapter 3. Construction of orthogonal wavelets

32

because

z + z −1 e−iξ + eiξ = = cos ξ. 2 2 Let us examine the roots of P , by examining the roots of pj .

1a) For vj ∈ R, |vj | > 1, the roots of pj are both real and if rj is a root of pj , then rj−1 is likewise a root of pj . 1b) For vj = ±1, the same argument as above applies, only the roots rj , rj−1 are degenerate, i.e. they equal. √ 1c) For vj ∈ R, |vj | < 1, the roots of pj are of the form vj ± i 1 − vj2 , implying they are mutually conjugated and of absolute value 1. Hence they can be written as exp(iaj ), resp. exp(−iaj ). By (3.26), these roots are also the roots of p, and since p ≥ 0, they must have even multiplicity. 2) If vj ∈ C\R, then with each vj there is also vj in the group of roots of p˜j .√The roots of ( )( ) the polynomial 12 − vj z + 21 z 2 21 − vj z + 12 z 2 form a quadruplet vj ± vj2 − 1, vj ± √ vj 2 − 1, also expressible as zj , zj−1 , zj , z j−1 . From the above discussion we deduce that the roots of P can be regrouped as P (z) = C

J ∏

(z − zj )(z − zj−1 )(z − zj )(z − z j−1 )·

j=1

[ ·

K ∏

]2 (z − e

iak

k=1

−iak

)(z − e

)

L ∏

(z − rl )(z − rl−1 ).

l=1

One can easily verify that |e−iξ − z j−1 | = |zj |−1 |e−iξ − zj |, which then implies |(e−iξ − zj )(e−iξ − z j−1 )| = |zj |−1 |e−iξ − zj |2 , or, for z = e−iξ on the unit circle, |(z − zj )(z − z j−1 )| = |zj |−1 |z − zj |2 . Hence we can write 2 J ∏ 0 ≤ p(ξ) = |p(ξ)| = |P (e−iξ )| = C1 (e−iξ − zj )(e−iξ − zj ) · j=1

2 2 L K ∏ ∏ · (e−iξ − eiak )(e−iξ − e−iak ) (e−iξ − rl ) , k=1

l=1

Chapter 3. Construction of orthogonal wavelets

33

with C1 > 0. If we then define q(ξ) =

L J K ∏ ∏ √ ∏ C1 (e−iξ − eiak )(e−iξ − e−iak ) (e−iξ − rl ), (e−iξ − zj )(e−iξ − zj ) j=1

k=1

l=1

where the degree of q clearly equals M , we obtain p(ξ) = |q(ξ)|2 . Furthermore, since q(ξ) =

J K L ∏ ∏ √ ∏ (e−2iξ − 2e−iξ Rezj + |zj |2 ) (e−2iξ − 2e−iξ cos ak + 1) (e−iξ − rl ), C1 j=1

k=1

l=1



the coefficients of q are real.

The last theorem can actually be used to determine the polynomial q, assuming the roots of p are given. From the real roots which form a duplet rj , rj−1 , we retain either rj or rj−1 . From the complex roots which form a quadruplet zj , zj−1 , zj , z j−1 , we select the two roots either inside or outside the unit circle. In case there are also roots with absolute value 1 (i.e. roots of the form eiaj , resp. e−iaj ), we know that they are present in the multiplicity of two, which means that we retain only one pair out of two. We will return to this issue later. Another fact to note is that the polynomial q is not determined uniquely. For example, instead of q we could consider eiξm q(ξ) (for some m ∈ Z) to achieve the same result. It also depends on which roots of p we retain during this process, as described above. Now we are able to determine the trigonometric polynomial H satisfying (3.16) and (3.15). In the theorem 3.9 there is a degree of freedom in choosing the polynomial R. In the simplest case, we can put R = 0. The resulting scaling function, resp. wavelet (which will be constructed in the following section) are then called the Daubechies scaling function, resp. the Daubechies wavelet, often denoted DN , where N specifies the number of vanishing wavelet moments. The associated polynomial q in the theorem 3.9 will then equal ) N −1 ( ∑ N −1+k k y , q(y) = k k=0

which is clearly a polynomial of degree N − 1, i.e. the lowest possible degree among all possible choices of R. We call this polynomial q the Daubechies polynomial. The degree of L(ξ) = |P (ξ)|2 is then equal N −1 and, after applying the Riesz-Fej´er theorem, the order ( −iξ )N of P is likewise equal N − 1. Multiplying this polynomial P with 1+e2 results in the polynomial H which we are interested in and the order of H equals 2N − 1, so we need to store 2N coefficients hn . The trigonometric polynomial H can be computed numerically, but let us first examine how to actually obtain φ and ψ from H.

3.5

Determining φ and ψ from hn

The above discussion suggests to start with H which satisfies (3.16) and (3.15). The theorems 3.9 and 3.10 give complete characterizations of such H. Now we need to construct

Chapter 3. Construction of orthogonal wavelets

34

the scaling function and the corresponding wavelet from this H. Let us first deduce a possible candidate for the scaling function φ. Iterating the scaling equation (2.7) gives 1 Φ(ξ) = √ H 2

( ) ( ) ( ) ( ) ( ) +∞ ∏ H(2−j ξ) ξ ξ ξ ξ 1 1 ξ √ H √ Φ =√ H Φ = Φ(0) , 2 2 2 4 4 2 2 2 j=1

if Φ(0) is well defined and the infinite product converges. Since we expect at least φ ∈ L1 and H to be a trigonometric polynomial (for compact support), lemma 3.4 ensures |Φ(0)| = √12π . For convenience, we can assume Φ(0) = √12π ,

because it can at most produce a change of phase of φ. Note also that |Φ(0)| = √ H(0) = 2. We have √ H(ξ) √ ≤ 1 + √1 |H(ξ) − 2| = 1 + √1 |H(ξ) − H(0)| = 2 2 2

√1 2π

implies

∑ 1 ∑ 1 ∑ =1+ √ | hn e−iξn − hn | ≤ 1 + √ | |hn ||e−iξn − 1| = 2 n 2 n n √ ∑ sin ξn ξn 1 ∑ 2 √ |hn | sin = 1 + |ξnhn | ξn ≤ 1 + C|ξ| ≤ eC|ξ| . =1+ 2 2 2 n n 2 We have used the identity |e−iξn − 1| = 2 sin ξn 2 and the fact that hn has enough decay ∑ (namely n |n||hn | < +∞), which holds if hn is finite. It then follows   +∞ ∏ ∑ H(2−j ξ) +∞ √ ≤ exp(C2−j |ξ|) = exp  C2−j |ξ| = eC|ξ| . 2

+∞ ∏ j=1

j=1

j=1

∏ H(2−j ξ) √ converges absolutely pointwise almost everywhere. Hence +∞ j=1 2 The only candidate (up to the factor −1) for the scaling function is thus φ defined in terms of the Fourier transform as +∞ 1 ∏ H(2−j ξ) √ √ Φ(ξ) := . 2π j=1 2

(3.27)

It is rather technical to prove that such φ defines a function in L2 with compact support and that, under modest requirements on the corresponding filter H, the family {φ(· − k) | k ∈ Z} is an orthonormal system. We conclude this section only by mentioning the appropriate assertions. The proofs can be found in [16, 7]. Theorem 3.11 (Mallat [16]). If H is a 2π-periodic function so that |H(ξ)|2 +|H(ξ +π)|2 = ∏ ∏ H(2−j ξ) H(2−j ξ) √ √ 2 and if +∞ converges pointwise almost everywhere, then Φ(ξ) = √12π +∞ ∈ j=1 j=1 2 2 L2 and ∥φ∥L2 ≤ 1.

Chapter 3. Construction of orthogonal wavelets

35

Theorem 3.12 (Cohen, Lawton [7]). If φ and ψ are given in terms of their Fourier transform by +∞ 1 ∏ H(2−j ξ) √ Φ(ξ) = √ , 2π j=1 2 ) ( ) ( ξ ξ 1 −i ξ +π Φ , Ψ(ξ) = √ e 2 H 2 2 2 √ where H satisfies |H(ξ)|2 + |H(ξ + π)|2 and H(0) = 2, then the ψj,k = 2j/2 ψ(2j · −k) constitute an orthonormal basis of L2 if and only if there exists a compact set K, congruent to [−π, π] modulo 2π 3) , containing a neighborhood of 0, so that inf inf |H(2−k ξ)| > 0,

k>0 ξ∈K

or, equivalently, if the eigenvalue 1 of the [2(N2 − N1 ) − 1] × [2(N2 − N1 ) − 1]-dimensional matrix A defined by N2 ∑ Al,k = hn hk−2l+n , n=N1

where −(N2 − N1 ) + 1 ≤ l, k ≤ (N2 − N1 ) + 1, is nondegenerate.

3.6

Numerical computation of the scaling filter hn

In this section we will examine the numerical algorithm which establishes the filter H (in terms of its coefficients hn ). To illustrate what is going on in a general case of N vanishing wavelet moments, let us first compute the coefficients of H in case N = 1 and N = 2 [15]. Example. 1. Let N = 1. This is very simple, because |P (ξ)|2 = 1. In the Riesz-Fej´er theorem we choose P (ξ) = 1 and H thus equals H(ξ) = implying h0 = h1 =

√1 . 2

√

2·

1 + e−iξ , 2

Daubechies D1 wavelet is also called the Haar wavelet.

2. Now let N = 2. By Daubechies theorem 3.9, L(y) = |P (y)|2 = 1 + 2y, where y = sin2 2ξ . Since y = sin2

) 1 ξ 1 ( iξ = −e + 2 − e−iξ = (−z −1 + 2 − z), 2 4 4

where z = e−iξ , we can also write

( ) ) 1 1 ( −1 1 2 −1 − + 2z − z . −z + 2 − z = z L(y) = 1 + 2 2 2

3)

This means that the size of K is 2π and that for all ξ in [−π, π], there exists l ∈ Z so that ξ + 2πl ∈ K.

Chapter 3. Construction of orthogonal wavelets

36

√ √ The polynomial − 12 + 2z − 12 z 2 has two real roots 2 + 3 and 2 − 3. We choose one √ of this duplet, e.g. 2 − 3 and obtain the “square root” of L: √ P (z) = C1 (z − 2 + 3). It follows

H(ξ) = C2 · (1 + e−iξ )2 (e−iξ − 2 +

√

3).

The constant C2 can be determined using the normalization condition H(0) = √ 3+1 which gives C2 = 4√2 . We can then write H(ξ) =

3 ∑

√

2,

hn e−iξn ,

n=0

where

√ √ √ √ 1+ 3 3+ 3 3− 3 1− 3 h0 = √ , h 1 = √ , h 2 = √ , h 3 = √ . 4 2 4 2 4 2 4 2

△ The above example suggests that the same principle can be applied to a general case of N vanishing wavelet moments, assuming that we have a routine to compute the roots of a polynomial. We can therefore describe the algorithm for computing the scaling filter coefficients hn in several steps: 1. Choose the number of vanishing moments N ≥ 1. 2. In this and subsequent steps, the computations will be expressed in z = e−iξ rather than in ξ itself. Prepare the polynomial g(z) = (1 + z)N . In Matlab this can be achieved easily using the conv command. 3. Compute the Daubechies polynomial L, where instead of y = sin2 ξ, use the substitution y = 14 (−z −1 + 2 − z), as in the previous example. 4. The polynomial L in the previous step is the square of the modulus of P . To extract the “square root” from L use the Riesz-Fej´er theorem 3.10. 5. Multiply the polynomials g and P , using the conv command to obtain H (up to a constant). √ ∑ 6. Finally normalize H so that H(0) = n hn = 2. The extraction of the “square root” (also called spectral factorization) here is done in such way, that among all the choices of roots to retain, only roots inside the unit circle are selected. This results in a minimal phase factorization. One could also select all the roots outside the unit circle to achieve maximal phase factorization, or just any root from a duplet (resp. two roots from a quadruplet – either inside or outside the unit circle). Another important fact to note is that the spectral factorization can be designed more effectively, not by the proof of Riesz-Fej´er theorem. The approach that we presented here is the simplest one, but for a large number of roots (which is affected by a higher values of vanishing wavelet moments), say, N > 25, this method should be replaced by a more numerically stable procedure.

Chapter 3. Construction of orthogonal wavelets

3.7

37

The Discrete Wavelet Transform - Mallat algorithm

In this section we will examine a fully discrete algorithm as proposed by Mallat [16]. Although the derivation is very simple, the result is central in our development. The multiresolution framework enables us to represent any f ∈ L2 as f=

∑

cjk0 φj0 ,k +

+∞ ∑ ∑

djk ψj,k ,

j=j0 k∈Z

k∈Z

which follows directly from (2.2). The index j0 represents the coarsest level corresponding to the subspace Vj0 . Orthonormality ensures that the coefficients cjk and djk can be computed in terms of inner products as ∫ j ck = ⟨f, φj,k ⟩ = f (x)φj,k (x) dx, ∫ djk

= ⟨f, ψj,k ⟩ =

f (x)ψj,k (x) dx.

The aim is to find these coefficients without the explicit integration. Since we work in the classical setting, it is indeed possible to write ∫ j ck = f (x)2j/2 φ(2j x − k) dx, ∫ djk

f (x)2j/2 ψ(2j x − k) dx.

=

The two-scale equations read ∑

φ(2j x − k) =

√ hm−2k 2φ(2j+1 x − m),

(3.28)

√ gm−2k 2φ(2j+1 x − m).

(3.29)

m∈Z

ψ(2j x − k) =

∑ m∈Z

Together, cjk

=

∑

∫ hm−2k

m∈Z

djk =

∑

f (x)2

j+1 2

φ(2j+1 x − m) dx,

f (x)2

j+1 2

φ(2j+1 x − m) dx.

∫ gm−2k

m∈Z

Since the above integrals equal ⟨f, φj+1,m ⟩, we finally obtain cjk =

∑

hm−2k cj+1 m

(3.30)

gm−2k cj+1 m .

(3.31)

m∈Z

djk =

∑ m∈Z

Chapter 3. Construction of orthogonal wavelets

38

The process of computing cjk , djk from cj+1 represents one step of the forward discrete k wavelet transform (analysis) and, naturally, it can be iterated so that we set cj+1 to be k samples of the function f , assuming the index j is large enough. Note also that the formulas (3.30) and (3.31) can be viewed as convolutions followed by downsampling (retaining every ∗ h−k and dj2k = cj+1 ∗ g−k respectivelly. This process second sample), that is, cj2k = cj+1 k k is often depicted graphically as in Figure 3.1. Figure 3.1: The forward discrete wavelet transform

Let us now examine the inverse process, i.e. the computation of cj+1 from cjk and djk . k For this reason, consider f ∈ Vj+1 , that is f (x) =

∑

cj+1 k 2

j+1 2

φ(2j+1 x − k),

(3.32)

k∈Z

or, by means of wavelets, f (x) =

∑

cjk 2j/2 φ(2j x − k) +

k∈Z

∑

djk 2j/2 ψ(2j x − k).

(3.33)

k∈Z

Inserting (3.28) and (3.29) in (3.33) yields ∑ j ∑ j+1 hm−2k 2 2 φ(2j+1 x − m) f (x) = ck k∈Z

+

∑ k∈Z

djk

m∈Z

∑

gm−2k 2

j+1 2

φ(2j+1 x − m).

m∈Z

Combining (3.32) with (3.33), multiplying both sides by φ(2j+1 x − l), and integrating with respect to x gives ∑ j ∑ j cj+1 = c h + dk gl−2k . l−2k l k k∈Z

k∈Z

It is now only a matter of taste to change the indexes so that ∑ ∑ j cj+1 = c h + djm gk−2m . k−2m m k m∈Z

m∈Z

(3.34)

Chapter 3. Construction of orthogonal wavelets

39

We have thus derived one step of the inverse discrete wavelet transform (synthesis) which can be iterated as the forward transform did. The process can be expressed as first upsampling (inserting zeros between neighboring samples) of cjk and djk followed by convolution with hn and gn , respectivelly. See Figure 3.2. It comes to question whether the synthesis of coefficients obtained by analysis equals to the original values. This property is called exact (or perfect) reconstruction and is discussed in more depth in the following chapter. Figure 3.2: The inverse discrete wavelet transform

3.8

Plots of several scaling functions and wavelets

In figures 3.3 - 3.7 the graphs of scaling functions and wavelets are depicted. They are obtained by the cascade algorithm.

Chapter 3. Construction of orthogonal wavelets

40

Figure 3.3: Daubechies scaling function and wavelet, N = 2

1.5

2 1.5

1

1 0.5

0.5 0 −0.5

0

−1 −0.5

0

1

2

3

−1.5 −1

0

1

2

Figure 3.4: Daubechies scaling function and wavelet, N = 4

1.2

1.5

1 1 0.8 0.6

0.5

0.4 0

0.2 0

−0.5 −0.2 −0.4

0

2

4

6

−1

−2

0

2

4

Chapter 3. Construction of orthogonal wavelets

41

Figure 3.5: Daubechies scaling function and wavelet, N = 6

1.5

1.5 1

1 0.5 0.5

0 −0.5

0 −1 −0.5

0

5

−1.5 −5

10

0

5

Figure 3.6: Daubechies scaling function and wavelet, N = 8

1.5

1

0.5 1 0 0.5 −0.5 0 −1

−0.5

0

5

10

15

−1.5

−5

0

5

Chapter 3. Construction of orthogonal wavelets

42

Figure 3.7: Daubechies scaling function and wavelet, N = 9

1

1

0.5 0.5 0

−0.5 0 −1

−0.5

0

5

10

15

−1.5

−5

0

5

Chapter 4. Construction of biorthogonal wavelets

43

Chapter 4

Construction of biorthogonal wavelets The orthogonal setting may seem a bit restrictive due to the fact that there are no orthogonal wavelets with either symmetry or antisymmetry axis, other than the simple Haar case. It turns out that, if the orthogonality requirement is dropped, it is possible to construct symmetric biorthogonal wavelets with corresponding exact reconstruction filters. In this setting we have two dual multiresolution ladders (Vj ) and (V˜j ) so that Vj = clos span{φj,k | k ∈ N} and V˜j = clos span{φ˜j,k | k ∈ N}. We also define the complement ˜ j , which are spanned by wavelets ψj,k , as Vj+1 = Vj ⊕ Wj and V˜j+1 = spaces Wj and W ˜ ˜ ˜ j = {0}. We also require Vj ⊕ Wj , where the sum is direct, i.e. Vj ∩ Wj = {0} and V˜j ∩ W ˜ ˜ that Vj ⊥ Wj and Vj ⊥ Wj . The multiresolution framework ensures that the scaling functions and wavelets satisfy the two-scale relations ( ) ( ) ( ) ( ) 1 ξ ξ 1 ˜ ξ ˜ ξ ˜ Φ(ξ) = √ H Φ , Φ(ξ) = √ H Φ , 2 2 2 2 2 2 ( ) ( ) ( ) ( ) 1 ξ ξ 1 ˜ ξ ˜ ξ ˜ √ √ G G Ψ(ξ) = Φ , Ψ(ξ) = Φ , 2 2 2 2 2 2 whilst biorthogonality implies ⟨φ, ˜ ψ(· − k)⟩ = 0, ⟨φ, ˜ φ(· − k)⟩ = δk,0 ,

⟨ ⟩ ˜ φ(· − k) = 0, ψ, ⟨ ⟩ ˜ ψ(· − k) = δk,0 . ψ,

The property ⟨φ, ˜ ψ(· − k)⟩ = 0 can be rewritten as ∫ 0=

iξk ˜ dξ = Φ(ξ)Ψ(ξ)e

∑∫ l∈Z

∫

2π

eiξk

= 0

∑ l∈Z

2π(l+1)

iξk ˜ Φ(ξ)Ψ(ξ)e dξ

2πl

˜ + 2πl)Ψ(ξ + 2πl) dξ. Φ(ξ

Chapter 4. Construction of biorthogonal wavelets Hence

∑

44

˜ + 2πl)Ψ(ξ + 2πl) = 0, a.e. Φ(ξ

l∈Z

In the same manner we derive ∑

˜ + 2πl)Φ(ξ + 2πl) = 0, a.e., Ψ(ξ

l∈Z

∑

˜ + 2πl)Φ(ξ + 2πl) = 1, a.e., Φ(ξ

l∈Z

∑

˜ + 2πl)Ψ(ξ + 2πl) = 1, a.e. Ψ(ξ

l∈Z

It is an easy exercise to substitute from the two-scale relations to these formulas to obtain ˜ ˜ + π)G(ξ + π) = 0, H(ξ)G(ξ) + H(ξ

(4.1)

˜ ˜ + π)H(ξ + π) = 0, G(ξ)H(ξ) + G(ξ

(4.2)

˜ ˜ + π)H(ξ + π) = 2, H(ξ)H(ξ) + H(ξ

(4.3)

˜ ˜ + π)G(ξ + π) = 2, G(ξ)G(ξ) + G(ξ

(4.4)

These equations can be expressed more compactly as ˜ (ξ)M T (ξ) = I, M where 1 M (ξ) = √ 2 and ˜ (ξ) = √1 M 2

(

(

H(ξ) H(ξ + π) G(ξ) G(ξ + π) ˜ ˜ + π) H(ξ) H(ξ ˜ ˜ + π) G(ξ) G(ξ

)

)

are called modulation matrices and I denotes the identity matrix. In following we will show that the formulas above fit neatly in the exact reconstruction filter scheme.

4.1

Exact reconstruction filters

In the same manner as in Section 3.7 we obtain a biorthogonal version of the Mallat algorithm, that is ∑ cjk = hm−2k cj+1 (4.5) m , m∈Z

djk

=

∑

gm−2k cj+1 m ,

(4.6)

m∈Z

for the analysis, and cj+1 = k

∑ m∈Z

˜ k−2m + cjm h

∑ m∈Z

djm g˜k−2m ,

(4.7)

Chapter 4. Construction of biorthogonal wavelets

45

for the synthesis. Now suppose that using the above algorithm we compute cjk and djk from cj+1 and cˆj+1 l l ˜ n , g˜n are exact reconstrucion filters if from these cjk and djk . We say that the hn , gn , h cj+1 = cˆj+1 , for all possible l. l l It is convenient to express the relations (4.5), (4.6) and (4.7) in terms of the z-transform (which arises from Fourier transform by substituting z := eiξ ). We ∑ the discrete-time ∑ −n −n and similarly for H(z), ˜ ˜ denote H(z) = , H(z) = G(z), G(z). n∈Z hn z n∈Z h−n z Let xn denotes the input sequence in the Mallat algorithm and X(z) the corresponding z-transform. In the analysis part we convolve X(z) with H(z) following by downsampling to produce Y0 (z) and similarly, we convolve X(z) with G(z) and downsample to produce Y1 (z). The downsampling can be expressed in terms of z-transform as Xdown (z) =

∑ n∈Z

x2n z −n =

) 1( ) 1 1 1 1 1 ∑( xn (z 2 )−k + xn (−z 2 )−k = X(z 2 ) + X(−z 2 ) . 2 2 n∈Z

(4.8) Together, for the analysis part, we have Y0 (z) =

) 1 1 1 1 1( H(z 2 )X(z 2 ) + H(−z 2 )X(−z 2 ) , 2

(4.9)

Y1 (z) =

) 1 1 1 1 1( G(z 2 )X(z 2 ) + G(−z 2 )X(−z 2 ) . 2

(4.10)

and

˜ upsampling The synthesis part consists of upsampling Y0 (z) and convolving with H, ˜ Y1 (z) and convolving with G, and finally adding these two intermediate results. The upsampling operation can be expressed in terms of z-transform as ∑ Yup (z) = yn z −2n = Y (z 2 ). n∈Z

Hence, for the synthesis part, we have 2 2 ˆ ˜ ˜ X(z) = H(z)Y 0 (z ) + G(z)Y1 (z ).

(4.11)

Inserting (4.9) and (4.10) in (4.11) yields ) ) 1(˜ 1(˜ ˆ ˜ ˜ X(z) = X(z) + X(−z). H(z)H(z) + G(z)G(z) H(z)H(−z) + G(z)G(−z) 2 2 ˆ The condition X(z) = X(z) for exact reconstruction now reads ˜ ˜ H(z)H(z) + G(z)G(z) = 2, ˜ ˜ + G(z)G(−z) = 0, H(z)H(−z) or, again in terms of the discrete-time Fourier transform, ˜ ˜ H(ξ)H(ξ) + G(ξ)G(ξ) = 2,

(4.12)

Chapter 4. Construction of biorthogonal wavelets ˜ ˜ H(ξ)H(ξ + π) + G(ξ)G(ξ + π) = 0.

46 (4.13)

It comes to question whether there are any explicit formulas to obtain some filters from the others so that the exact reconstruction properties are satisfied. It turns out that, given ˜ the pair H(ξ), H(ξ), it suffices to define ˜ + π), G(ξ) = e−iξ H(ξ

(4.14)

˜ G(ξ) = e−iξ H(ξ + π).

(4.15)

The property (4.13) is then automatically satisfied. The property (4.12) can be rewritten, using (4.14), (4.15), as H(ξ)H(ξ) + H(ξ + π)H(ξ + π) = 2, which is, by a coincidence, exactly the equation (4.3) we encountered before. Also the other equations (4.1), (4.2), and (4.4) can be derived using (4.14), (4.15) and (4.12), (4.13). The exact reconstruction filters are therefore assigned to biorthogonal wavelets, which is a powerful result. In following we will focus on construction of such wavelets. We will examine the classical construction as proposed in [5] and an interesting alternative based on so called lifting which was developed in [19, 20].

4.2

Construction of symmetric biorthogonal wavelets with compact support

In following we will examine how to design perfect reconstruction filters corresponding to compactly supported and symmetric biorthogonal wavelets as proposed by Cohen, Daubechies, Feauveau in [5]. We start by examining the property of vanishing moments. In the same manner as we have done in lemma 3.6 we can derive Lemma 4.1. Let r ∈ N0 , and ψ, ψ˜ be not identically constant and biorthogonal in the sense that ⟨ ⟩ ψj,k , ψ˜j ′ ,k′ = δj,j ′ δk,k′ , where ˜ j · −k). ψj,k = 2j/2 ψ(2j · −k), ψ˜j,k = 2j/2 ψ(2 Assume that ˜ |ψ(x)| ≤

C0 , (1 + |x|)r+1+ε

for some ε > 0, C0 ≥ 0, and ψ ∈ C r . Then ψ˜ has r + 1 vanishing moments, i.e., ∫ ˜ dx = 0, ∀l = 0, 1, . . . , r. xl ψ(x)

Chapter 4. Construction of biorthogonal wavelets

47

Again, this has two important consequences. First off, one can not achieve biorthogonal wavelets to possess infinite smoothness. The second thing is that if ψ ∈ C r and ψ˜ ∈ C r˜, ˜ are divisible by (1 + e−iξ )r+1 and (1 + e−iξ )r˜+1 , respectively, where H then H and H ˜ and H are determined by the two-scale relations. Note that since we require the scaling ˜ are automatically trigonometric functions and wavelets to have compact support, H and H polynomials. We only consider filters with real coefficients, although complex examples are possible. We want our scaling functions to have either symmetry or antisymmetry axis. For filters with odd number of taps we require φ(x) = φ(−x), or H(ξ) = H(−ξ), and for filters with even number of taps we suppose φ to be symmetric around 21 , hence the requirement is φ(1 − x) = φ(x), or H(ξ) = e−iξ H(−ξ). In the first case have H(ξ) = p0 (cos ξ), where p is a polynomial, possibly generalized ∑ we n 2 so that p0 (x) = N n=N1 an x . Of course, the exact reconstruction property ˜ ˜ + π) = 2, H(ξ)H(ξ) + H(ξ + π)H(ξ

(4.16)

must be satisfied. By a simple substitution we deduce that, for a]fixed H with H(ξ) = [ 1 ˜ # ˜ ˜ ˜ is likewise a solution, H(−ξ), if H is a solution to (4.16) then H (ξ) = 2 H(ξ) + H(−ξ) # # ˜ of the same form as H, with H (ξ) = H (−ξ). Therefore it suffices to consider H ˜ i.e. H(ξ) = p˜0 (cos ξ). By the analysis above, we also expect p0 (cos ξ) to be divisible by (1 + e−iξ )l , with l ≥ 1. Since p0 is a polynomial in cos ξ and (1 + e−iξ )2 = 4e−iξ cos2 2ξ = 2e−iξ (1 + cos ξ), it follows that H must be expressible as ( ) ξ 2l H(ξ) = cos q0 (cos ξ). 2

(4.17)

iξ/2 In the second case we observe that the function f (ξ) ( )= e H(ξ) is 4π periodic and ξ symmetric around 0. Hence f is a polynomial in cos 2 and e−iξ/2 f (ξ) is reduced to a ( ) trigonometric polynomial in ξ. This means that cos 2ξ is present in H(ξ) only in odd power. All in all we have ( ) ξ −iξ/2 H(ξ) = e cos p0 (cos ξ), 2

∑ 2 n where p0 is a polynomial, possibly generalized so that p0 (x) = N n=N1 an x . As in the ˜ to be of the same form as H and it follows that previous case, we expect H −iξ/2

H(ξ) = e

( ) ξ 2l+1 cos q0 (cos ξ). 2

Substituting (4.17) and (4.18) in (4.16), we derive ( ) ( ) ˜ ˜ ξ 2(l+l) ξ 2(l+l) cos q0 (cos ξ)˜ q0 (cos ξ) + sin q0 (− cos ξ)˜ q0 (− cos ξ) = 2, 2 2

(4.18)

Chapter 4. Construction of biorthogonal wavelets

48

for the first case and ( ) ( ) ˜ ˜ ξ 2(l+l+1) ξ 2(l+l+1) cos q0 (cos ξ)˜ q0 (cos ξ) + sin q0 (− cos ξ)˜ q0 (− cos ξ) = 2, 2 2 for the second case. This can be written in more compact form, (1 − x)m P (x) + xm P (1 − x) = 1, where m = l + ˜l in the first case and m = l + ˜l + 1 in the second case, and ( ) 1 2 ξ P sin = q0 (cos ξ)˜ q0 (cos ξ), 2 2 with x = sin2 2ξ . This equation was already solved in the proof of theorem 3.9. The solution is m−1 ∑ (m − 1 + k ) P (x) = xk + xm R(x), k k=0

where R is a polynomial, antisymmetric with respect to 21 . We can now present examples of biorthogonal symmetric scaling functions and wavelets with compact cupport. This amounts to choosing l, ˜l, the polynomial R and finally the ˜ = 2˜l and N = 2l factorization of P . One can choose q˜0 ≡ 1, R ≡ 0 and arbitrary l, ˜l. For N we have )˜ ( √ ξ N ˜ H(ξ) = 2 cos 2 and H(ξ) =

√

)( ( ) l+˜l−1 ( ) ξ k ξ N ∑ l + ˜l − 1 + k sin 2 cos . k 2 2 k=0

˜ = 2˜l + 1 and N = 2l + 1, we have In the second case, i.e. N ˜ H(ξ) = and H(ξ) = √

√

2e

−iξ/2

(

ξ cos 2

)N˜

( ) l+˜l ( ) )( √ −iξ/2 ξ N ∑ l + ˜l + k ξ k cos 2e sin . 2 2 k k=0

√ ˜ Note that the 2 is a normalization factor (we must have H(0) = H(0) = 2). The resulting scaling functions and wavelets are depicted on figures 4.1 - 4.4. They are obtained by a procedure similar to that of orthogonal case - for convenience the computations are carried in the z-transform form. A drawback of the above examples is the fact that the lengths of the resulting filters ˜ n tend to be very dissimilar. To obtain filters with closer lengths we need a better hn , h factorization of the polynomial P into q0 , q˜0 . As in the case of orthogonal compactly

Chapter 4. Construction of biorthogonal wavelets

49

supported wavelets one may write the polynomial P as a product of first and second order polynomials (assuming we know all roots of P ): P (x) = C

∏

(x − xj )

∏ (x2 − 2Rezj x + |zj |2 ).

j

j

Then q0 , q˜0 are obtained as q0 =

∏ (x − xj ), j

q˜0 =

∏

(x2 − 2Rezj x + |zj |2 ).

j

˜ = 4 are very popular in image processing community The resulting wavelets for N = N and they are perhaps better known as CDF 9/7 wavelets, according to the filter lengths. The corresponding scaling functions and wavelets are depicted on figure 4.5. We have not discussed convergence questions nor the conditions for ψ, ψ˜ to constitute dual Riesz bases for it is much more complicated than in the case of orthogonal bases. In following we only state the main assertions. Theorem 4.2 (Cohen, Daubechies, Feauveau [5]). Suppose that φ, φ, ˜ as defined by +∞ 1 ∏ H(2−j ξ) √ , Φ(ξ) = √ 2π j=1 2

+∞ ˜ −j ξ) 1 ∏ H(2 ˜ √ Φ(ξ) =√ , 2π j=1 2

satisfy |Φ(ξ)| ≤ C(1 + |ξ|)−1/2−ε , ˜ |Φ(ξ)| ≤ C(1 + |ξ|)−1/2−ε . Define ψj,k = 2j/2 ψ(2j · −k), ˜ j · −k), ψ˜j,k = 2j/2 ψ(2 with ψ, ψ˜ ∈ L2 defined as Ψ(ξ) = e

iξ/2

(

) ( ) ξ ξ , +π Φ 2 2

(

) ( ) ξ ˜ ξ . +π Φ 2 2

˜ H

˜ Ψ(ξ) = eiξ/2 H Then, for all f ∈ L2 , f=

⟩ ∑ ⟨ ∑ f, ψ˜j,k ψj,k = ⟨f, ψj,k ⟩ ψ˜j,k , j,k∈Z

where the sequences converge strongly.

j,k∈Z

Chapter 4. Construction of biorthogonal wavelets

50

˜ n be finite real sequences satTheorem 4.3 (Cohen, Daubechies, Feauveau [5]). Let hn , h √ ˜ ˜ + π) = 2 and H(0) = H(0) ˜ isfying H(ξ)H(ξ) + H(ξ + π)H(ξ = 2, where H(ξ) =

∑

˜ hn e−iξn , H(ξ) =

n∈Z

∑

˜ n e−iξn . h

n∈Z

Define φ, φ˜ as +∞ 1 ∏ H(2−j ξ) √ Φ(ξ) = √ , 2π j=1 2

+∞ ˜ −j 1 ∏ H(2 ξ) ˜ √ Φ(ξ) =√ . 2π j=1 2

Then the following three statements are equivalent: ∫ 1. φ, φ˜ ∈ L2 and φ(x − k)φ(x ˜ − l) dx = δk,l . 2. There exist strictly positive trigonometric polynomials f0 , f˜0 and a compact set K congruent to [−π, π] modulo 2π so that i) P0 f0 = f0 and P˜0 f˜0 = f˜0 , where P0 and P˜0 are operators acting on 2π-periodic functions f as ( ) 2 ( ) ( ) 2 ( ) ξ ξ H ξ + π f ξ + π , (P0 f )(ξ) = H f + 2 2 2 2 ( ) 2 ( ) ( ) 2 ( ) ˜ ξ f ξ + H ˜ ξ +π f ξ +π , (P˜0 f )(ξ) = H 2 2 2 2 ii) the interior of K contains 0, iii) for all ξ ∈ K, all k ∈ N \ {0} and some strictly positive C (independent of ξ and k), ˜ −k ξ)| ≥ C. |H(2−k ξ)|, |H(2 3. There exist strictly positive trigonometric polynomials f0 , f˜0 so that P0 f0 = f0 , P˜0 f˜0 = f˜0 and these are the only trigonometric polynomials (up to normalization) invariant under P0 , P˜0 respectively.

Chapter 4. Construction of biorthogonal wavelets

51

˜ = 2, N = 2 Figure 4.1: Cohen-Daubechies-Feauveau scaling functions and wavelets, N

1.5

1.5 1

1

0.5 0

0.5

−0.5 0 −1

−0.5

0

0.5

1

10

−1 −1

0

1

2

0

1

2

10 5

5

0 0

−5 −2

−5

−1

0

1

2

−10 −1

Chapter 4. Construction of biorthogonal wavelets

52

˜ = 2, N = 4 Figure 4.2: Cohen-Daubechies-Feauveau scaling functions and wavelets, N

1.5

1.5 1

1

0.5 0

0.5

−0.5 0 −1

−0.5

0

0.5

1

3

−1 −2

−1

0

1

2

3

−1

0

1

2

3

2 1

2

0 1 −1 0 −1 −4

−2 −2

0

2

4

−3 −2

Chapter 4. Construction of biorthogonal wavelets

53

˜ = 1, N = 3 Figure 4.3: Cohen-Daubechies-Feauveau scaling functions and wavelets, N

1.5

1 0.5

1

0 −0.5

0.5

−1 0

0

0.5

1

−1.5 −1

1.5

2

1

1

0.5

0

0

−1

−0.5 −2

−1

0

1

2

3

−2 −1

0

1

2

0

1

2

Chapter 4. Construction of biorthogonal wavelets

54

˜ = 3, N = 5 Figure 4.4: Cohen-Daubechies-Feauveau scaling functions and wavelets, N

0.8

1

0.6

0.5

0.4

0

0.2

−0.5

0 −1

0

1

2

2

−1

−2

0

2

4

−2

0

2

4

4 2

1

0 0

−1 −5

−2

0

5

−4

Chapter 4. Construction of biorthogonal wavelets

55

Figure 4.5: Cohen-Daubechies-Feauveau scaling functions and wavelets with closer filter ˜ =N =4 lengths, N

1.5

2

1

1

0.5 0

0 −0.5

0

−1

2

1.5

2

1

1

0.5

0

0

−1

−0.5 −4

4.3

−2

−2

0

2

4

−2

−2

0

2

4

−2

0

2

4

Alternative construction - the lifting scheme

In this section we will discuss an alternative method of biorthogonal wavelet construction by means of so-called lifting scheme, developed by W. Sweldens. We will consider the general framework developed in Chapter 2 because this method does not rely on Fourier transform. For convenience we will recapitulate the general two-scale relations: ∑ φj,k = hj,k,l φj+1,l , (4.19) l∈Kj+1

ψj,m =

∑

gj,m,l φj+1,l .

(4.20)

l∈Kj+1

The set hj,k,l in the above relation is called a finite filter if for each j and k only finite number of coefficients are non-zero and the same for j and l. We will consider only

Chapter 4. Construction of biorthogonal wavelets

56

finite filters. Consequently, we require that the sets Lj,k = {l ∈ Kj+1 | hj,k,l ̸= 0} and Kj,l = {k ∈ Kj | hj,k,l ̸= 0} are finite. As in the classical setting, there is no analytic formula for the scaling functions φj,k . However, the method for obtaining the graphs of φj,k is essentially the same: start with the Kronecker sequence {cjk0 = δk0 ,k | k ∈ Kj0 } and then generate the sequences {cjk | k ∈ Kj } as ∑ cj+1 = hj,k,l cjk . l k∈Kj,l

This cascade algorithm ensures that the limit functions satisfy lim cj j→∞ k

= φj0 ,k0 (xk ).

For details about this, see [21]. From the two-scale relations we also immediately obtain the general discrete wavelet transform, that is ∑ ˜ j,k,l cj+1 , cjk = h l l∈L˜j,k

djm =

∑

g˜j,m,l cj+1 , l

l∈L˜j,k

for the forward transform, and = cj+1 l

∑ k∈Kj,l

hj,k,l cjk +

∑

gj,m,l djm ,

m∈Mj,l

for the inverse transform, where cjk = ⟨f, φ˜j,k ⟩ and

⟨ ⟩ djm = f, ψ˜j,m .

Note that we have, for this moment, interchanged the role of the primal and dual filters which is permited if the sequences cjk and djk are defined as above (i.e. with φ˜j,k , ψ˜j,k instead of φj,k , ψj,k ). The reason is to stay compatible with the article [21]. The formulas for the discrete wavelet transform can be expressed more compactly using operators Hj and Gj mapping ℓ2 (Kj+1 ) 7→ ℓ2 (Kj ) and ℓ2 (Kj+1 ) 7→ ℓ2 (Mj ) respectively. If we denote the sequences involved as α = {αl | l ∈ Kj+1 ∈ ℓ2 (Kj+1 )}, β = {βk | k ∈ Kj ∈ ℓ2 (Kj )}, γ = {γm | m ∈ Mj ∈ ℓ2 (Mj )}, ∑ then we define β = Hj α if and only if βk = l∈Kj+1 hj,k,l αl and γ = Gj α if and only ∑ ˜ j and G ˜ j . The general discrete if γm = l∈Kj+1 gj,m,l αl and analogously for operators H wavelet transform can be now written as ˜ j cj+1 , cj = H

Chapter 4. Construction of biorthogonal wavelets

57

˜ j cj+1 , dj = G where cj = {cjk | k ∈ Kj } and dj = {djm | m ∈ Mj }. To obtain the inverse transform we first derive the form of adjoint operators Hj∗ , G∗j . Since ⟨Hj α, β⟩ =

∑ k

βk

∑ l

hj,k,l αl =

∑∑ k

hj,k,l αl βk =

∑

l

l

αl

∑

⟨ ⟩ hj,k,l βk = α, Hj∗ β ,

k

∑

it follows that α = Hj∗ β if and only if αl = k hj,k,l βk . A similar argument applies to the adjoint operator G∗j . The inverse transform can now be written as cj+1 = Hj∗ cj + G∗j dj . We have thus derived one step of general discrete wavelet transform, which is displayed on figure 4.6. The full wavelet transform can be obtained by iteration. Figure 4.6: The general discrete wavelet transform

It is an easy exercise to obtain the conditions for exact reconstruction in this general setting: ˜jH∗ = G ˜ j G∗ = 1, H (4.21) j j ˜ j Hj∗ = H ˜ j G∗j = 0, G

(4.22)

˜ j + G∗j G ˜ j = 1. Hj∗ H

(4.23)

˜ j and G ˜ j are called biorthogonal filter operators if the The filter operators Hj , Gj , H conditions (4.21) - (4.23) are satisfied. For later convenience we will extend the operators Hj and Gj to sequences of functions so that φj = Hj φj+1 , ψj = Gj φj+1 , and

˜ ∗ φj + G ˜ ∗ ψj , φj+1 = H j j

where φj = {φj,k | k ∈ Kj } and ψj = {ψj,m | m ∈ Mj }. ˜ old , Gold and We will now state the lifting scheme in the operator notation. Let Hjold , H j j ˜ old be biorthogonal filter operators. Then the filter operators Hj , Gj , H ˜ j and G ˜ j given G j by Hj = Hjold ,

Chapter 4. Construction of biorthogonal wavelets

58

˜j = H ˜ old + Sj G ˜ old , H j j ∗ old Gj = Gold j − Sj Hj ,

˜j = G ˜ old , G j where Sj is arbitrarily chosen operator mapping ℓ2 (Mj ) 7→ ℓ2 (Kj ), are likewise biorthogonal filter operators. The proof simply follows by substituting these relations in the conditions (4.21) - (4.23) and exploiting the biorthogonality of the old filter operators. Such defined process is called (primal) lifting because it modifies the primal wavelet (which will become apparent below). In the index notation the lifting scheme can be written as hj,k,l = hold j,k,l , ∑

˜ j,k,l = h ˜ old + h j,k,l

old sj,k,m g˜j,m,l ,

m∈Mj

∑

old − gj,m,l = gj,m,l

sj,k,m hold j,k,l ,

k∈Kj old g˜j,m,l = g˜j,m,l .

From the above scheme we can deduce how the scaling functions and wavelets change after lifting. Using the extension of the filter operators we obtain φj = φold j , ˜ old φ˜j+1 + Sj G ˜ old φ˜j+1 = H ˜ old φ˜j+1 + Sj ψ˜j , φ˜j = H j j j ∗ old old old ψ˜j = Gold j φj+1 − Sj Hj φj+1 = ψj − Sj φj ,

˜ old ψ˜j = G ˜j , j φ or, in the index notation

φ˜j,k =

∑

φj,k = φold j,k , ˜ old φ˜j+1,l + h j,k,l

l∈Kj+1

∑

old ψj,m = ψj,m −

ψ˜j,m =

∑

∑

sj,k,m ψ˜j,m ,

m∈Mj

sj,k,m φold j,k ,

k∈Kj old g˜j,k,m φ˜j+1,l .

l∈Kj+1

We also introduce dual lifting, which alters the dual wavelets. In operator notation it can be formulated as Hj = Hjold + S˜j Gold j , ˜j = H ˜ jold , H Gj = Gold j ,

Chapter 4. Construction of biorthogonal wavelets

59

˜ old ˜j = G ˜ old − S˜∗ H G j j j The proof is analogous to the previous case. The scaling functions and wavelets are changed due to the dual lifting as follows: ∑ ∑ φj,k = hold s˜j,k,m ψj,m , j,k,l φj+1,l + m∈Mj

l∈Kj+1

ψj,m

φ˜j,k = φ˜old j,k , ∑ old = gj,k,m φj+1,l , l∈Kk+1

old ψ˜j,m = ψ˜j,m −

∑

s˜j,k,m φ˜old j,k .

k∈Kj

The purpose of lifting is to start with a (preferably simple) multiresolution analysis and through the sequences sj,k,m and s˜j,k,m (which are arbitrary!) build a new multiresolution with desired properties. One can, for example, start with the Haar wavelet, increase the number of vanishing moments of the primal wavelet by primal lifting and then increase the number of vanishing moments of the dual wavelet by dual lifting (see [20] how this is done). One can also consider selecting the sequences sj,k,m , s˜j,k,m so that the resulting wavelet resembles some particular shape.

4.4

Lifted wavelet transform

After discussing the basics of lifting we can introduce the fast lifted wavelet transform which is based on lifting. The idea is to start with a trivial orthogonal multiresolution ˜ j = E : ℓ2 (Kj+1 ) 7→ ℓ2 (Kj ) and Gj = G ˜j = D : analysis defined by operators Hj = H ℓ2 (Kj+1 ) 7→ ℓ2 (Mj ) where β = Eα if and only if βk = αk for k ∈ Kj and γ = Dα if and only if γm = αm for m ∈ Mj . The perfect reconstruction properties are clearly satisfied so that we have a pair of orthogonal filter operators. The resulting transform is called lazy wavelet transform. Generally, the lifting scheme can be written as ˜ old cj+1 − S˜j∗ Hjold cj+1 , dj = G ˜ jold cj+1 + Sj dj . cj = H Note that we have first computed dj and later reused it in the calculation of cj . In our situation we have dj = Dcj+1 − S˜j∗ Ecj+1 , cj = Ecj+1 + Sj dj . According to the definition of the operators E and D we can formulate the algorithm for lifted wavelet transform: 1. For all k ∈ Kj , cjk := cj+1 k

Chapter 4. Construction of biorthogonal wavelets

60

2. For all m ∈ Mj , djm := cj+1 m 3. For all m ∈ Mj , djm −=

∑

s˜j,k,m cjk

k

4. For all k ∈ Kj , cjk +=

∑

sj,k,m djm .

m

One of the wonderful properties of lifting is that the inverse is found simply by reversing the order of operations and toggling the signs: 1. For all k ∈ Kj , cjk −=

∑

sj,k,m djm

m

2. For all m ∈ Mj , djm +=

∑

s˜j,k,m cjk

k

3. For all m ∈ Mj , cj+1 := djm m 4. For all k ∈ Kj , cj+1 := cjk . k One step of the lifted wavelet transform is depicted on figure 4.7. The full transform is, again, obtained by iteration. Figure 4.7: Lifted wavelet transform

It comes to question how to find the sequences sj,k,m , s˜j,k,m . There are various approaches how to do it. Later we will present one particularly useful method, that is factoring classical wavelet transforms into lifting steps.

Chapter 4. Construction of biorthogonal wavelets

4.5

61

Interpretation of lifting

We begin this section with a simple example. Suppose we are given a sequence cj = {cjn | n ∈ Z} which may represent samples of some function where some correlation between neighboring samples is expected. To decorrelate the data one may apply the simple Haar transform to obtain two sequences cj−1 and dj−1 with half the size of cj (we use different normalization for convenience): cj−1 = n

cj2n + cj2n+1 , 2

dj−1 = cj2n+1 − cj2n . n Thus cj−1 contain averages of neighboring samples and dj−1 contain differences. The inverse transform can be obtained by a simple substitution: − cj2n = cj−1 n

dj−1 n , 2

dj−1 n = + . 2 To make the transform in-place we may consider reordering the computations so that we first compute the differences as cj2n+1

cj−1 n

dj−1 = cj2n+1 − cj2n , n in the computation of the average, cj−1 and then we reuse dj−1 n : n cnj−1

=

cj2n

dj−1 n + . 2

in place of cj2n+1 because cj2n+1 is not needed in the Now it is possible to store dj−1 n in place of cj2n . The process is depicted computation of cj−1 and then we can store cj−1 n n on figure 4.8. We see that it is a simple example of lifting we defined before. Figure 4.8: Lifting the Haar wavelet

Chapter 4. Construction of biorthogonal wavelets

62

In general, a lifting step consists of three consecutive operations, namely split, predict and update. The split operation simply splits the data according to the parity of indices. This corresponds to the lazy wavelet transform defined before. We will denote this operation as (evenj−1 , oddj−1 ) = split(cj ). In the simple Haar example the predict operation is the measure of failure for the even data to be the same as the odd data. This is expressed by the formula dj−1 = cj2n+1 − cj2n . n In general, the predict operator can be more complicated, but in case of linear wavelet transform it can be expressed as a linear combination of neighbouring samples. We will denote this operation as dj−1 = oddj−1 − P (evenj−1 ). Note that the predict operation corresponds to the dual lifting defined before. In case of the Haar transform the update operation is expressed as cj−1 n

=

cj2n

dj−1 n . + 2

It is easy to show that this operation is necessary to preserve the average across scales, namely 2j−1 −1 2j −1 2j−1 ∑−1 1 ∑ j 1 ∑ j j j−1 (c2n + c2n+1 ) = ck . cn = 2 2 n=0

n=0

n=0

In general, however, the update operation can be more complicated, but in case of linear wavelet transform it is always expressible as a linear combination of neighbouring data. We will denote cj−1 = evenj−1 + U (dj−1 ). The update operation corresponds to the primal lifting defined before. An in-place implementation of a general lifting step can be written as 1. (evenj−1 , oddj−1 ) := split(cj ), 2. oddj−1 −= P (evenj−1 ), 3. evenj−1 += U (oddj−1 ). The general lifting procedure is displayed on figure 4.9. The inverse transform is simply obtained by running the code backwards: 1. evenj−1 −= U (oddj−1 ), 2. oddj−1 += P (evenj−1 ), 3. cj := merge(evenj−1 , oddj−1 ). The inverse transform is depicted on figure 4.10. Another amazing property of lifting is that the operators P and U can be generalized to be non-linear, even irreversible! The reason is that the inverse lifted transform does not require any knowledge of the inverse operators. This property is essential in implementing

Chapter 4. Construction of biorthogonal wavelets

63

Figure 4.9: General lifting step

Figure 4.10: General inverse lifting step

integer to integer wavelet-like transform where instead of P and U we work with ⌊P ⌋ and ⌊U ⌋, respectively. From the analysis in the previous section it follows that after dual and primal lifting we obtain new filters on which we can apply lifting again. This results in an important generalization of the interpretation we discussed above. An example is given on figure 4.11.

4.6

Lifting in terms of z-tranform

We will find useful to examine lifting in terms of the z-transform and relate it with so-called polyphase representaton. This and the following section originate from [14]. Let x∑ n be the input sequence (e.g. a discrete signal) and X(z) its z-transform so that X(z) = n xn z −n . The splitting of X(z) into even and odd indices is obtained as X(z) = Xeven (z 2 ) + z −1 Xodd (z 2 ), where Xeven (z) =

∑ n

x2n z −n ,

Chapter 4. Construction of biorthogonal wavelets

64

Figure 4.11: A transform with two lifting steps

Xodd (z) =

∑

x2n+1 z −n .

n

Analogously, as in (4.8) we derive Xeven (z) =

) 1( X(z 1/2 ) + X(−z 1/2 ) , 2

(4.24)

) z 1/2 ( X(z 1/2 ) − X(−z 1/2 ) . (4.25) 2 In the prediction step we subtract a linear combination of the even samples from the odd, i.e., given Xeven (z) and Xodd (z) we end up with Xeven (z) and Xodd (z) − T (z)Xeven (z), where T (z) is the corresponding Laurent polynomial. This operation can be expressed in the matrix form as ) ) ( )( ( Xeven (z) Xeven (z) 1 0 . = Xodd (z) − T (z)Xeven (z) Xodd (z) −T (z) 1 Xodd (z) =

Similar argument applies to the update step: ) ) ( )( ( Xeven (z) + S(z)Xodd (z) Xeven (z) 1 S(z) , = Xodd (z) Xodd (z) 0 1 where S(z) is the Laurent polynomial used in the update step. In one scale of a discrete wavelet transform there may be a number of predict and update steps. If we define ( H(z) =

K 0 0 K −1 (

then we can write H(z)

)∏ )( ) 1 ( 1 Si (z) 1 0 , 0 1 −Ti (z) 1

(4.26)

i=N

Xeven (z) Xodd (z)

)

( =

Y0 (z) Y1 (z)

) .

(4.27)

The purpose of the normalization factor K will become apparent in the following section. If H(z) is given as in (4.26), we immediately obtain the inverse matrix H−1 (z): [N ( )( )] ( −1 ) ∏ 1 0 1 −S (z) K 0 i −1 H (z) = . Ti (z) 1 0 1 0 K i=1

Chapter 4. Construction of biorthogonal wavelets

65

However, if the matrix H(z) is not factored, the inverse may not be immediately obvious. In such case we have ) ( H00 (z) H01 (z) , H(z) = H10 (z) H11 (z) and the equation (4.27) is called the polyphase representation of the wavelet transform and H(z) is called the polyphase matrix. It has already been used before lifting was invented. It turns out that the invertibility of the polyphase matrix can be determined (without lifting) according to its determinant. To show this, we will first derive a little lemma. Lemma 4.4. A non-zero Laurent polynomial p(z) can be inverted if and only if it is of the form p(z) = αz m , i.e. it is a monomial. ∑N2 Proof. 1. if 0 ̸= p(z) = αn z −n can be inverted then there exists a Laurent ∑M2 n=N1 −n polynomial q(z) = n=M1 βn z such that p(z)q(z) = 1, for all z ∈ C.

(4.28)

If p(z) was not a monomial, then in (4.28) we would have at least two different powers with non-zero coefficients, leading to a contradiction. Hence p(z) must be a monomial. 2. If p(z) = αz m , then it is invertible because p(z) · α−1 z −m = 1.  Now how to apply the lemma to our situation? The reader can easily check that ) ) ( )( ( H11 (z) −H01 (z) 1 0 H00 (z) H01 (z) , = ∆(z) −H10 (z) H00 (z) 0 1 H10 (z) H11 (z) where ∆(z) = det H(z) = H00 (z)H11 (z) − H01 (z)H10 (z). It follows that H(z) is invertible if and only if ∆(z) is invertible which is, by the above lemma, equivalent to saying that ∆(z) is a monomial. Hence for the polyphase matrix H(z) it suffices to check whether its determinant is a monomial and if this is the case it also follows that the inverse is given by ( ) 1 H11 (z) −H01 (z) −1 H (z) = . (4.29) ∆(z) −H10 (z) H00 (z) It turns out that the filter representation introduced in section 4.1 is equivalent to the ˜ are related to the polyphase matrix components polyphase representation if the filters H, H H00 (z), H01 (z), H10 (z), H11 (z) as H(z) = H00 (z 2 ) + zH01 (z 2 ),

(4.30)

G(z) = H10 (z 2 ) + zH11 (z 2 ).

(4.31)

Chapter 4. Construction of biorthogonal wavelets

66

It is a matter of substituting (4.24), (4.25) and (4.9), (4.10) to (4.27) (after upsampling, for convenience) to obtain ( 1 ) ( ) [(X(z) + X(−z)] Y0 (z 2 ) 2 2 H(z ) z = Y1 (z 2 ) 2 [(X(z) − X(−z)] if and only if the relations (4.30), (4.31) hold. So far we have discussed the transition from a factored polyphase matrix to a non-lifted polyphase representation. This was merely a matter of multiplying the matrices involved together. We have also shown that this is equivalent with the filter approach. Now we are ready to discuss the opposite approach, that is the factorization of a polyphase matrix into lifting steps.

4.7

Factorization of a polyphase matrix into lifting steps

In this section we will show how any classical wavelet transform can be implemented using lifting so that it will possess all the advantages that lifting provides, in particular it allows for in-place calculations, reduced time complexity, and simple computation of the inverse transform. For this reason we again impose the condition for classical setting, implying the space we are working with is the space L2 and all scaling functions φj,k involved are obtained from one fixed function φ by means of scaling and translation, i.e. φj,k = 2j/2 φ(2j · −k). Our survey is based on [8, 14]. We want to show that for for any invertible matrix H(z) whose components are Laurent polynomials, i.e. ) ( H00 (z) H01 (z) , H(z) = H10 (z) H11 (z) there exist K ̸= 0 and Laurent polynomials T1 (z), . . . , TM (z) and S1 (z), . . . , SM (z) so that ( H(z) =

K 0 0 K −1

)∏ M ( i=1

1 Si (z) 0 1

)(

1 0 −Ti (z) 1

) .

(4.32)

Note that invertibility in this case means that det H(z) = H00 (z)H11 (z) − H01 (z)H10 (z) is a monomial, that is det H(z) = αz m for some α ̸= 0 and m ∈ Z. We can always assume that det H(z) = 1, for if it was not the case, we can take ( ) H00 (z) H01 (z) ′ H (z) = , α−1 z −m H10 (z) α−1 z −m H11 (z) whose determinant equals one. This H′ (z) then merely corresponds to different z-transform of H10 (z) and H11 (z). It turns out that the desired factorization obtained using the Euclidean algorithm ∑N2 can be −n be a Laurent polynomial. We define for Laurent polynomials. Let a(z) = n=N1 αn z the degree of a(z) as deg(a) = N2 − N1 . If a(z) ≡ 0 we define deg(a) = −∞. Note that by

Chapter 4. Construction of biorthogonal wavelets

67

this definition a monomial z p has degree 0 while as an ordinary polynomial it would have degree p. For Laurent polynomials division with remainder is possible. Given two Laurent polynomials a(z), b(z) ̸= 0 with deg(b) ≤ deg(a) then there exist Laurent polynomials q(z), r(z) with deg(q) = deg(a) − deg(b) and deg(r) < deg(b) such that a(z) = b(z)q(z) + r(z).

(4.33)

We will denote this as q(z) = a(z) ÷ b(z) and r(z) = a(z) mod b(z). The division (4.33) can be iterated and since the degree of the remainder is decreasing with each step, we have the following theorem. Theorem 4.5 (Euclidean algorithm). Let a(z) and b(z) be Laurent polynomials with deg(b) ≤ deg(a). Let a0 (z) = a(z), b0 (z) = b(z) and an+1 (z) = bn (z), bn+1 (z) = an (z) mod bn (z). If N denotes the smallest integer with N < deg(b) + 1 such that bN (z) = 0 then aN (z) is a greatest common divisor of a(z) and b(z). Note that a greatest common divisor is given uniquely up to a factor αz m , since if d(z) divides both a(z) and b(z) and is of maximal degree, then αz m d(z) is also a divisor of the same degree. Let us now apply the Euclidean algorithm to the first row of the polyphase matrix, that is H00 (z) and H01 (z). We set a0 (z) = H00 (z), b0 (z) = H01 (z) and qn+1 (z) = an (z) ÷ bn (z). The first step is then a1 (z) = b0 (z), b1 (z) = a0 (z) − b0 (z)q1 (z). We can write this in matrix form as ) )( ) ( )( ) ( ( a(z) 0 1 H00 (z) 0 1 a1 (z) . = = b(z) 1 −q1 (z) H01 (z) 1 −q1 (z) b1 (z) The second step is a2 (z) = b1 (z), b2 (z) = a1 (z) − b1 (z)q2 (z), or, (

a2 (z) b2 (z)

)

( =

0 1 1 −q2 (z)

)(

a1 (z) b1 (z)

)

( =

0 1 1 −q2 (z)

)(

0 1 1 −q1 (z)

After N steps, we obtain (

aN (z) 0

[

) =

)] ( ) 1 ( ∏ 0 1 H00 (z) . 1 −qn (z) H01 (z)

n=N

)(

H00 (z) H01 (z)

) .

Chapter 4. Construction of biorthogonal wavelets This is equivalent to (

)

H00 (z) H01 (z)

68

[

)] ( ) N ( ∏ qn (z) 1 aN (z) = . 1 0 0 n=1

However, since det H(z) = H00 (z)H11 (z) − H01 (z)H10 (z) = 1,

(4.34)

the greatest common divisor u(z) of H00 (z) and H01 (z) must divide the entire left side of (4.34) and therefore u(z) must divide 1. Consequently, u(z) must be a monomial, and since aN (z) is one of the greatest common divisors, we must have aN (z) = Kz m . By a suitable shifting of H00 (z) and H01 (z) we can always achieve ( ′ ) ( −m ) [∏ )] ( ) N ( H00 (z) z H00 (z) qn (z) 1 K = = . ′ (z) H01 z −m H01 (z) 1 0 0 n=1

Since

(

qn (z) 1 1 0

)

( = (

and

1 qn (z) 0 1 0 1 1 0

)(

)(

0 1 1 0

0 1 1 0

)

( =

)

( =

0 1 1 0

1 0 0 1

)(

1 0 qn (z) 1

)

) ,

we can also write   ) ) ( )( ) ( N/2 ( ∏ 1 0  K 1 q2n−1 (z) H00 (z)  , = 0 q2n (z) 1 0 1 H01 (z)

(4.35)

n=1

which holds for N even. For N odd such that N = 2N ′ − 1 we set q2N ′ = 0 so that (4.35) holds for N odd as well. We will now observe what happens if we replace ) ( K 0 (

with

Indeed, it is possible to write (

′ (z) H00 (z) H10 ′ (z) H01 (z) H11

)



=

N/2 (

∏

n=1

K 0 0 K −1

1 q2n−1 (z) 0 1

) .

)(

1 0 q2n (z) 1

 ) ( ) 0  K , 0 K −1

or, after transposing, ( ) ( ) ∏ ( )( ) 1 H00 (z) H01 (z) K 0 1 q2n (z) 1 0 = . ′ (z) H ′ (z) H10 0 K −1 0 1 q2n−1 (z) 1 11

(4.36)

n=N/2

′ (z), H ′ (z) and valid H (z), H (z). For this We want to find a relation between H10 10 11 11 purpose we have the following lemma:

Chapter 4. Construction of biorthogonal wavelets

69

Lemma 4.6. If a filter pair H, G has a polyphase representaton with determiant equal one then any other filter pair H, G′ is related by G′ (z) = G(z) + s(z 2 )H(z),

(4.37)

where s(z) is a Laurent polynomial. Proof. The polyphase representation of H and G is obtained using (4.30) and (4.31) as H(z) = H00 (z 2 ) + zH01 (z 2 ), G(z) = H10 (z 2 ) + zH11 (z 2 ). Substituting this into (4.37) yields [ ] [ ] ′ ′ G′ (z) = H10 (z 2 ) + s(z 2 )H00 (z 2 ) + z H11 (z 2 ) + s(z 2 )H01 (z 2 ) = H10 (z 2 ) + zH11 (z 2 ), where

′ H10 (z) = H10 (z) + s(z)H00 (z),

′ H10 (z) = H10 (z) + s(z)H00 (z).

(4.38)

The new polyphase matrix is thus given by ( ) H00 (z) H01 (z) ′ H (z) = H10 (z) + s(z)H00 (z) H11 (z) + s(z)H01 (z) and its determinant equals det H′ (z) = H00 (z) [H11 (z) + s(z)H01 (z)] − H01 (z) [H10 (z) + s(z)H00 (z)] = H00 (z)H11 (z) − H01 (z)H10 (z) = det H(z) = 1, where det H(z) denotes the polyphase matrix of the original filters H, G. Its determinant is by the assumptions equal one.  From (4.38) in the above lemma it also follows that ′ H10 (z) = H10 (z) − s(z)H00 (z), ′ H11 (z) = H11 (z) − s(z)H01 (z),

or, in matrix form, (

′ (z) H00 (z) H10 ′ (z) H01 (z) H11

)(

−s(z) 1

)

( =

H10 (z) H11 (z)

) .

From (4.29) it follows that ( )( ) ( ) ′ (z) ′ (z) −s(z) H11 −H10 H10 (z) = . ′ (z) 1 −H01 (z) H00 H11 (z) ′ (z)H (z) − H ′ (z)H (z). Multiplying (4.36) with Hence s(z) = H10 11 10 11 ( ) 1 0 , −s(z) 1

Chapter 4. Construction of biorthogonal wavelets

70

results in ) ( ) ( )( ) ∏ )( 1 ( H00 (z) H01 (z) 1 0 K 0 1 q2n (z) 1 0 = H10 (z) H11 (z) −s(z) 1 0 K −1 0 1 q2n−1 (z) 1 n=M

( =

K 0 0 K −1

)(

1 0 −K 2 s(z) 1

) ∏ )( ) 1 ( 1 0 1 q2n (z) , 0 1 q2n−1 (z) 1 n=M

where M = N/2. It is now only a matter of taste to reorder the Laurent polynomials qi (z) and set qM +1 = −K 2 s(z) to obtain the relation (4.32). We have thus derived the factorization of any polyphase matrix (and therefore any wavelet transform) into lifting steps. Note that since the greatest common divisor is not unique, the same applies to the factorization of a polyphase matrix. Let us apply the above procedure on the Daubechies wavelet with 2 vanishing moments derived in section 3.6. We choose the z-transform of the low-pass filter H as H(z) = h0 z 3 + h1 z 2 + h2 z + h3 , where

√ √ √ √ 3+ 3 3− 3 1− 3 1+ 3 h0 = √ , h 1 = √ , h 2 = √ , h 3 = √ . 4 2 4 2 4 2 4 2

The highpass filter G is given by G(z) = h3 z − h2 + h1 z −1 − h0 z −2 . We separate the even and odd terms according to (4.30) and (4.31) to assemble the polyphase matrix ) ( h3 + h1 z h 2 + h0 z . (4.39) H(z) = −h2 − h0 z −1 h3 + h1 z −1 The first step of the Euclidean algorithm is to set a0 (z) = H00 (z) and b0 (z) = H01 (z). We want to find q1 (z) and r1 (z) with deg(q1 ) = 0 and deg(r1 ) = 0 (hence q1 (z) = c and r1 (z) = dz m ) such that a0 (z) = b0 (z)q1 (z) + r1 (z). Inserting all the terms yields ( √ √ ) √ √ 1− 3 3+ 3 3− 3 1+ 3 √ + √ z= √ + √ z c + r1 (z). 4 2 4 2 4 2 4 2 √

3. Then r1 (z) is found as √ 1− 3 r1 (z) = a0 (z) − b0 (z)q1 (z) = √ . 2

We choose to match the terms with z and obtain q1 (z) =

Next we set

√ √ 3− 3 1+ 3 a1 (z) = b0 (z) = √ + √ z, 4 2 4 2 √ 1− 3 b1 (z) = r1 (z) = √ . 2

Chapter 4. Construction of biorthogonal wavelets

71

Now q2 (z) = c + dz, r2 (z) = 0 and the division is exact. We have a1 (z) = b1 (z)q2 (z). Inserting all the terms yields √ √ 2+ 3 3 q2 (z) = − z− . 4 4 √ 1− 3 K = a2 (z) = b1 (z) = √ 2

We also have

and K

−1

√ 1+ 3 =− √ . 2

We can now write (

H00 (z) H01 (z) ′ (z) H ′ (z) H10 11

(

) =

√ 1− √ 3 2

0

√ − 1+√2 3

0 ( =

)(

H00 (z) H01 (z) √ √ 3+ 3 √ − 2 − 1+√2 3

√

√

1 − 2+4 3 z − 0 1

√

3 4

)(

√1 0 3 1

)

) .

√

′ (z) = − 3+ √ 3 and H ′ (z) = − 1+ √ 3 . According to (4.39) we also have Therefore H10 11 2 2

√ √ 3 − 3 1 + 3 −1 H10 (z) = −h2 − h0 z = √ − √ z , 4 2 4 2 √ √ 1 − 3 3 + 3 −1 −1 H11 (z) = h3 + h1 z = √ + √ z . 4 2 4 2 −1

The term s(z) can then be computed as ′ ′ s(z) = H10 (z)H11 (z) − H11 (z)H10 (z).

The last lifting step q3 (z) is obtained as ( q3 (z) = −K 2 s(z) =

√ )2 √ 1− 3 √ (2 + 3)z −1 = z −1 . 2

The complete factorization is given by )( ( 1−√3 √ )( √ 0 1 0 1 − 2+4 3 z − 2 √ H(z) = z −1 1 0 1 0 − 1+√2 3

√

3 4

)(

√1 0 3 1

) .

To obtain the actual equations for implementation, we will apply the factorized matrix stepwise on the even and odd components of an input signal S(z). The first step is ( )( ) ( ) ( ) S0 (z) S0 (z) S0 (z) √1 0 √ = = , S1 (z) 3 1 S1 (z) + 3S0 (z) D(1) (z)

Chapter 4. Construction of biorthogonal wavelets where S0 (z) =

∑

n s2n z

−n ,

S1 (z) =

∑

n s2n+1 z

d(1) n The second step is

(

−n .

Hence the first equation reads √ = s2n+1 + 3s2n . )(

) S0 (z) D(1) (z) ( ) ( √ √ ) S (1) (z) S0 (z) − 2+4 3 zD(1) (z) − 43 D(1) (z) = = . D(1) (z) D(1) (z) √

1 − 2+4 3 z − 0 1

√

3 4

It follows that the second equation is s(1) n

√ √ 2 + 3 (1) 3 (1) = s2n − dn+1 − d . 4 4 n

The third step is ) ) ( (1) ) ( ) ( (1) ( S (z) S (1) (z) S (z) 1 0 . = = z −1 1 D(2) (z) D(1) (z) + z −1 S (1) (z) D(1) (z) Consequently, the third equation reads (1)

(1) d(2) n = dn + sn−1 .

The final normalization step gives √ 1 − 3 (1) s˜n = √ sn , 2 √ 1 + 3 (2) ˜ dn = − √ dn . 2 We summarize all the equations:

s(1) n

√ d(1) 3s2n , = s + 2n+1 n √ √ 2 + 3 (1) 3 (1) = s2n − dn+1 − d , 4 4 n (1)

(1) d(2) n = dn + sn−1 , √ 1 − 3 (1) s˜n = √ sn , 2 √ 1 + 3 (2) ˜ dn = − √ dn . 2 The inverse transformation is simply found by running the code backwards: √ 1− 3˜ (2) dn = √ dn , 2

72

Chapter 4. Construction of biorthogonal wavelets

s(1) n

73

√ 1+ 3 = − √ s˜n , 2 (1)

s2n

(2) d(1) n = dn − sn−1 , √ √ 2 + 3 (1) 3 (1) (1) = sn + dn+1 + d , 4 4 n √ s2n+1 = d(1) n − 3s2n .

An important fact to note is that the lifting implementation performs better than the the standard Mallat algorithm. In our example of lifted Daubechies wavelet with two vanishing moments there are 4 additions and 5 multiplications while in the Mallat algorithm there are together 6 additions and 8 multiplications. For longer filters the complexity reduces by a factor 1/2 (see [8]).

Bibliography

74

Bibliography [1] A. Bultheel, Wavelets with Applications in signal and image processing, 2003. [2] C. S. Burrus, R. A. Gopinath, and H. Guo, Introduction to Wavelets and Wavelet Transforms, Prentice Hall, New Jersey, 1998. [3] O. Christensen and T. K. Jensen, An Introduction to the Theory of Bases, Frames, and Wavelets, Technical University of Denmark, 1999. [4] C. K. Chui, An Introduction to wavelets, Academic Press, Inc., San Diego, 1992. [5] A. Cohen, I. Daubechies, and J. Feauveau, Biorthogonal bases of compactly supported wavelets, Comm. Pure Appl. Math, 45 (1992), pp. 485–560. [6] I. Daubechies, Orthonormal bases of compactly supported wavelets, Comm. Pure Appl. Math., 41 (1988), pp. 909–996. [7] I. Daubechies, Ten Lectures on Wavelets, Society for Industrial & Applied Mathematics, Philadelphia, 1992. [8] I. Daubechies and W. Sweldens, Factoring wavelet transforms into lifting steps, J. Fourier Anal. Appl., 4 (1998), pp. 247–269. [9] R. Duffin and A. Schaeffer, A class of nonharmonic Fourier series, Trans. Amer. Math. Soc., 72 (1952), pp. 341–366. [10] H. Dym and H. P. McKean, Fourier Series and Integrals, Academic Press, New York, 1972. [11] G. B. Folland, Real Analysis: Modern Techniques and Their Applications, WileyInterscience, second ed., 1999. [12] C. Heil and D. Walnut, Continuous and discrete wavelet transforms, SIAM Review, 31 (1989), pp. 628–666. ´ ndez and G. Weiss, A First Course on Wavelets, CRC Press, 1996. [13] E. Herna [14] A. Jensen and A. la Cour-Harbo, Ripples in Mathematics, Springer, 2001. [15] F. Keinert, Wavelets and Multiwavelets, CRC Press, 2004.

Bibliography

75

[16] S. Mallat, A theory for multiresolution signal decomposition: The wavelet representation, IEEE Trans. on Pattern Analysis and Machine Intelligence, 11 (1989), pp. 674– 693. [17]

, A Wavelet Tour of Signal Processing, Academic Press, New York, second ed., 1999.

[18] W. Rudin, Real and Complex analysis, McGraw-Hill, London, 1970. [19] W. Sweldens, The lifting scheme: A new philosophy in biorthogonal wavelet constructions, in Wavelet Applications in Signal and Image Processing III, A. F. Laine and M. Unser, eds., Proc. SPIE 2569, 1995, pp. 68–79. [20]

, The lifting scheme: A custom-design construction of biorthogonal wavelets, Appl. Comput. Harmon. Anal., 3 (1996), pp. 186–200.

[21]

, The Lifting Scheme: A Construction of Second Generation Wavelets, SIAM J. Math. Anal., 29 (1997), pp. 511–546.

[22] D. F. Walnut, An introduction to wavelet analysis, Birkhauser, Boston, 2002. [23] R. M. Young, An Introduction to Nonharmonic Fourier Series, Academic Press, 1980.