Linear Algebra - Concepts and Techniques on Euclidean Spaces

Ma Siu Lun • Ng Kah Loon • Victor Tan

LINEAR ALGEBRA Concepts and Techniques on Euclidean Spaces

Ma Siu.Lun N gKahLoon Victor Tan

Singapore • Boston • Burr Ridge, IL • Dubuque, lA • Madison, WI • New York San Francisco • St. Louis • Bangkok • Kuala Lumpur • Lisbon • London • Madrid Mexico City • Milan • Montreal • New Delhi • Seoul • Sydney • Taipei • Toronto

Linear Algebra Concepts and Techniques on Euclidean Spaces

IJ Higher Education Copyright © 2012 by McGraw-Hill Education (Asia). All rights reserved. No part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system without the prior written permission of the publisher, including, but not limited to, in any network or other electronic storage or transmission, or broadcast for distance learning.

Cover image © 123RF

~------e~ The Asian Customized Edition is a speedy publishing service for )i._lecturers and has not undergone the full McGraw-Hill Education (Asia) editorial process.

.

10 9 8 7 6 5 4 3 2 CTP MPM 20 15 14 13 12

When ordering this title, use ISBN 978-1-259-01151-1 or MHID 1-259-01151-8.

Printed in Singapore

Preface This publication is the successor of the third edition of the book "Linear Algebra I" published by the authors with their previous publisher. It is used as the course lecture notes for the undergraduate module MA1101R, Linear Algebra I , offered by the Department of Mathematics at the National University of Singapore. This module is the first course on linear algebra and it serves as an introduction to the basic concepts of linear algebra that are routinely applied in diverse fields such as science, engineering, statistics, economics and computing. Mindful that majority of the students taking this module are new to the subject , we have chosen to introduce the concepts of linear algebra in the context of Euclidean spaces rather than to jump straight into abstract vector spaces, which will be covered in the second course. The set up in Euclidean spaces also facilitates the connections between the algebraic and geometric viewpoints of linear algebra. Formal proofs of most of the basic theorems in linear algebra have been included to enhance a proper understanding of the fundamental ideas and techniques. Several applications of linear algebra in some of the fields mentioned above are also highlighted. At the end of every chapter is a good collection of problems, all of which are culled from tutorial problems , test and examination questions from the same module taught by the authors in the past . These problems range from the straightforward computational ones to some highly challenging questions. In order to achieve a deeper understanding of the topic, students are advised to work through these problems. Significant updates and revisions have been made in this new edition, including the discussion , examples and exercises. Many of the changes are done in response to feedback received from students , and teaching after-thoughts that the authors have accumulated over the past years. We believe that the

lV

Linear Algebra: Concepts and Techniques on Euclidean Spaces

revisions made to the presentation of materials will further enhance learners' understanding of the critical concepts in this subject. Finally, the authors would like to thank their colleagues from the Mathematics Department in NUS who have contributed to the very first version of the lecture notes in 1998, especially Chan Onn, Tan Hwee Huat , Roger Tan Choon Ee and Tang Wai Shing. More recently, Toh Pee Choon (now with the National Institute of Education) has also given many useful comments on areas of improvement from the previous edition.

m ers' _:athe_.::ion of

Contents Chapter 1 Section 101 Section 102 Section 103 Section 104 Section 1.5 Exercise 1 Chapter 2 Section 201 Section 202 Section 203 Section 2.4 Section 205 Exercise 2 Chapter 3 Section 301 Section 302 Section 303 Section 304 Section 305 Section 306 Section 307 Exercise 3 Chapter 4 Section 401 Section 402

Linear Systems and Gaussian Elimination Linear Systems and Their Solutions Elementary Row Operations Row-Echelon Forms Gaussian Elimination Homogeneous Linear Systems 0 Matrices Introduction to Matrices Matrix Operations 0 Inverses of Square Matrices Elementary Matrices Determinants Vector Spaces 0 Euclidean n-Spaces Linear Combinations and Linear Spans Subspaces Linear Independence Bases Dimensions 0 Transition Matrices Vector Spaces Associated with Matrices Row Spaces and Column Spaces Ranks 0

1 1 6 8 11 22 24 33 33 36 45 48 58 69 83 83 88 95 99 103 109 113 117 127 127 136

Linear Algebra: Concepts and Techniques on Euclidean Spaces

ection 4.3 Nullspaces and Nullities . Exercise 4 Chapter 5 Orthogonality . Section 5.1 The Dot Product Section 5.2 Orthogonal and Orthonormal Bases Section 5.3 Best Approximations Section 5.4 Orthogonal Matrices . Exercise 5 Chapter 6 Diagonalization . Section 6.1 Eigenvalues and Eigenvectors Section 6.2 Diagonalization Section 6.3 Orthogonal Diagonalization . Section 6.4 Quadratic Forms and Conic Sections . Exercise 6 Chapter 7 Linear Transformations Section 7.1 Linear Transformations from IR.n to IR.m Section 7.2 Ranges and Kernels Section 7.3 Geometric Linear Transformations Exercise 7 Index

138 142 147 147 152 159 164 169 177 177 184 191 195 201 209 209 214 218 229 237

uclidean Spaces

138 142 147 147 152 159 164 169 177 177 184 191 195 201 209 209 214 218 229 237

Chapter 1

Linear Systems and Gaussian Elimination

Section 1.1

Linear Systems and Their Solutions

Discussion 1.1.1 A line in the xy-plane can be represented algebraically by an equation of the form ax+ by= c

where a and b are not bot h zero. An equation of t his kind is known as a linear equation in t he variables of x and y. In general. we have the following definition. Definition 1.1.2 A linear equation in n variables x 1 . x 2 . . . .

, Xn

has t he form

where a 1 , a 2 . . .. . an and bare real constants. The variables in a linear equation are also called the unknowns. \Ve do not need t o assume that a 1 , a 2 , ... , an are not all zero. If all a 1 , a 2 , ... , an and b are zero, the equation is c,alled a zero equation. A linear equation is called a nonzero equation if it is not a zero equation. Example 1.1.3 1. The equations x + 3y = 7, x 1 + 2x 2 x 1 + x2 + · · · + Xn = 1 are linear.

+ 2x 3 + x 4

x - ~z

+ 4.5

and

Chapter 1. Linear Systems and Gaussian Elimination

2

2. The equations are not linear.

xy

= 2, sin(B) +cos(¢) = 0.2 , xi+ x~ + · · · + x~

1 and x

=

eY

3. The linear equation ax+ by+ cz = d, where a, b, c, d are constants and a, b, c are not all zero , represents a plane in the three dimensional space. For example, z = 0 (i.e. Ox+ Oy + z = 0) is the xy-plane contained inside the xyz-space .

z

-----7

/ / /

y

/ / /

/

""' - - - - - -

z = O X

Definition 1.1.4 Given n real numbers s1, s2 , .. . , sn , we say that X1 = s1 , x2 = s2 , ... , Xn = Sn is a solution to a linear equation a1x 1 + a2x2 + · · · + anXn = b if the equation is satisfied when we substitute the values into the equation accordingly. The set of all solutions to t he equation is called the solution set of the equation and an expression that gives us all these solutions is called a general solution for t he equation.

Example 1.1.5 1. Consider the linear equation 4x - 2y X = {

y

t

= 2t- ~

=

1. It has a general solution

where t is an arbitrary parameter.

The equation also has another general solution

x = ls 2 { y=s

+ l4

where s is an arbitrary parameter.

Though the two general solutions above look different , they gives us the same set of solutions including 1 y = 1.5,

and infinitely many other solutions.

1.5 y = 2.5,

{

-1 y = - 2.5 , X=

X =

X =

{

{

;an Elimination

and x = eY

a, b, c are not )le, z = 0 (i.e.

Section 1. 1. Linear System s and Their Solutions

3

2. Consider the equation x 1 - 4x 2 + 7x 3 = 5. It has a general solution

{

x1 = 5 + 4s - 7t X2 = S X3

where s, t are arbitrary parameters .

=t

3. (Geometrical Interpretation)

y

(a) In the xy-plane, solutions to the equation x + y = 1 are points

(x,y) = (1- s, s)

x+y= 1

where s is any real number. These points form a line as shown in the diagram on the right.

X

z

1> X2 =52, ... ' the equation is of all solutions hat gives us all

(b) In the xyz-space , solutions to the equation x + y = 1 (i.e. x + y + Oz = 1) are points

r

(x,y,z) = (1- s, s, t)

------

I I

where s and t are any real numbers . These points form a plane as shown in the diagram on the right. X

4. Consider t he zero equation Ox 1 + Ox2 + · · · + Ox n = 0. Any values of x1, x2, . .. , Xn give us a solution. Thus t he general solution is x 1 = t 1, x2 = t2, .. . , Xn = tn where h , t2 , ... , tn are arbitrary parameters. 5. For an equation Ox 1 + Ox 2 + · · · + Oxn = b, where b is nonzero , any values of X1 , x2, ... , Xn does not satisfy the equation and hence the equation has no solution.

the same set of

Definition 1.1.6 A finite set of linear equations in t he variables x 1, x 2, ... , system of linear equations (or a linear system): aux1

a21X1

+ +

a12x2 a22X2

+ ·· · + + ·· · +

a1nXn a2nXn

{

am1 X1

+ am2X2 + · · · + amnXn

= b1 = b2

~

bm

Xn

is called a

4

Chapter 1. Linear Systems and Gaussian Elimination

where a11,a12,····amn and b1. b2 . ... , bm are real constants. If all a 11 , a 12 .... , amn and b1, b2 , . : . , bm are zero , the system is called a zero system. A linear system is called a nonzero system if it is not a zero system. Given n real numbers s1 , s2, .. . , Sn, we say that x 1 = s 1 , X2 = s2, . . . , Xn = Sn is a solution to t he system if x1 = s 1 , x2 = s2, ... , Xn = s 11 is a solution to every equation in the system. The set of all solutions to the system is called the solution set of the system and an expression that gives us all t hese solutions is called a general solution for the system. Example 1.1. 7 Consider the system of linear equations 4xl - X2 { 3x1 + x2

+ 3x3 = + 9x3 =

-1 -4 .

x 1 = 1, x 2 = 2, x3 = -1 is a solution to the system and x 1 solution to the system .

= 1, X2 = 8,

X3

= 1 is not a

Remark 1.1.8 Not all systems of linear equations have solutions . For example. the following system has no solution as it is impossible to have a solution that satisfies both equations simultaneously. x + y=4 { 2x+2y = 6. Definition 1.1.9 A system of linear equations that has no solution is said to be inconsistent. A system that has at least one solution is called consistent. Remark 1.1.10 Every system of linear equations has either no solution, only one solution, or infinitely many solutions. (See Question 2.22.) Discussion 1.1.11 1. In the xy-plane, the two equations in the system

+ b1 y = a2 x + b2y =

a1 x {

c1 c2,

where a 1 , b1 are not both zero and a 2, b2 are not both zero, represent two straight lines. A solution to the system is a point of intersection of the two lines. (a) The system has no solution if and only if L 1 and L 2 are different but parallel lines. (b) The system has only one solution if and only if L 1 and L 2 are not parallel lines. (c) The system has infinitely many solutions if and only if L 1 and L 2 are the same line.

ian Elimination

Section 1.1. Linear Systems and Their Solutions

5

' · .. . , amn and om is called a

Xn = Sn is a ry equation in he system and the system.

No solution

One solution

Infinitely many solutions

Note t hat t he two lines L 1 and L 2 are parallel if and only if a 1 for a nonzero real number k.

= ka 2 and b1 = kb 2

2. In the xyz-space, the two equations in the system

·3

=

1 is not a

ple, the followboth equations

a1x +b1y+c1z = d1 { a2X + b2y + C2Z = d2, where a 1 , b1 , c1 are not all zero and a 2 , b2 , c2 are not all zero, represents two planes. A solution to the system is a point of intersection of the two planes. (a) The system has no solution if and only if P 1 and P 2 are different but parallel planes.

to be inconsis-

(b) The system cannot have only one solution. (c) The system has infinit ely many solutions if and only if either P 1 and P 2 intersect at a line or H and P 2 are the same plane.

ly one solution,

nt two straight es.

No solut ion

Infinitely many solutions

Infinitely many solutions

·nt but parallel

,t parallel lines.

,2

are the same

Not e that the two planes P 1 and P 2 are parallel if and only if a 1 = ka 2 , b1 = kb 2 and c 1 = kc2 for a nonzero real number k .

Chapter 1. Linear Systems and Gaussian Elimination

6

Section 1.2

Elementary Row Operations

Definition 1.2.1 A system of linear equations + a12x2 + · · · + a21X1 + a22X2 + · · · +

anx1

{

am1X1

~ am2X2 + · · · +

a1nXn

=

a2nXn

= b2

amnXn

b1

~ brn

can be represented by a rectangular array of numbers

This array is called the augmented matrix of the system.

Example 1.2.2 The array 1 4 6

2

-3 -5

is the augmented matrix of the system of linear equations X1 + X2 + 2X3 = 9 2x1 +4x2- 3x3 = 1 { 3xl + 6x2 - 6x3 = 0 .

Discussion 1.2.3 The following are the basic techniques for solving a system of linear equations: 1. Multiply an equation by a nonzero constant. 2. Interchange two equations. 3. Add a multiple of one equation to another equation. In terms of the augmented matrix, these correspond to: 1. Multiply a row by a nonzero constant. 2. Interchange two rows. 3. Add a multiple of one row to another row.

---

--

---

--~

,;;;;:;

--~;.:·.:::.

H

::,,., '"'

-

:::?=

an Elimination

Section 1.2. Elementary Row Operations

7

Definition 1.2.4 The three operations of the augmented matrix described above are known as eleme'(l,tary row operations. Example 1.2.5 The following is an example of solving a system of linear equations by equation additions and subtractions. First, we start with the following system and its augmented matrix.

x+ y+3z = 0 2x -2y+2z = 4 { 3x+9y = 3

(1)

(2)

(

(3)

~ ~2 ~ ~ ) 3

9

0

3

Add -2 times of Equation (1) to Equation (2) to obtain Equation (4). This is equivalent to adding -2 times of the first row of the matrix to the second row.

{

x+ y+3z ~ 0 - 4y -4z = 4 3x+9y =3

u n u n

(1) (4) (3)

1 -4 9

3 -4

0

Add -3 times of Equation (1) to Equation (3) to obtain Equation (5). This is equivalent to adding -3 times of the first row of the matrix to the third row.

{ x+ y+3z ~ 0 -4y-4z = 4 6y- 9z = 3

(1) (4) (5)

1 -4 6

3 -4 -9

Add ~ times of Equation (4) to Equation (5) to obtain Equation (6). This is equivalent to adding ~ times of the second row of the matrix to the third row.

x+ ·stem of linear

{

y+ 3z = 0 -4y- 4z = 4 -15z = 9

(1) (4) (6)

1

3

-4

-4

0

-15

By Equation (6), z = -~. Substituting z = -~ into Equation (4),

-4y - 4(- ~)

=

4

{=?

Substituting y = -~ a:nd z = -~ into Equation (1),

Thus we find that x =

V, y = -~, z = -~ is the only solution to the system.

Definition 1.2.6 Two augmented matrices are said to be row equivalent if one can be obtained from the other by a series of elementary row operations.

Chapter 1. Linear Systems and Gaussian Elimination

8

Theorem 1.2. 7 If augmented matrices of two systems of linear equations are row equivalent, then the two systems have the same set of solutions. Example 1.2.8 All augmented matrices in Example 1.2.5 are row equivalent. Thus all systems of linear equations in Example 1.2.5 have the same solution. Remark 1.2.9 To see why Theorem 1.2.7 is true, we only need to check that every elementary row operation applied to an augmented matrix will not change the solution set of the corresponding linear system. Since it is easier to work with linear systems using the matrix representation discussed in Chapter 2, we shall postpone the proof of the theorem

to Section 2.4, see Remark 2.4.6.

Section 1.3

Row-Echelon Forms

Definition 1.3.1 An augmented matrix is said to be in row-echelon form if it has the

following properties: 1. If there are any rows that consist entirely of zeros, then they are grouped together at the bottom of the matrix. 2. In any two successive rows that do ·not consist entirely of zeros, the first nonzero number in the lower row occurs farther to the right than the first nonzero number in the higher row. (The first nonzero number in a row is called the leading entry of the row.) In a row-echelon form, the leading entries of nonzero rows are also called pivot points. A column of a row-echelon form is called a pivot column if it contains a pivot point: otherwise. it is called a non-pivot column. An augmented matrix is said to be in reduced row-echelon form if it is in row-echelon form and has the following additional properties: 3. The leading entry of every nonzero row is 1. 4. In each pivot column, except the pivot point, all other entries are zero . Remark 1.3.2 In some textbooks, row-echelon forms are required to have the additional

property 3 in Definition 1.3.1. Example 1.3.3

1. The following augmented matrices are in reduced row-echelon form. The underlined numbers are pivot points (the leading entries of nonzero rows).

an Elimination

~e

Section 1.3. Row-Echelon Forms

row equiva-

(1 213)

(~

9

~I~)

ent. Thus all

hat every elesolution set of ems using the ,f the theorem

1

1 2 0

0 D0 Du D 0

0

0

1 0 0 1

1

0 0 0

0

0 0 0

1 0 0

2. The following augmented matrices are in row-echelon form but not in reduced rowechelon form. The underlined numbers are pivot points (the leading entries of nonzero rows).

(J

2

11) (

~ ~1 ~ ~, ~ ~ ~ ~ ~ 1

)

(

)

(

-

:

)

(

-

HHi) 00000

3. The following augmented matrices are neither in row-echelon form nor in reduced

row-echelon form.

(~ if it has the

0 D0 Dc

~I~)

0 1 1

2 0 1

0 1 0 0

0 -1 0

0 2 0 0

0 0 1 0

D

,ed together at

Discussion 1.3.4 If the augmented matrix of a system of linear equations is in row-echelon form or reduced row-echelon form, we can get the solutions to the system easily.

' first nonzero ~ero number in 1g entry of the

Example 1.3.5

1icot points. A Jint; otherwise,

1 0 1. The acmy ( 0 1 0 0

1 2 ) ;, the augmented matcix of a 'Y'tem of Hneac equation' 3 in 3 variables x1, x2, X3:

iY-echelon form

0 0 1

x1 + Ox2 + Ox 3 = 1 Ox1 + x2 + Ox3 = 2 { Ox1 + Ox2 + X3 = 3

or

=

1, x 2

The system has only one solution: x 1

the additional

The underlined

2. The array (

~ ~ ~ ~ 0

Ox1 Ox1 { Ox1

0

0

0

-2 2) 1 2

3 4

=1 = 2

X2 X3

=

2, x 3

=

= 3.

3.

is the augmented matrix of a system of linear

+ 2x2 + 2x3 + X4 - 2xs = 2 + Ox2 + x3 + X4 + xs = 3 + Ox2 + Ox3 + Ox4 + 2xs = 4

2x 2 + 2x3 or

{

+ x 4 - 2xs = 2 X3 +x4 + xs = 3 2x 5 = 4.

Chapter 1. Linear Systems and Gaussian Elimination

10

The coefficients of x 1 are zero in all the three equations and this means that x1 is arbitrary (see also Example l.l.5.3(b)). By the third equation, x 5 = 2. Substituting

x 5 = 2 into the second equation,

Substituting x 3 = 1 - x 4 and x 5 = 2 into the first equation,

Thus the linear system has a general solution X1

=S

xz

= 2 + ~t

x3

=

X4

=t

1- t

where s, t are arbitrary parameters,

xs = 2,

and has infinitely many solutions.

(The method we use here is called the back-

substitution.) 1

-1

0

3

0 0

1

2 0

3. The array ( 0 0

0

~2)

equations in 4 variables x 1, xz,

X3,

is the augmented matrix of a system of linear x4:

x1- xz + Ox3 + 3x4 = -2 {

Ox1 + Oxz +

= Ox4 =

X3 + 2x4

Ox 1 + Oxz + Ox3 +

5 0

or

X1 -xz {

+3x4 = -2 X3 +2x4 = 5.

The linear system has a general solution x1 = -2+s-3t

=s X3 = 5- 2t X4 = t,

Xz

{

where s, t are arbitrary parameters,

and has infinitely many solutions.

4. The array (

~

0 0

oo\oo) is the augmented matrix of a system of linear equations Ox1 + Ox2 + Ox3 {

Ox1 + Ox2 + Ox3

=0 = 0.

ian Elimination

ans that x 1 is ~. Substituting

Section 1.4. Gaussian Elimination

11

This is a zero system and the general solution is

x1 {

=r

X2

=

S

X3

=

t.

where r, s, t are arbitrary parameters,

and has infinitely many solutions.

5 The acmy (

~

2 variables x 1 ,

1

2 : ) i' the augmented matcix of a 'Y'tem of lineae equation' in 0 X2:

3xl + x 2 = 4 Ox 1 + 2x 2 = 1 { Ox1 + Ox 2 = 1.

Since the last equation has no solution, the system is inconsistent.

llled the back-

;stem of linear

-2 5.

Section 1.4

Gaussian Elimination

Definition 1.4.1 Let A and R be row-equivalent augmented matrices. i.e. R can be obtained from A by a series of elementary row operations. If R is in row-echelon form. R is called a row-echelon form of A and A is said to have a row-echelon form R. Similarly. if R is in reduced row-echelon form, R is called a reduced row-echelon form of A and A is said to have a reduced row-echelon form R. Algorithm 1.4.2 (Gaussian Elimination) The following procedures reduce an augmented matrix to a row-echelon form by using elementary row operations.

rs.

.inear equations

Step 1: Locate the leftmost column that does not consist entirely of zeros. Step 2: Interchange the top row with another row, if necessary, to bring a nonzero entry to the top of the column found in Step 1. Step 3: For each row below the top row, add a suitable multiple of the top row to it so that the entry below the leading entry of the top row becomes zero . Step 4: Now cover the top row in the matrix and begin again with Step 1 applied to the submatrix that remains. Continue in this way until the entire matrix is in row-echelon form.

Chapter 1. Linear Systems and Gaussian Elimination

12

Algorithm 1.4.3 (Gauss-Jordan Elimination) Given an augmented matrix, use Algorithm 1.4.2 to reduce it to a row-echelon form. Then follow the following procedures to reduce it to a reduced row-echelon form. Step 5: Multiply a suitable constant to each row so that all the leading entries become 1. Step 6: Beginning with the last nonzero row and working upward, add suitable multiples of each row to the rows above to introduce zeros above the leading entries. Example 1.4.4 1. In the following we use Gaussian Elimination to reduce the augmented matrix 0 2

2 4

-4

-5

4

2

5 -4

3 3

to a row-echelon form. Step 1: The first column is the leftmost nonzero column. Step 2: Interchange the first and second rows.

U,

2 0

-4

5 4 -4

4 2 -5

3 2 3

-9 8 6

)

Step 3: Add 2 times of the first row to the third row.

u 2

0 0

4 5 2 4 3 6

3 2

-9 8

9 -12

)

applied to the submatrix Step 4: Cover the first row and begin again with Step 1 that remains. 2 4 5 3 -9 ) 0 2 4 2 0 3 6 9

u

-~2

Step 1: The third column is the leftmost nonzero column. Step 2: No' action is needed. Step 3: Add -~ times of the second row to the third row. 2 0

4 2

5 4

0

0

0

3 2 6

Step 4: The augmented matrix is already in row-echelon form.

ian Elimination

mtrix, use Alprocedures to

Section 1.4. Gaussian Elimination

2. Now we use the Gauss-Jordan Elimination to reduce the augmented matrix

ries become 1.

:able multiples ries.

0 2

2

4

2

4

5

3

-4

-5

-4

3

to a reduced row-echelon form. First, we follow the steps of the previous example to obtain a row-echelon form 2 0 0

l matrix

, the submatrix

13

Step 5: Multiply ~ and

4

5

2

4

3 2

0 0 6

-9 )

8 -24

.

i to the second and third rows respectively.

u 2

0 0

4 1

5 2

0

0

3

1 1

~9) -4

Step 6: Add -1 times of the third row to the second row.

u u u 2

0 0

4 1

5 2

3

0

0

1

0

-9 8 -4

)

3 8 -4

)

Add -3 times of the third row to the first row.

2

0 0

4 1

5 2

0 0

0

0

1

Add -4 times of the second row to the first row.

2

0

-3

0 0

1

2 0

0

0 0

1

-29 8 -4

)

The matrix is now in reduced row-echelon form.

Remark 1.4.5 1. Every matrix has a unique reduced row-echelon form but can have many different row-echelon forms. 2. In the actual implementation of the algorithms, the steps mentioned in Algorithm 1.4.2 and Algorithm 1.4.3 are usually modified to avoid the roundoff errors during the computation. (See Question 1.21 for effects of roundoff errors.)

Chapter 1. Linear Systems and Gaussian Elimination

14

Discussion 1.4.6 To solve a system of linear equations, what we need to do is to apply either Gaussian Elimination or Gauss-Jordan Elimination to reduce the augmented matrix to a row-echelon form or the reduced row-echelon form. As we have seen in Theorem 1.2. 7, the row operations do not change the solutions to the system. Thus by solving the system corresponding to the augmented matrix in a row-echelon form or the reduced row-echelon form, we can find the solutions to the original system easily. Example 1.4. 7 Let us consider the system of linear equations 2x3 + 4x4 + 2xs = 8 x 1 + 2x2 + 4x3 + 5x4 + 3xs = -9 { -2x1- 4x2- 5x3- 4x4 + 3xs = 6. The augmented matrix is

0

2

2 -4

4 -5

4 5 -4

2 3 3

We solve the system in two different ways.

Method 1: Using Gaussian Elimination as in Example 1.4.4.1, we find a row-echelon form of the augmented matrix

u

0. 2

5 4

0

0

2

0

4

3 2 6

-9 ) 8 . -24

which corresponds to the system

x 1 + 2x2 + 4x3 + 5x4 + 3xs = -9 2x 3 + 4x 4 + 2x 5 = 8 { 6xs = -24. By the third equation, x 5 = -4. Substituting x 5 = -4 into the second equation,

Substituting

X3

= 8 - 2x 4 and

X5

= -4 into the first equation,

So the given linear system has a general solution

x1

=

-29- 2s

X2

=

S

X3

= 8- 2t

X4

=t

xs

=

-4

+ 3t where s, t are arbitrary parameters.

ian Elimination

Section 1.4. Gaussian Elimination

do is to apply nented matrix [heorem 1.2.7, mg the system ;d row-echelon

Method 2: Using Gauss-Jordan Elimination as in Example 1.4.4.2, we find the reduced row-echelon form of the augmented matrix

15

u 2

0 0

0 1 0

-3 2 0

0 0 1

-29 8 -4

)

which corresponds to the system

{",+2x,

= -29

-3x4 x 3 + 2x4 X5

=

8 -4.

So the given linear system has a general solution

x1

=

-29- 2s

+ 3t

X2 = S X3

t

X4 = X5

where s, t are arbitrary parameters.

8- 2t

=

-4

=

IY-echelon form For both methods, to get a general solution, the variables corresponding to non-pivot columns are first set to be arbitrary. Then we equate the other variables accordingly. In this example, the second and fourth columns are non-pivot columns. We set x 2 = s and X4 = t, where s, t are arbitrary parameters, and equate X1, X3, X5 in terms of S and t.

Remark 1.4.8 1. A linear system is inconsistent, i.e. has no solution, if the last column of a row-echelon form of the augmented matrix is a pivot column, i.e. there is a row with nonzero last entry but zero elsewhere:

l equation,

~* 0 (9

0

* * * * *

0

@

mogeneous if it

y

y

)geneous system ution is called a

X

only the trivial solution

X

infinitely many solutions

-1), (1, 3, 3) and 2. In the three dimensional space, the two equations in the system the equation, we

a1x+b1y+c1z=O { a2x+b2y+c2z = 0,

Chapter 1. Linear Systems and Gaussian Elimination

24

where a 1, b1, c1 are not all zero and a 2, b2, c2 are not all zero, represent two planes containing the origin (i.e. the point (0, 0. 0)). The system always has non-trivial solutions, see Remark 1.5.4.2.

z

z

y

y

X

X

infinitely many solutions

infinitely many solutions Remark 1.5.4

1. A homogeneous system of linear equations has either only the trivial solution or infinitely many solutions in addition to the trivial solution. 2. A homogeneous system of linear equations with more unknowns than equations has infinitely many solutions.

Exercise 1 Question 1.1 to Question 1.15 are exercises for Sections 1.1 to 1.3. 1. Which of the following are linear equations in x 1, x 2 and x 3 ? In Parts (i)-(l), m is a

constant. (a) v3xl ---:- x2(c) x1

=

0,

= -7x2 + 3x3,

(e) y'xl- 2x2

(g)

X3

+ X3

2x1 +x2+x3 =

(i) x1 + x2 + X3 (k) 3xl

+ x2

-

= 1,

5, =

x'f{'

(b) (d) x1

+ 2x 2 + x§

(f)

+ ~X2 + ~X3

(h)

X1

= 1.

_l_+_l_+_l_ _ X1

X2

cos(m),

(j) cos(xi)

= 0,

(l) mx 1 -

X3

-

= 1.3337r, _l_

10'

+ cos(x2) + cos(x3) m 2 x 2 = 9.

=

cos(m),

1ssian Elimination

t two planes con·trivial solutions,

Exercise 1

25

2. Write down a general solution for each of the following linear equations. (a) 2x

+ 5y =

0,

(b) 8w-2x-5y+6z=-1.5.

(c) 3xl - 8x2 + 2x3 + 3.

X4 -

4x5 = 1.

(a) Find a linear equation in the variables x and y that has a general solution x = 1 + 2t and y = t where t is an arbitrary parameter. (b) Show that x = t and y = ~t - ~, where t is an arbitrary parameter. is also a general solution for the equation in Part (a).

7

4.

(a) Find a linear equation in the variables x, y and z that has a general solution

y

{

x = 3- 4s + t y=s z

where s, t are arbitrary parameters.

=t

(b) Express a general solution for the equation in Part (a) in two other different ways. (c) Write down a linear system of two different nonzero linear equations such that the system has the same general solution as in Part (a).

utions

al solution or in-

5.

(a) Give a geometrical interpretation for the linear equation x + y + z = 1. (b) Give geometrical interpretations for the linear equation x- y = 0 in (i) the xy-plane; and (ii) the xyz-space.

an equations has

(c) Give a geometrical interpretation for the solutions to the system of linear equations x+y+z=l { X -y = 0. 6. Consider the system of linear equations

{

.3.

uts (i)-(1), m is a

3x + 4y - 5z = -8 X -2y+ Z = 2.

(a) For any real number t, verify that x = i'(-4 + 3t), y = i'(-7 + 4t), z =tis a solution to the linear system. (b) Write down two particular solutions to the system. 7. Each equation in the following linear system represents a line in the xy-plane:

= cos(m),

a1x+b1y=c1 a2x + b2y = c2 { a3x + b3y = c3, where a1, a2, a3, b1, b2, b3, c1, c2, c3 are constants and for each i = 1. 2, 3. ai, b; are not both zero. Discuss the relative positions of the three lines when the system

Chapter 1. Linear Systems and Gaussian Elimination

26

(a) has no solution; ·(b) has only one solution; (c) has infinitely many solutions. 8. Each equation in the following linear system represents a plane in the xyz-space:

a1x + b1y + c1z = d1 a2x + b2y + c2z = d2 { a3x + b3y + c3z = d3, where a 1, a 2, a3, b1, b2, b3, c 1, c2, c3, d 1, d2, d3 are constants and for each i = 1, 2, 3, ai, bi, ci are not all zero. Discuss the relative positions of the three planes when the linear system (a) has no solution; (b) has only one solution; (c) has infinitely many solutions. 9.

(a) Does an inconsistent linear system with more unknowns than equations exist? (b) Does a linear system which has only one solution, but more equations than unknowns, exist? (c) Does a linear system which has only one solution, but more unknowns than equations, exist? (d) Does a linear system which has infinitely many solutions, but more equations than unknowns, exist? Justify your answers.

10. Show that the following augmented matrices are row equivalent to each other.

A=(~~~~), B=(~ ~~~), c

~

= (

1351120 ) '

D = (

~ ~ I~

) .

11. Find a series of elementary row operations that bring the augmented matrix A to the augmented matrix B where

A~u

1 1 0

1 1 1

1 1 1

)

and

B~(

3 0 0

0 6 0

0 0 9

0 0 9

)

tssian Elimination

Exercise 1

27

12. For each of the following augmented matrices, (i) determine whether the matrix is in row-echelon form, reduced row-echelon form, both, or neither: and (ii) find a system of linear equations corresponding to the augmented matrix and then solve the system (if possible). You may assume that the variables are x 1 . x 2 , x 3 , etc.

e xyz-space:

(a)

r each i = 1, 2, 3, planes when the

(c)

(e)

u~ ~ D

0 -1 0

u~ ~1 D (

~ ~ ~

0 0

0 0

0 0

-2 1

3 3

0 1

0 5

H) -1 0 0 -2 0 0 0

-7 3 1 0 0

2 2

1 0

-1 0

~

)

0 0 0 1

1')

-1

'

13. Determine whether the following augmented matrices are row equivalent to each other.

quations exist?

1uations than un~

39 0) 0 ' C=(~ ~ ~ ~) A=(~~~~) B=(~ ~ 2 4 6 0 1 1 1 0 0 0 3 0

unknowns than

t more equations

~ach

14. The following is the reduced row-echelon form of the augmented matrix of a linear system:

(~

~ ~

0 0

other.

c

:). f

where a, b, c, d. e, f are constants. Write down the necessary conditions on a. b. c. d, e, f so that the solution set of the linear system is a plane in the three dimensional space that does not contain the origin.

15. Consider the augmented matrix

:l matrix A to the

)

where a, b, c. d, e, J, g, h are constants. (a) Write down all possible reduced row-echelon forms for the augmented matrix. (b) Which of the reduced row-echelon forms in Part (a) represent inconsistent systems?

Chapter 1. Linear Systems and Gaussian Elimination

28

(c) Which of the reduced row-echelon forms in Part (a) are row equivalent to each other?

Question 1.16 to Question 1.30 are exercises for Sections 1.4 and 1.5. 16. Solve each of the following systems by Gaussian Elimination or Gauss-Jordan Elimination. +z = -1 w-x X1 + x2 +2x3 = 4 2 y 2w-x(b) -x1 + 2x2- X3 = 1 (a) 3 +3yz = -2w +3x3 = -2 2xl

(c)

(e)

{ {

x1 + 3x 2 + 3x3 + 2x4 = 1 2x1 + 6x2 + 9x3 + 5x4 = 5 =5 -x1 - 3x2 + 3x3

{

u- v+2w = 6 2u+2v- 5w = 3 2u+5v+ w = 9

(d)

(f)

{ {

x+ y+2z = 4 x- y- z = -1 2x- 4y- 5z = 1

w+5x-2y+ 2w- 2x+ y+ w -7x+3y+ 5w+ X

{

6z = 3z = 3z = 12z =

0 1 1 2

2x2 + x 3 + 2x4- X5 = 4 X2 + X4- X5 = 3 4xl + 6x2 + X3 + 4x4 - 3x5 = 8 + X4- X5 = 2 17. Solve the following system of non-linear equations: x2 {

y2 + 2z2 = 6

2x 2 + 2y 2 - 5z 2 = 3 2x 2 + 5y 2 + z 2 = 9.

18. Solve the following system of non-linear equations if 0 :::;

e < 27f and 0 :::; dJ < 27f :

cos(e) -sin(0)- tan(¢)= 0 { 3cos(8) -sin(¢) -2tan(c:p) = 0. 19. In a three-commodity market, the supply and demand for each commodity depend on the prices of the commodities. Let D 1, D2, D3 be the respective demands for products 1, 2 and 3, S 1, S 2, S 3 the respective supplies and P 1, P 2 , P3 the respective prices for the commodities. Suppose the market can be described by the linear equations S1 = -P1 +2P2 +2P3 -1, S2 = 2Pl + P2 + 2P3- 2,

S3 =

P1 + 2P2 + 3P3 - 1.

ssian Elimination

uivalent to each

Exercise 1

29

Use Gaussian Elimination or Gauss-Jordan Elimination to find the equilibrium solution (where supplies equal to demands). 20. In the downtown section of a certain city, two sets of one-way streets intersect as shown in the following:

1.5.

o;s-J ordan Elimi-

·Z

= -1 2

·Z

=

3

4 -1 1

=0 3z = 1 3z = 1 12z = 2 6z

The average hourly volume of traffic entering and leaving this section during rush hour is given in the diagram. (a) Do we have enough information to find the traffic volumes x 1 , x 2 . x 3 , x 4 ? Explain your answer. (b) Given that

. 0 ::; ¢

< 27T :

X4

= 500.

find x1,

x2, X3.

(The average hourly volume of traffic entering an intersection must be equal to the volume that leaving.) 21. Consider the two linear systems

. { x+ y=0 X+ (1 + 6)y = 1

(!)

nodity depend on ands for products ;pective prices for Lr equations -1,

-2, -1.

and

(ii) { X+ X-

y

(1

=

+ 6)y =

0 1

where 6 is a real number close to zero. (a) Solve the two linear systems for 6 = 0.0001 and 0.0002. (b) Compare the effect on the solution to the two systems when we change 6 from 0.0001 to 0.0002. (c) Explain your answer in Part (b) geometrically. (Hint: Plot the lines to see the difference between the two systems.)

----------.---..-. ..,,,,,,==

Chapter 1. Linear Systems and Gaussian Elimination

30

(This question shows that solutions of some linear systems can be largely affected by small changes in the coefficients, say, due to roundoff errors.) 22. For each of the following linear systems, determine the values of a such that the system has (i) no solution; (ii) only one solution; and (iii) infinitely many solutions.

x+ y+az = 0 x+ay+ z = 0 ax+ y+ z=O

(b)

x+ y+ z = 1 2x + ay + 2z = 2 { 4x + 4y + a 2 z = 2a

ax+ ay = 1

(c) { ax-\a\y=1 23. Determine the values of a and b so that the linear system

+ bz = 2 ax ax+ay+4z = 4 { ay+2z = b (a) has no solution: (b) has only one solution; (c) has infinitely many solutions and a general solution has one arbitrary parameter: (d) has infinitely many solutions and a general solution has two arbitrary parameters. 24. Determine the values of a, b and c so that the linear system

ax + ay + az = c by+ bz =a { cz = b (a) has no solution: (b) has only one solution; (c) has infinitely many solutions. 25. Without using pencil and paper. determine which of the following homogeneous systems have non-trivial solutions.

x-y=O x+y=O 3xl - x2 X1

6x1

(b)

+ 11x3

+ X2-

=0

+

X4

= 0

+ x2 - 4x3 + 2x4

= 0

7X3

For Part (d), a, b, c, d, e,

f

are constants.

{

x -4y+6z = 0 2y- 5z = 0 3z

=0

ax+ by+ cz = 0 dx+ ey+ fz = 0 (a+ d)x + (b + e)y + (c + f)z = 0

'sian Elimination

gely affected by

that the system utions.

31

Exercise 1

26. When propane gas burns. the propane combines with oxygen to form carbon dioxide and water: where w and x are the numbers of propane and oxygen molecules, respectively, required for the combustion: and y and z are the numbers of carbon dioxide and water molecules, respectively, produced. (a) By equating the numbers of carbon, hydrogen and oxygen atoms, respectively. on both sides of the chemical equation, write down a homogeneous system of three equations in terms of w. x, y, z. (b) Find a general solution for the homogeneous system obtained in Part (a). (c) Find the (non-trivial) solution of w, x. y. z with smallest values. w, x, y, z are positive integers.)

(Note that

27. Consider the homogeneous system of equations ax + by + cz { dx + ey + fz

trary parameter; rary parameters.

where a, b, c. d, e,

f

=0 =

0

are constants.

(a) Let x = x 0 , y = y 0 , z = z0 be a solution to the system and k is a constant. Show that X= kxo, y =kyo, Z = kzo is also a solution to the system. (b) Let x = x 0 , y = y 0 , z = z 0 and x = x 1 , y = y 1 , z = z1 be two solutions to the system. Show that x = x 0 + x 1 , y = y 0 + y 1 , z = zo + z1 is also a solution to the system. 28. Consider the homogeneous linear system

10mogeneous sys-

a1x a2x { a3x

+ b1y + c1z = 0 + b2y + c2z = 0 + b3y + c3 z = 0

where a 1 , a2. a 3 , b1 , b2 , b3 . c 1 , c2, c3 are constants. Determine all possible reduced row-echelon forms of the augmented matrix of the system and describe the geometrical meaning of the solutions obtained from various reduced row-echelon forms. cz

=

fz

= 0

0

(c+f)z=O

29. The following is the reduced row-echelon form of the augmented matrix of a linear system: b c e f 0 h

32

Chapter 1. Linear Systems and Gaussian Elimination

where a, b, c, d, e, j, g, h, k are constants. Suppose the solution set of this system is . represented by a line that passes through the origin and the point ( 1, 1, 1). Find the values of a, b. c, d, e. f. g, h, k. Justify your answers. 30. Determine which of the following statements are true. Justify your answer. (a) A homogeneous system can have a non-trivial solution. (b) A non-homogeneous system can have a trivial solution. (c) If a homogeneous system has the trivial solution, then it cannot have a non-trivial solution. (d) If a homogeneous system has a non-trivial solution, then it cannot have a trivial solution. (e) If a homogeneous system has a unique solution, then the solution has to be trivial. (f) If a homogeneous system has the trivial solution, then the solution has to be unique. (g) If a homogeneous system has a non-trivial solution, then there are infinitely many solutions to the system.

;sian Elimination

)f this system is .1.1). Find the

1swer.

Chapter 2

ave a non-trivial

Matrices

.ot have a trivial has to be trivial.

lution has to be

e infinitely many

Introduction to Matrices

Section 2.1

Definition 2.1.1 A matrix (plural matrices) is a rectangular array of numbers. The numbers in the array are called entries in the matrix. The size of a matrix is given by m x n where m is the number of rows and n is the number of columns. The (i. j)-entry of a matrix is the number which is in the ith row and jth column of the matrix. Example 2.1.2

1.

(~ ~ ) 0

is a 3 x 2 matrix.

-1

The (1,2)-entry of the matrix is 2 and the (3,1)-entry is 0. 1

0)

(~

3.1

2. (2

3.

is a 1

1

2

X

3 matrix.

-2) is a 3 x 3 matrix. ~

7f

4. (4) is a 1 x 1 matrix.

(A 1 x 1 matrix will usually be treated as a real number in computation.)

5

m,

a3

X

I matcix.

Chapter 2. Matrices

34

Definition 2.1.3 A column matrix (or a column vector) is a matrix with only one column. A TOW matrix (or a TOW vector) is a matrix with only one row. Example 2.1.4 The matrix in Example 2.1.2.5 is a column matrix and the matrix in Example 2.1.2.2 is a row matrix. The matrix in Example 2.1.2.4 is both a column and row matrix. Notation 2.1.5 In generaL an m x n matrix can be written as

A=

a,n)

a12 a22

a21

C'

a2n

.

.

aml

'

amn

am2

or simply A = (aij)mxn, where aij is the (i,j)-entry of A. Sometimes, if the size of the matrix is already known, we may just write A = ( aij).

Example 2.1.6 Write down the following matrices explicitly: 1. A= (aij)2x3 where aij

2. B

=

(bijhx2 where bij

= i + j. 1

Answers 1. A=

G 4). 3

4

5 ,

if i

+j

is even

= { -1 if i + j is odd.

2. B =

(1 -1) -1

1

1

-1

.

Definition 2.1. 7 The following are some special types of matrices: 1. A matrix is called a square matrix if it has the same number of rows and columns. In particular, an n x n square matrix is called a square matrix of order n.

= (aij) of order n, the diagonal of A is the sequence of entries a 11 , a22, ... , ann. The entries aii are called the diagonal entries while a;1 . i # j, are called non-diagonal entries. A square matrix is called a diagonal matrix if all its non-diagonal entries are zero, i.e.

2. Given a square matrix A

A= (aij)nxn is diagonal

B

aij

= 0 whenever i #

j.

3. A diagonal matrix is called a scalar matrix if all its diagonal entries are the same, i.e.

A= (aij)nxn is scalar

B

aij

={

~

if i # j if i = j

for a constant c.

~pter

2. Matrices

nly one column.

i the matrix in column and row

Section 2.1. Introduction to Matrices

35

4. A diagonal matrix is called an identity matrix if all its diagonal entries are 1. We use In to denote the identity matrix of order n. Sometimes we write I instead of In when there is no danger of confusion.

5. A matrix with all entries equal to zero is called a zero matrix. \Ve denote the m x n zero matrix by Omxn, or simply 0. 6. A square matrix (a;j) is called symmetric if

a;j = aji

for all i, j.

7. A square matrix (a;j) is called upper triangular if a;j = 0 whenever i > j; and a square matrix (a;j) is called lower triangular if a;j = 0 whenever i < j. Both upper and lower triangular matrices are called triangular matrices. Example 2.1.8 1. The following are some examples of square matrices.

if the size of the

1

(~ ~) ~1 ~ D U, ~ ~ ~ )

(4)'

(

2. The following are some examples of diagonal matrices.

4 ( )

G~) ' G~ D (f

~ ~ ~)

3. The following are some examples of scalar matrices.

and columns. In " n.

4 ( ),

G~)

(~ ! ~) (~ ~ ~ ~)

4. The following are some examples of identity matrices.

, the sequence of ~ntries while a;j, iiagonal matrix if

)·

5. The following are some examples of zero matrices.

are the same, i.e.

,nstant c.

01x1 =

(o),

02x4 =

0 (0

0 0

0 0

0) , 0

36

Chapter 2. Matrices

6. The following are some examples of symmetric matrices.

(4) '

(~ ~)'

(

~1

-1 3 2

D U,

1 3

0

-1

0

0

0

-1

0

1

6

-2)

.

7. The following are some examples of triangular matrices. The first three matrices are upper triangular while the first and the last matrices are lower triangular.

(4) '

Section 2.2

G ~).

00 00)

0 3 0 -1

-1 3 0

0 0 0

.

1

Matrix Operations

Definition 2.2.1 Two matrices are said to be equal if they have the same size and their corresponding entries are equal. That is, given A= (a;j)mxn and B = (b;j)pxq, A is equal to B if m = p. n = q and a;j = b;j for all i, j.

Example 2.2.2 Let A=

G ~),

B =

G -1) 4

and C

=

(1

2

-1 4

~)

where x is a

constant. Then 1. A

= B if and only if x = -1.

2. A -=f. C for all values of x.

Definition 2.2.3 Let A= (a;j)mxn, B = (b;j)mxn and c a real constant. \Ve define the matrices A + B, A - B and cA as follows: 1. (Matrix Addition)

A+ B = (a;j + b;j)mxn·

2. (Matrix Subtraction)

A- B

(a;j- b;j)mxn·

cA = (ca;j)mxn• where cis usually called a scalar.

3. (Scalar Multiplication)

Example 2.2.4 Let A= ( 2 4

=

3 5

4) and B 6

= ( _1

1

2 -1

~pter

2. Matrices

Section 2.2. Matrix Operations

1. A+B=( 2+1

4+(-1)

3+2 5+(-1)

4+3 ) 6+(-1) -

2. A-B=( 2-1 4- ( -1)

3-2 5-(-1)

4-3 ) 6-(-1) -

.

)

3. 4A = (

tree matrices are ~ular.

37

4 2 . 4·4

4·3 4·5

4. 4) ( 8 4. 6 16

12 20

e 3

5 4

7)5 .

5

1 6

71) .

c

16). 24

Remark 2.2.5 1. Given a matrix A, we normally use -A to denote the matrix (-1)A.

2. The matrix subtraction can be defined using the matrix addition: Given two matrices A and B of the same size. A- B is defined to be the matrix A+ (-B). Theorem 2.2.6 Let A, B. C be matrices of the same size and c, d scalars. Then 1. (Commutative Law for Matrix Addition) A+ B = B +A,

2. (Associative Law for Matrix Addition) A+ (B +C)= (A+ B)+ C. 3. c(A+B)=cA+cB.

11e size and their j)pxq, A is equal

4.

(c +d) A= cA + dA,

5. c(dA) 6. A

~)

1t.

= (cd)A = d(cA),

+0 = 0+A =

A,

7. A- A= 0 and where x is a

8. OA = 0 (in here. 0 on the LHS is the number zero and 0 on the RHS is a zero matrix). Proof Since the sizes of all the matrices are the same. to verify the rules above. we only need to show that the (i. j)-entries of the resulting matrix on LHS are equal to the (i,j)-entries of the resulting matrix on RHS. In the following, we illustrate the proof of A+ (B +C) = (A+ B) + C:

We define the

lled a scalar.

Let A= (a;j), B = (b;j) and C = (c;j)· Then for any i,j. the (i.j)-entry of A+ (B +C) = a;j + [the (i, j)-entry of B + C]

= a;j + [b;j + c;j] =

[aij

+

b;j]

+

Cij

(by the Associative Law for real number addition)

= [the (i, j)-entry of A+ B] +

Cij

=the (i,j)-entry of (A+ B)+ C. (Proofs of the other parts of the theorem are left as exercises. See Question 2.18.)

Chapter 2. Matrices

38

Remark 2.2. 7 Let A 1 , A 2 , . . . . Ak be matrices of the same size. By the Associative Law for Matrix Addition, we can write A 1 + A 2 + · · · + Ak to represent the sum of the matrices without using any parentheses to indicate the order of the matrix additions. Definition 2.2.8 (Matrix Multiplication) Let A= (aij)mxp and B = (bij)pxn be two matrices. The product AB is defined to be an m x n matrix whose (i, j)-entry is p

ail b1j

+

ai2b2j

+ ··· +

aipbpj

= 2:..:::: aikbkj k=l

for i = 1, 2, ... , m and j = L 2, .... n. (See Question 2.3 for the meaning of the notation

2:-) Example 2.2.9

1. (14

2 3) ( 5 6

~

~)

-1

= (1 . 1 + 2 . 2 + 3 . ( -1) 1 . 1 + 2. 3 + 3. ( -2)) 4·1+5·2+6·(-1)

-2 =

2u

-1

~J (!

2 5

4·1+5·3+6·(-2)

G~).

( 3 ) = 6

1·2+1·5 ) 1·3+1·6 1·1+1·4 2·3+3·6 2·2+3·5 2 ·1 + 3. 4 (-1)·1+(-2)·4 (-1). 2 +(-2). 5 (-1)·3+(-2)·6

u:

-9

7 19 -12

:. )

-15

Remark 2.2.10 L We can only multiply two matrices A and B (in the manner AB) when the number of columns of A is equal to the number of rows of B. 2. The matrix multiplication is not commutative, i.e. in general, AB and BA are two different matrices even if the products exist. For example, let A= (

and hence AB =/= B A.

~ 1 ~)

and B

=

G~).

Then

tpter 2. Matrices

Associative Law 1 of the matrices

s.

: (bij )pxn be two ·ntry is

Section 2.2. Matrix Operations

39

3. In view of Part 2 above, it would be ambiguous to say "the multiplication of a matrix A to another matrix B" since it could mean AB or BA. To distinguish the two, we refer to AB as the pre-multiplication of A to B and BA as the post-multiplication of A to B. 4. AB = 0 does not imply A = 0 or B = 0. For example, let A =

GD AB

~

G~) . DG ~) (~

and B =

(~

=

We have A

=

~) =

f.

0 and B

f.

0 while

0.

of the notation Theorem 2.2.11 1. (Associative Law for Matrix Multiplication) If A, B and C are m x p, p x q and q x n matrices respectively, then

-2)) -2)

A(BC) = (AB)C. 2. (Distributive Laws for Matrix Addition and Multiplication) If A, B 1 and B 2 are m x p, p x n and p x n matrices respectively. then

1·3+1·6 ) 2·3+3·6 -1)·3+(-2)·6

~

If A.

cl

and

c2

are p

X

n, m

X

p and m

X

p matrices respectively. then

3. If A, B are m x p, p x n matrices. respectively, and c is a scalar. then

c(AB)

=

(cA)B

=

A(cB).

4. If A is an m x n matrix, then when the number Proof To prove the matrix identities above, we need to check that and BA are two (i) the size of the resulting matrix on the LHS is equal to that on the RHS. and (ii) the (i,j)-entries of the resulting matrix on the LHS are equal to those on the RHS. In the following, we illustrate the proof of A(B1 + B 2 ) = AB1

+ AB2:

(i) Since the size of A is m x p and the size of B1 + B2 is p x n, the size of A(B1 + B2) is m x n. On the other hand, both the sizes of AB1 and AB2 are m x n and hence the size of AB 1 + AB2 is m x n.

40

Chapter 2. Matrices

Thus the sizes of the resulting matrices on both sides of the identity are the same. (ii)' Let A= (aij), B 1 = (bij) and B 2 = (b~j). For any i,j, the (i.j)-entry of A(B 1

+B 2)

+ B 2 ] + ai2[the (2, j)-entry of B 1 + B 2 ] + · · · + a;p[the (p,j)-entry of B 1 + B2]

ail [the (1, j)-entry of B 1

=

= ai1[b1j + bU + ai2[b2j + b;j] + · · · + aip[bpj + b~j] [ailblj

+ ai1b~j] + [ai2b2j + ai2b;j] + · · · + [aipbpj + aipb~j]

+ a;2b2J + · · · + aipbpJ] + [ailb~J + ai2b;j + · · · + aipb~j] [the (i, j)-entry of AB1 ] + [the (i. j)-entry of AB2] [ailblJ

= the (i,j)-entry of AB1 + AB2. By (i) and (ii), we have proved A(B1

+ B2)

=

AB1 + AB2.

(Proofs of the other parts of the theorem are left as exercises. See Question 2.19.) Definition 2.2.12 (Powers of Square Matrices) Let A be a square matrix and n a nonnegative integer. \Ve define An as follows:

if n = 0 ifn2:1.

I

A"

{

AA···A

'-v--'

n times

Example 2.2.13 Let A=

G ~).

Then

Remark 2.2.14

1. Let A be a square matrix and m. n nonnegative integers. Then Am An

= A m+n.

2. Since the matrix multiplication is not commutative, in generaL for two square matrices A and B of the same order, (AB)n may not be equal to An Bn. For example, let A = 2

A B

2

=

G~)

G~)

and B =

and hence (AB)2

(~ ~).

i= A 2 B 2 .

Then (AB)2 =

G~)

and

1pter 2. Matrices

Section 2.2. Matrix Operations

are the same.

Notation 2.2.15 Let

41

aip) is the ith row of A.

and let

blj) ie the jth column of B. ( b2 :'

bp]

(a,b,

Then

AB=

a2b1 .

arnbl

a1b2 a2b2

a,bn)

arnb2

arnbn

a2bn

(2.1)

on 2.19.) e matrix and n a

of AB. Also we can write (2.2) where Abj is the jth column of AB, or

aalB) 2B . ,

AB=

(2.3)

( arnB where aiB is the ith row of AB. (See Question 2.23 for other ways of multiplying matrices in blocks.)

nAn= Am+n.

wo square matrices

. = (2 0)

)2

0

2

Example 2.2.16 Let

and

B = (

~ -2~ )

-1

. b1 = (

~)

-1

. b2 = (

~)

-2

.

Chapter 2. Matrices

42

(a

1 Note that A= ) and B = (b 1 . a2 and (2.3) in Notation 2.2.15:

(1

(a1b1 a2b1

2

a1b2) = a2b2 (4

(Ab1

b2 ). We work out the RHS of Equations (2.1), (2.2)

Ab,)

5

3)UJ 6)UJ

~ ( G~ !) (

All three forms give us the matrix AB =

RemMk 2.2.11 Let

A~

(a,)m"w x

(1

2

=

(4

5

ue 4

.(2 1) 7

8

3)UJ

2 5

6)(i,) !)(i,)) ~ G~)

.

~ (~~)

Md b

=

(n

The system of linear equations

+ a21X1 + aux1

{

.. .

+ ·· · + a22X2 + · · · + a12X2

a1nXn

=

b1

a2nXn = b2

.. .

GmlXl + Gm2X2 + · · · + GmnXn =

can be written as a2n aln)

.. . Omn

X2 (Xl)

.. .

Xn

G ~),

bm

(bl) b2

...

bm

!apter 2. Matrices

Ltions (2.1), (2.2)

43

Section 2.2. Matrix Operations

or

Ax= b. The matrix A is called the coefficient matrix, x the variable matrix and b the constant matrix of the linear system. Furthermore, a solution x 1 = 111, x2 = 112, ... , Xn = Un to the linear system can be represented by an n x 1 matrix

3

i.e. u is said to be a solution to the linear system Ax = b if Au = b. 2

8

Let A=

(c 1

en)

where

is the jth column of A. The linear system can also

Cj

be written as

X1

(

a21 a22 au) (a12) + .

X2

.

.

0

X1C1

+ ' ' ' + Xn

b2 (a1n) (b1)

•

~1

i.e.

.

+ x2c2 + · · · + XnCn

a2n .

.

.

.

a~

~

•

~

•

n

= b or

L

XjCj

=b.

j=1

Example 2.2.18 The system of linear equations

4x + 5y + 6z = 5 x- y =2 { y- z = 3 can be written as

or

.\leo,

c~J

;,

G~~ ~J G) m x(D +u(~~) +z(~,) G)

the only eolution to the 'yetem. (Check it.)

Definition 2.2.19 Let A = (aij) be an m x n matrix. The transpose of A. denoted by A r (or At), is the n x m matrix whose (i, j)-entry is aji.

Chapter 2. Matrices

44

Example 2.2.20 Let A =

G

2

3

6

7

Remark 2.2.21

1. For a matrix A, the rows of A is the columns of AT and vice versa. 2. In Definition 2.1.7.6, a square matrix A= (aij) is called symmetric if aij = aji for all i, j. Thus a square matrix A is symmetric if and only if A = AT. Theorem 2.2.22 Let A be an m x n matrix.

2. If

B is an m

x n matrix, then

(A+ B)T =AT+ BT.

3. If cis a scalar, then (cA)T =cAT.

4. If B is ann x p matrix, then (ABf = BT AT. Proof In the following, we illustrate the proof of (AB)T = BT AT :

(i) Since the size of AB is m x p, the size of (AB)T is p x m. On the other hand. since the size of BT is p x n and AT is n x m, the size of BT AT is p x m. Thus the sizes of the resulting matrices on both sides of the identity are the same. (ii) Let A= (aij) and B = (bij)· The (i,j)-entry of AB is ailblj So by Definition 2.2.19,

+ ai2b2j + · · · + ainbnj·

the (i,j)-entry of (ABf =the (j,i)-entry of AB = ajlbli

+ aj2b2i + · · · + ajnbni·

On the other hand, let AT = (a~j) and BT = (b~j). By Definition 2.2.19, a~J = aji and b~j = bji· We have I th e (2,. J") -ent ry o f BTAT = b1il alj

+ bi2a2j + . . . + binanj 1

I

1

I

+ b2iaj2 + · · · + bniajn = ajlhi + aj2b2i + · · · + ajnbni· =

hiajl

Hence the (i,j)-entry of (AB)T =the (i,j)-entry of BT AT. By (i) and (ii), we have proved (AB)T = BT AT. (Proofs of the other parts of the theorem are left as exercises. See Question 2.20.)

apter 2. Matrices

Section 2.3. Inverses of Square Matrices

Section 2.3

45

Inverses of Square Matrices

Discussion 2.3.1 Let a, b be two real numbers such that a =J 0. Then the solution to the equation ax = b is x = ~ = a- 1 b. Now, let A and B be two matrices. It is much harder to solve the matrix equation AX = B because we do not have "division'' for matrices. However, for some square matrices, we can find their "inverses" which have the similar property as a- 1 in the computation of the solution to ax = b above. f

aij

=

aji

for all

Definition 2.3.2 Let A be a square matrix of order n. Then A is said to be invertible if there exists a square matrix B of order n such that AB =I and BA =I. Such a matrix B is called an inverse of A. A square matrix is called singular if it has no inverse. Example 2.3.3 1. Let A= (

2 -1

-5) 3

(3 5) = ( 2 -5) (3 5) = (1 0)

and B =

AB and other hand, since Thus the sizes of

BA =

1

-1

2

. Then

3

1

2

0

1

=I

(3 5) ( 2 -5) = (1 0) = 1

2

-1

3

0

1

I.

So A is invertible and B is an inverse of A.

2. Consider the matrix equation

By pre-multiplying the matrix 2.2.19,

(~

D,

which is an inverse of

( 2 -5) _

1

3

. to both

sides of the equation, we have

3. Show that A=

C~)

is singular.

Solution (Proof by Contradiction) Suppose on the contrary A has an inverse B = ( ac

db).

By the definition of inverses, we have

ion 2.20.)

BA=I=

G~).

······----·-

---------

46

Chapter 2. Matrices

On the other hand,

Thus

G ~)

= (::

~ ~)

which is impossible.

(In Section 2.4, we shall see a systematic method to check whether a square matrix is invertible. See Remark 2.4.10.)

Remark 2.3.4 1. (Cancellation Law for Matrices) Let A be an invertible m x m matrix. (a) If B 1 and B 2 are m x n matrices with AB1 = AB2, then B1 = B2. (b) If cl and c2 are n

m matrices with ClA = C2A, then cl = C2.

X

2. If the matrix A in Part 1 is singular, then the Cancellation Law may not hold. For example, take A

AB2

=

(~ ~)

such that C 1 A

=

but Bl

G~), #

=

B1

G~)

and B2

=

B2. (As an exercise, find two 2

G~). X

Then AB1

2 matrices cl and c2

= C 2 A but C1 # 02.)

Theorem 2.3.5 (Uniqueness of Inverses) If B and Care inverses of a square matrix A, then B =C. Proof Since B is an inverse of A, we have

BA =I

and

AB =I.

and

AC=I.

Also, since Cis an inverse of A, we have

CA=I So AB

=I

=?

CAB = C I

=?

IB = C

=?

B = C.

Notation 2.3.6 Let A be an invertible matrix. By Theorem 2.3.5, we know that there is only one inverse of A and we use the symbol A - 1 to denote this unique inverse of A. Remark 2.3. 7 If we are asked to show that A - 1 = B where A and Bare square matrices of the same size, what we need to do is to check that AB =I and BA =I. Actually. in the next section, we shall learn that we only need to check any one of the two conditions. (See Theorem 2.4.12.)

···~-~~:!5

-~----.-.--

---~=-===--=-~

~apter

2. Matrices

Section 2.3. Inverses of Square lviatrices

Example 2.3.8 Let A

= (~

~).

A-1=

If ad- be -1- 0, show that A is invertible and

1

ad- be

't

square matrix is

Solution Let B

=

(d-e -b) = (ad~be a -=-"--

(ad~ebe ad~bc). ad-be

47

ad-be

-b ) ad~be

.

ad-be

Note that B is well-defined if ad- be -1- 0. It suffices

ad-be

to show that B is the inverse of A. By Remark 2.3.7, we need to check that AB =I and

BA=I:

rz matrix.

AB= BA =

ay not hold.

cl

and

b) (-=-"-ad~be d

-b ) ad-be a ad-be

ad-be

(ad~be ad-~bc) -e ad-be

G b) d

= (ad-be ad-be ed-de ad-be

-

ad-be

(

ad-be da-bc -ca+ac ad-be

-ab+ba) ad-be -cb+da ad-be -db+bd) ad-be _ -cb+ad ad-be

(1 0) =I (1 0) =I 0

0

1

1

'

.

Thus A is invertible and A - 1 = B.

:) . Then AB1

l.trices

(~

c2

Theorem 2.3.9 Let A, B be two invertible matrices of the same size and e a nonzero scalar. Then 1. eA is invertible and (eA)- 1 = ~A - 1 .

of a square matrix

2. AT is invertible and (AT)- 1 = (A- 1)T. 3. A- 1 is invertible and (A- 1 ) - 1 =A. 4. AB is invertible and (AB)- 1 = B- 1 A - 1 .

Proof In the following, we only illustrate the proof of Part 2 of the theorem: To show that AT is invertible, we only need to verify that (A - 1 ) T is the inverse of AT. Note that know that there is inverse of A.

are square matrices l = I. Actually, in the two conditions.

By Remark 2.3.7, we have proved that AT is invertible and (AT)- 1 = (A- 1)T. (Proofs of the other parts of the theorem are left as exercises. See Question 2.30.)

Remark 2.3.10 By Theorem 2.3.9.4, if A 1 , A 2 , ... , Ak are invertible matrices of the 1 1 same size, then A1A2 · · · Ak is invertible and (A 1A 2 · · · Ak)- 1 = Ak - 1 · · · A 2- A 1- .

Chapter 2. Matrices

48

Definition 2.3.11 (Negative Powers of Square Matrices) matrix and n a positive integer. We define A-n as follows:

Let A be an invertible

A-n= (A-1)n = A-1A-1. ··A-1. n times Example 2.3.12 Let A=

(1 2) . 1

3

By Example 2.3.8. A- 1 = .

- 2) ( 3 1 -1

- 2) ( 3 1 -1

(

3 -2) -30).

-1

1

- 2) = ( 41 1 -15

.

Then

11

Remark 2.3.13 Let A be an invertible matrix. 1. Ar A 8 = Ar+s for any integers r and s. 2. An is invertible and (An)- 1 =A-n for any integer n.

Elementary Matrices

Section 2.4

Definition 2.4.1 In Definition 1.2.4, Definition 1.2.6, Definition 1.3.1. Definition 1.4.1. Algorithm 1.4.2 and Algorithm 1.4.3, the concepts of elementary row operations, row equivalent matrices, row-echelon forms, reduced row-echelon forms, Gaussian Elimination. GaussJordan Elimination are defined for augmented matrices. Form now on, these terms will also be used for matrices. Discussion 2.4.2 Consider the three types of elementary row operations on matrices. 1. Multiply a row by a constant: 0 -1 4

~ ~) and B = 4

0

(! ~ 1

2 4

2 6 4

~). Note that B ie obtained fcom

A by multiplying the second row of A by 2.

Let E,

~ G~ E,A

D.

Oh"Bcve that

~ (H DG~[

H) G

0 -2 4

2 6 4

~~)~B.

apter 2. Matrices

49

Section 2.4. Elementary Matrices

be an invertible In general, let A be an m x n matrix and 1

0 ~).

1

E=

Then

k

011 ( 1 0 1 z Elimination 0 0 2

X y

)

.

z-x+y

the system is consistent regardless of the values of x, y. z. So we have shown that span{(1,0,1), (1,1,0), (0,1.1)} =IR 3 . (Explicitly, if we solve the system, we get a = ~(x- y + z), b = ~(x + y- z) and c= ~(-x+y+z).)

5. Show that span{ (L 1, 1), (L 2, 0). (2, 1, 3), (2, 3, 1)} # JR 3 . Solution We follow the method used in Part solve the vector equation a(L

1,

1,

For any vector (x. y, z) in IR 3 , we (3.1)

1) + b(1, 2, 0) + c(2, 1, 3) + d(2, 3, 1) = (x, y, z)

where a, b. c, d are variables. The linear system is a + b + 2c + 2d = x a+ 2b + c +3d = y { a +3c + d = z. Since 1 1 ( 1

1 2 0

2 1 3

2 3 1

x ) y z

Gaussian

( 1 0 Elimination 0 ---->

the system is inconsistent if y + z- 2x

#

1 1 0

2 -1 0

2 1 0

x y-x y + z - 2x

)

0.

For example, if (x, y, z) = (1, 0, 0) where y+z-2x = -2 # 0, then Equation (3.1) does not have a solution and hence (L 0, 0) rf_ span{ (L 1, 1), (L 2. 0), (2. 1, 3), (2, 3, 1)}. So span{(1, 1, 1), (1. 2, 0), (2, 1, 3), (2, 3, 1)}

#

JR 3 .

· 3. Vector Spaces

Section 3.2. Linear Combinations and Linear Spans

91

Discussion 3.2.5 In the following, we present a method to determine whether a set of vectors spans the whole ~n:

to verify that for

U), (1, 1, 0) and

LetS= {U1, u2, ... , uk} doee nol have

'. Vector Spaces

a vector space

Section 3.5. Bases

105

2. Let V = span{(1, 1, 1, 1), (1, -1, -1, 1), (1, 0, 0.1)} and S = {(1, L L 1), (L -1, Show that S is a basis for V.

-1,

1)}.

Solution (a) The equation (0, 1, 1)}.

c1(L L L 1)

+ c2(L -L -L 1) =

(0, 0, 0, 0)

gives us a system of linear equations

c1 + c2 = 0 c1- c2 = 0 { c1- c2 = 0 C1 + C2 = 0

is a subspace of not a subspace

3

nall as possible, h a set can then

which has only the trivial solution (check it). So Sis linearly independent. (b) Since (1, 0, 0, 1) = ~(1, L

1,

1) + ~(L -1, -1, 1), by Theorem 3.2.12, span(S) = V.

By (a) and (b), Sis a basis for V.

ce V. Then S is

3. IsS= {(1, L L 1), (0, 0, 1, 2), ( -L 0, 0, 1)} a basis for IR 4 ?

Solution By Theorem 3.2.7. span(S)

cJ IR 4 .

So Sis not a basis for IR 4 .

4. Let V = span(S) where S = {(L 1, 1). (0, 0, 1), (L L 0)}. IsS a basis for V?

Solution Note that Sis linearly dependent (check it). So Sis not a basis for V. Remark 3.5.6 1,

A basis for a vector space V contains the smallest possible number of vectors that can span V.

2. For convenience. we s~y that the empty set,

0, is the basis for the zero space.

3. Except the zero space, any vector space has infinitely many different bases. Theorem 3.5. 7 If S = {u 1 , u 2 , .... uk} is a basis for a vector space V, then every vector v in V can be expressed in the form independent.

,ich does not have

in exactly one way, where c1 , c 2 ,

... , Ck

are real numbers.

Proof Since S spans V, every vector v can be expressed in the form v = c1 u 1 · · · + ckuk. Suppose that a vector v can be expressed in two ways

+ c2 u 2 +

..

.--~~'"''"''~'~:7-.

·-~~"··-·

-·- -· ''"""'

Chapter 3. Vector Spaces

106

where c 1 , c2, ... , Ck, d 1 , d 2 , ... , dk are real numbers. Subtracting the second equation to th~ first equation, we obtain

Since S is linearly independent, the only possible solution is

i.e. c1 = d1, c2 = d2, ... , Ck = dk. So the expression is unique. Definition 3.5.8 LetS= {u 1 , u 2 , ... , uk} be a basis for a vector space V and v a vector in V. By Theorem 3.5.7, v is expressed uniquely as a linear combination

The coefficients c1 , c2 ,

...•

ck are called the coordinates of v relative to the basis S.

The vector is called the coordinate vector of v relative to the basis S. (Here we assume the vectors of S are in a fixed order so that u 1 is the first vector, u 2 is the second vector, etc.) Example 3.5.9 1. LetS= {(1, 2, 1), (2, 9, 0), (3, 3, 4)}. It is easy to check that Sis a basis for JR

3

.

(a) Find the coordinate vector of v = (5, -1, 9) relative to S. (b) Find a vector win JR 3 such that (w)s = ( -1, 3, 2). Solution (a) Solving the equation a(1, 2, 1)

+ b(2, 9, 0) + c(3, 3, 4) =

(5, -1, 9),

we obtain only one solution a= 1, b = -1 and c = 2, i.e.

v = (1, 2, 1)- (2, 9, 0) + 2(3, 3, 4) and (v)s = (1. -1, 2). (b) w = -(1, 2, 1)

+ 3(2, 9, 0) + 2(3, 3, 4)

=

(11, 31, 7).

2. Let v = (2, 3) E JR 2 . Find the coordinate vector of v relative to each of the following bases for JR 2 .

3. Vector Spaces

Section 3.5. Bases

ond equation to

(a) (b)

107

sl = {(1,0), (0,1)}; s2 = {(1, -1), (1, 1)}:

(c) S3 = {(L 0), (1, 1)}. Solution (a) Since v = (2, 3) (v)s, = (2, 3).

2(1, 0)

+ 3(0, 1),

v

/ (0, 1)

V and v a vector

=

(2. 3)

I

I

v

(L 0)

Le basis S.

(b) Since v = (2, 3) = -~(1, -1) 1 (v)s2 = ( -2, ~).

+ ~(1, 1),

-(1, 0)

+ 3(1, 1),

ne the vectors of tr, etc.)

~h

(c) Since v = (2, 3) (v)s3 = (-1,3).

(L 0)

3. (Standard Basis for lRn) Let E = {e1,e 2 , .... en} where e1 = (LO ..... O), e2 = (0, 1, 0, ... , 0), .... en = (0, ... , 0, 1) are vectors in lRn. By Remark 3.3.2.2. E spans lRn. Also, it is easy to show that E is linearly independent. Thus E is a basis for lRn which is called the standard basis for lRn. For any u = ( u 1 , u 2 , ... , Un) E lRn,

(u)E of the following

=

(u1, u2, ... ,

Un) =

u.

Remark 3.5.10 The following are some useful rules in working with coordinate vectors: Let S be a basis for a vector space V.

108

Chapter 3. Vector Spaces

1. For any u, v E V, u = v if and only if (u)s = (v)s. ·2. For any

VI,

v2, ... , Vr E V and

C1,

c2, ... , Cr E JR.,

Theorem 3.5.11 LetS be a basis for a vector space V where lSI = k. Let be vectors in V. Then 1.

VI.

v2, ... , Vr

v2, ... , Vr are linearly dependent (respectively, independent) vectors in V if and only if (vi)s, (v2)s, ... , (vr)s are linearly dependent (respectively, independent) vectors in JR. k; and VI,

2. span {VI, v2, ... , Vr} = V if and only if span { (VI) s, (V2) s, ... , (Vr) s} = JR. k.

Proof 1. By Remark 3.5.10,

c1 VI

B B

+ c2v2 + · · · + CrVr = 0 (c1vi + c2v2 + · · · + Crvr)s = (O)s cl(vi)s + c2(v2)s + · · · + cr(vr)s = (O)s,

where (O)s = (0, ... ,0) is the zero vector in JR.k. The equation c 1vi + c2v 2 + · · · + Cr Vr = 0 has non-trivial solution if and only if the equation c1 (VI) s + c2 (V2) s + · · · + cr(vr)s = (O)s has non-trivial solution. Hence VI, v 2 , ... , Vr are linearly dependent (respectively, independent) vectors in V if and only if (VI) s, (v2) s, ... , (Vr) s are linearly dependent (respectively, independent) vectors in JR. k. 2. LetS= {ui, u2, .... uk}· (=?)Suppose span{vi, v2, ... , vr} = V. Take any vector (a 1.a 2, ... ,ak) E JR.k. Let w = a1ui + a2u2 + · + akuk E V. Since VI, v 2 , ... , Vr span V, there exist real numbers c1, c2, ... , Cr such that w = c1 VI+ c2v2 + · · · + CrVr. By Remark 3.5.10. cl(vi)s

+ c2(v2)s + · · · + cr(vr)s

= (clvi =

(w)s

+ c2v2 + · · · + CrVr)s =

Since every vector in JR. k is a linear combination of have span{(vi)s, (v2)s, ... , (vr)s} = JR.k.

(a1, a2, .... ak). (VI) s.

(v2) s, ... , (Vr) s. we

(~) Suppose span{(vi)s, (v 2 )s, ... , (vr)s} = JR.k. Take any vector wE V. Since

(w)s is a vector in JR.k and (vi)s, (v2)s, ... , (vr)s span JR.k, there exist real numbers c 1, c2, ... , Cr such that (w)s = cl(vi)s + c2(v2)s + · · · + cr(vr)s = (clvi

+ c2v2 + · · · + CrVr)s

3. Vector Spaces

109

Section 3.6. Dimensions

and hence w = c1 VI + c2v2 + · · · + CrVr. Since every vector in V is a linear . combination of VI, v2, ... , Vr, we have span {VI, v2, ... , Vr} = V.

~tors in

V if and

Section 3.6

Dimensions

1dependent) vee-

Theorem 3.6.1 Let V be a vector space which has a basis with k vectors. Then 1. any subset of V with more than k vectors is always linearly dependent: 2. any subset of V with less than k vectors cannot span V.

Proof Let S be a basis for V and

'I+ C2V2 + ... + + c2(v2)s + · · · +

nearly dependent Is, ... , (vr)s are

. . , ak)

E

JEtk. Let

/, there exist real 3y Remark 3.5.10,

·+CrVr)s

lSI

= k.

1. LetT= {vi, v 2 , ... , vr} be a subset of V with r > k. Since (vi)s, (v2)s, ... , (vr)s are vectors in JEtk, by Theorem 3.4.7, (vi)s, (v 2 ) 5 , ... , (vr)s are linearly dependent. Thus by Theorem 3.5.ll.l. Tis linearly dependent.

2. LetT= {vi, v 2, ... , vr} be a subset of V with r < k. By Theorem 3.2.7, (vi)s. (v 2 ) 5 , ... , (vr)s cannot span JEtk. Thus by Theorem 3.5.11.2, T cannot span V. Remark 3.6.2 By Theorem 3.6.1, all bases for a vector space have the same number of vectors. This number gives us a way to measure the '·size" of a vector space . Definition 3.6.3 The dimension of a vector space V, denoted by dim(V), is defined to be the number of vectors in a basis for V. In addition, we define the dimension of the zero space to be zero .

. , ak)· )s, ... , (vr)s, we

tor w E V. Since k. there exist real

Example 3.6.4

2. Except { 0} and JR. 2, all subs paces of JR 2 are lines through the origin and they are of dimension 1.

3. Except { 0} and JR 3 , all subspaces of JR 3 are either lines through the origin, which are of dimension 1, or planes containing the origin, which are of dimension 2.

110

Chapter 3. Vector Spaces

4. Find a basis for and determine the dimension of the subspace W = { (x, y, z) I y = z } of IR 3 . Solution Every vector in W is of the form

(x, y, y) = x(1, 0, 0) + y(O, L 1). SoW= span{(1,0,0), (0, 1, 1)}. It is easy to check that (1,0,0), (0, 1, 1) are linearly independent. Hence {(1, 0, 0), (0, 1, 1)} is a basis for Wand dim(W) = 2. Discussion 3.6.5 Given a homogeneous linear system Ax = 0, we want to find a basis for and determine the dimension of the solution space.

First, use Gauss-Jordan Elimination to convert the augmented matrix to the reduced rowechelon form. Then set the variables corresponding to non-pivot columns as arbitrary and equate the other variables accordingly. Finally, write a general solution for the system in the form of t1u 1 + t2u2 + · · · + tkuk where t1, t2, ... , tk are arbitrary parameters and u 1 , u 2 , ... , uk are fixed vectors. In this way, the vectors u1, u2, ... , uk are always linearly independent and hence {u 1 , u 2 , ... , uk} is a basis for the solution space. Let R be a row-echelon form of A. Then the number of arbitrary parameters needed in the general solution is equal to the number of non-pivot columns in R. So the dimension of the solution space of the system is equal to the number of non-pivot columns in R. (See also Theorem 4.3.4.) Example 3.6.6 Find a basis for and determine the dimension of the solution space of the homogeneous system 2v + 2w - x +z = 0 -v - w + 2x - 3y + z = 0 { v

x+ y+z=O - z = 0.

+ w- 2x

Solution Since

(

~1

2 -1

0 1

-1 2 1 -2

0

1

-3

1 1 -1

1 0

~ ) Gam~n!an ( O

Elimination

1

0 0 O 0

:

0 1 0 0

0 0 1 0

1 1 0 0

0 0 0 0

)

the linear system has a general solution

X

-s- t s -t

y

0

z

t

v w

-1 1

=s

0 0 0

+t

-1 0 -1 0 1

where s, t are arbitrary parameters.

· 3. Vector Spaces

(x,y,z) I y = z}

Section 3. 6. Dimensions

111

Then { ( -1, 1, 0, 0, 0), ( -1, 0, -1, 0. 1)} is a basis for the solution space and the dimension of the solution space is 2. Theorem 3. 6. 7 Let V be a vector space of dimension k and S a subset of V. The following are equivalent:

. 1, 1) are linearly

)=

2.

nt to find a basis

the reduced row; as arbitrary and m for the system y parameters and ue always linearly

1. S is a basis for V . 2. S is linearly independent and lSI

= k.

3. S spans V and lSI = k.

Proof "1

=}

2" and "1

=}

3" follows from Remark 3.6.2 directly.

2 ::::} 1: Suppose S is linearly independent and lSI = k. Assume that S is not a basis for V, i.e. span(S) =/= V. Take a vector u in V but not in span(S). By Theorem 3.4.10, S' = S U { u} is a set of k + 1 linearly independent vectors. But this contradicts Theorem 3.6.1.

ters needed in the 2 dimension of the 1s in R. (See also

3 ::::} 1: SupposeS spans V and lSI = k. Assume that Sis not a basis for V, i.e. Sis linearly dependent. Take a vector v in S which is a linear combination of other vectors in S. By Theorem 3.2.12, S" = S- {v} is a set of k -1 vectors that spans V. But this also contradicts Theorem 3.6.1.

lution space of the

Example 3.6.8 Show that u 1 JR3.

1·

= (2, 0, -1), u 2 = (4, 0, 7), u 3 = (-1.1, 4) form a basis for

Solution Consider the vector equation

i.e. c 1 (2, 0, -1)

+ c2( 4, 0, 7) + c 3 ( -1. 1, 4) = (0, 0, 0). This implies

On solving the system, we get c 1 = 0, c 2 = 0 and c 3 = 0. So u 1 , u 2 , u 3 are linearly independent. As dim(JR 3 ) = 3, by Theorem 3.6.7, {u 1 , u 2 , u 3 } is a basis for JR 3 .

parameters.

Theorem 3.6.9 Let U be a subspace of a vector space V. Furthermore, if U =/= V, then dim(U) < dim(V).

Then dim(U) :::; dim(V).

Proof Let S be a basis for U. Since U r;;; V, Sis a linearly independent subset of V. By Theorem 3.6.1.1, dim(U) = lSI :::; dim(V).

112

Chapter 3. Vector Spaces

Assume dim(U) = dim(V). As S is linearly independent and lSI = dim(U) = dim(V), by Theorem 3.6.7, S is a basis for V. But then U = span(S) = V. Hence if U of= V, then dim(U) < dim(V). Example 3.6.10 Let V be a plane in JR3 containing the origin. Note that V is a vector space of dimension 2, see Example 3.6.4.3. Suppose U is a subspace of V such that U of= V. By Theorem 3.6.9, dim(U) < 2. Hence U is either {(0, 0, 0)} (which is of dimension 0) or a line through the origin (which is of dimension 1).

v

v

u

u

•the

origin

the origin

Theorem 3.6.11 (This theorem is a continuation of Theorem 2.4.7 and forms part of our main theorem on invertible matrices, see Theorem 6.1.8.) Let A be ann x n matrices. The following statements are equivalent. 1. A is invertible. 2. The linear system Ax

=

0 has only the trivial solution.

3. The reduced row-echelon form of A is an identity matrix. 4. A can be expressed as a product of elementary matrices. 5. det(A) of= 0. 6. The rows of A form a basis for JRn. 7. The columns of A form a basis for JRn. Proof By Theorem 2.4.7, statements 1 to 4 are equivalent. By Theorem 2.5.19. we have "1 '¢=} 5''. The rows of A are the columns of AT. Since A is invertible if and only if AT is invertible (see Theorem 2.3.9), we only need to show either "1 {o} 6" or "1 {o} 7". In the following, we prove "1 Let A= (a 1 • a 2 ,

... ,

{o}

7":

an) where ai is the ith column of A.

3. Vector Spaces

113

Section 3. 7. Transition Matrices

') = dim(V), by if U

#

V, then

(by Theorem 3.6.7) 9

a row echelon form of A has no zero row

(by Discussion 3.2.5)

at V is a vector

9

A is invertible.

(by Remark 2.4.10)

1(U) < 2. Hence

Example 3.6.12

igin (which is of

1. Let u 1

= (1,1,1), u 2 = (-L1.2) and u 3 = (1,0,1). Since -1 1 2

1 1 1

2. Let u 1

= (1, 1.1, 1),

u2

= (1. -1, 1, -1), u 3 = (0, 1, -1. 0) and u 4 = (2, 1.1. 0). Since 1 1 0 2

forms part of our x n matrices. The

1 1 -1 1

1 -1 1 1

{u1, u2, u3, u 4 } is not a basis for llt4

Section 3. 7

1 0 = 3, 1

1 -1 = 0. 0 0

o

Transition Matrices

Notation 3. 7.1 Let S = { u 1 , u 2 , uk} be a basis for a vector space V and let v be a vector in V. Recall that if v = c1u 1 + c2u2 + + ckuk, then the vector 0

0

0

,

0

em 2.5.19, we have

; if AT is invertible

0

0

is called the coordinate.vector of v relative to So Sometimes, it is more convenient to \Vrite the coordinate vector in the form of a column vector. Thus we define

[v]s~ CJ and also call it the coordinate vector of v relative to So

114

Chapter 3. Vector Spaces

Discussion 3.7.2 LetS= {u 1 , u 2 , ... , uk} and T = {v 1 , v 2 , ... , vk} be two bases for a vector space V. For any vector w in V, we want to study the relation between [w]s and [w]r.

[w],~(D. Since Tis a basis for V, we can write each ui as a linear combination of v 1 , v 2 , u1 = au v1 u2 = a12V1

.... vk,

+ a21 v2 + · · · + akl Vk, + a22V2 + · · · + ak2Vk,

where au, a 12 , ... , akk are real constants, i.e. a21 a22 au) (a12) : . [u2Jr = : ,

[ul]T =

[uk]T =

a2k (alk)

:

( akl

ak2

akk

Then

+ C2U2 + · · · + CkUk = c1 (au v1 + a21 v2 + · · · + akl vk) + c2 (a12v1 + a22v2 + · · · + ak2vk) + · · · + ck(alkvl + a2kv2 + · · · + akkvk) = (c1au + c2a12 + · · · + ckalk)vl + (c1a21 + c2a22 + · · · + cka2k)v2 + · · · + (c1ak1 + c2ak2 + · · · + ckakk)vk

W =

C1 U1

i.e.

can

· c1a21 +c,a, + c2a22 [w]r=

c1ak1

+ + ... ···+ + ''"") cka2k .

+ c2ak2 + · · · + ckakk

a12 a21

a22

ak1

ak2

(""

= ([ul]T

[u2]r

[uk]T). Then

[w]r = P[w]s

for all wE V.

")n .. .

c2 .. .

akk

ck

a2k

[uk]T) [w]s.

say

oter 3. Vector Spaces

Section 3. 7. Transition Matrices

1k} be two bases for n between [w]s and

Definition 3.7.3 LetS= {u 1 , u 2 , ... , uk} and T be two bases for a vector space. The square m.atrix P = ([u 1]r [u 2]r [uk]T) in Discussion 3.7.2 is called the transition matrix from S to T.

115

Example 3.7.4 1. Consider the following two bases S and T for JR 3:

S = {u 1 , u 2 , u 3 }, where u 1 = (1,0, -1), u2 = (0, -1,0) and

U3

= (L0,2), and

T = {v 1 , v 2 , v 3}, where v 1 = (1. L 1), v2 = (1, 1,0) and v3 = (-1,0,0). (a) Find the transition matrix from S toT. (b) Let w be a vector in JR 3 such that (w)s = (2, -1, 2). Find (w)r. Solution (a) First, we need to find an, a2 1, ... , a33 such that

~.:)

u1 =an v1 u2 = a12v1 U3 = a13V1

.

·k

+ a21 v2 + a31 v3, + a22v2 + a32V3, + a23V2 + a33V3.

By 1 1 1

(

1 1 0

-1 0 0

1 0 -1

1 0 2

0 -1 0

we have

)

Gauss-Jordan ----+

Elimination

1 0 0 1 0 0

(

0 0 1

-1 1 -1

0 -1 -1

2 -2 -1

)

u1 = -v1 + v2 - V3, V2- V3, U2= u3 = 2vl - 2v2 - v3.

(See Example 3.2.11 for the method of finding the linear combinations.)

)C)

1

SoP=

(~ ~1 ~2) -1

-1

is the transition matrix from S toT.

-1

(~ ~1 ~2) (~1) (~1). 1

(b) Since [w]r = P[w]s =

-1

-1

-1

2

-3

(w)r = (2, -1. -3).

Chapter 3. Vector Spaces

116

2. LetS= {ui, u 2 }, where ui = (1, 1) and u 2 = (L -1), and letT= {vi, v2}. where VI= (1,0) and V2 = (L 1). Note that SandT are two bases for ]!{2 . It is easy to check that

ui = v2, u2 = 2vi- v2. Thus P = (

0 1

2 ) is the transition matrix from S toT. -1

Also, it is easy to get 1

Thus Q =

(l 1) ~

VI= 2 ui v 2 = ui

1 + 2U2,

is the transition matrix from T to S. Note that Q = p-

1

.

0

Theorem 3.7.5 Let SandT be two bases of a vector space and let P be the transition matrix from S to T. Then

1. P is invertible: and 2. P- 1 is the transition matrix from T to S. Proof Let Q be the transition matrix from T to S. By Theorem 2.4.12, it suffices to show

that QP =I. SupposeS= {ui, u 2 , .... uk}· Note that 1 0 0

0 0

0 1 0

0 1

0

0

Thus for i = 1, 2, ... , k, the ith column of QP = QP[ui]s = Q[ui]T = [ui]s.

Example 3.7.6 In Example 3.7.4.1. the transition matrix from S toT is

(~ ~1 ~2) 1

p

=

-1

-1

-1

3. Vector Spaces

)e the transition

Exercise 3

117

So the transition matrix from T to S is

p-1

-1 1

=

~2)-1

0 -1 -1

( -1

-1

Note that =

[w]s.

Exercise 3 Question 3.1 to Question 3.14 are exercises for Sections 3.1 and 3.2. 1. Find the vectors u, v, u below.

+ v and 3u- 2v in ]!{ 2 where u and v are shown in the figure

· suffices to show

y 2

u

/

/

I

I

-2

6

\

v

X I

\

' '

/ /

-2

2. Express each of the following by the set notation in both implicit and explicit form: (a) the line in (b) the plan in (c) the line in

]!{ 2 ]!{ 3 ]!{ 3

passing through the points ( 1, 2) and ( 2, -1). containing the points (0, 1, -1), (1. -1, 0) and (0, 2, 0). passing through the points (0, 1, -1) and (1. -1, 0).

Chapter 3. Vector Spaces

118

3. Consider the following subsets of JR;. 3 :

A= a line passes through the origin and (9, 9, 9) B

= { (k, k, k) IkE JR;, },

C = { (x1, x2, x3) I x1 = x2 = X3 }, D

= { (x, y, z)

I

2x - y - z

= 0 }.

E={(a,b,c)l2a-b-c=O and a+b+c=O}, F={(u,v,w)l2u-v-w=O and 3u-2v-w=O}. Which of these subsets are the same? 4. Let U, V, W be three planes in JR;, 3 where U = { (x, y, z)

W

= {

I

2x- y

+ 3z =

(x, y, z) I x- 3y- 2z

0 }, V = { (x, y, z)

I

3x

+ 2y- z =

0},

= 1 }.

(a) Determine which of U, V, W contain the origin. (b) Write down the sets U n V and V n W explicitly. 5. Let A

= { (1 + t, 1 + 2t, 1 + 3t) I t E JR;,} be a subset of JR;, 3 .

(a) Describe A geometrically. (b) Show that A = { ( x, y, z) I x

+y -

z = 1 and x - 2y

+z

= 0 }.

(c) Write down a matrix equation M x = b where M is a 3 x 3 matrix and b is a 3 x 1 matrix such that its solution set is A. 6. Determine whether the following subsets of JR;. 4 are equal to each other.

s= T

{ (p,q,p,q) I p,q E JR;,},

= { (x, y, z, w) I x + y- z- w = 0 }, 1 0

0 1

1 0

0 } 1 =0 .

1 0 0 1 b c d

a

Briefly explain why one subset is equal (or not equal) to another subset. 7. Let P represent a plane in JR;. 3 with equation x- y + z = 1 and A, B, C represent three different lines given by the following set notation:

A= {(a, a, 1) I a E JR;, },

B = { (b, 0, 0) I bE JR;, },

C = { (c, 0, -c) I c E JR;, }.

r 3. Vector Spaces

Exercise 3

119

(a) Express the plane Pin explicit set notation. (b)· Does any of the three lines above lie completely on the plane P? Briefly explain your answer. (c) Find all the points of intersection of the line B with the plane P. (d) Find the equation of another plane that is parallel to (but not overlapping) the plane P, and contains exactly one of the three lines above. 0 }.

(e) Can you find a nonzero linear system whose solution set contains all the three lines? Justify your answer. 8. Let u1 = (2, 1, 0, 3), u 2 = (3, -1, 5, 2), and u 3 = ( -1, 0, 2, 1). Which of the following vectors are linear combinations of u 1 , u 2 , U3?

- z = 0 },

(a) (2,3,-7,3),

(b) (0, 0, 0, 0),

(c) (1, 1, 1, 1),

(d) (-4,6,-13,4).

9. For each of the following sets, determine whether the set spans 1Et 3 . (a)

sl

= {(1, 1, -1), ( -2, 2, 1)}.

(b) 52= {(1,1,-1), (-2,-2,2)}. (c) S 3 = {(1, 1, -1), ( -2, 2, 1), (L 5, -2)}. (d) 54={(1,1,-1), (-2,2,1), (4,0,3)}.

, matrix and b is a

,ther.

(e)

s5 =

(f)

s6

{(1, 1, -1), ( -2, 2, 1), (1, 5, -2) (0, 8, -2)}.

= {(1,1,-1), (-2,2,1), (4,0,3), (2,6,-3)}.

10. Let V = { (x, y, z) I x - y - z = 0 } be a subset of JEt 3 . (a) LetS= {(1, 1, 0), (5, 2. 3)}. Show that span(S) = V. (b) LetS'= {(1, 1,0), (5,2,3), (0,0, 1)}. Show that span(S') = JEt 3 . 11. Determine whether span { u (a)

u1

= (2,-2,0),

(b)

u1

= (1,6,4),

u2

u2

1, u 2, u 3}

= span {v 1 , v 2 } if

= (-1,1,-1), u 3 = (0,0,9), v 1 = (1,-1.-5), v 2 = (0,1,1).

= (2,4,-1),

u3

= (-1,2,5), v 1 = (1.-2,-5), v 2 = (0.8,9).

subset.

B, C represent three

0,-c)lcElEt}.

12. Let u 1 = (2,0,2,-4), u 2 = (1,0,2,5), u 3 = (0,3,6,9), u 4 = (1,1,2,-1). v 1 = (-1,2,1,0), v2 = (3,1,4,0). V3 = (0,1,1,3), v 4 = (-4,3,-1,6). Determine if the following are true.

Chapter 3. Vector Spaces

120

(b) span{v 1 , v 2 , v 3 , v4} S: span{u 1 , u2, us, u4}. (c) span{u 1 , u 2 , u 3 , u 4 } = JR 4 . (d) span{v1, v 2 , v 3 , v 4}

= JR 4 .

13. Let u, v, w be vectors in JRn and let

S1={u,v},

s4

S2={u-v,v-w,w-u},

= {u, u+v, u+v+w},

s5 =

S 3 ={u-v,v-w.u+w},

{u+v, v+w, u+w, u+v+w}.

Suppose n = 3 and span{u,v,w} = JR 3 . Determine which of the sets above span JR 3 . 14. Determine which of the following statements are true. Justify your answer.

(a) If u is a nonzero vector in JR 1 , then span{ u} = JR 1 . (b) If u, v are nonzero vectors in JR 2 such that u of. v, then span {u, v} = JR 2. (c) If S 1 and S 2 are two subsets of lRn, then span(S1

n S 2 ) = span(SI) n span(S2 ).

(d) If S 1 and S 2 are two subsets of lRn, then span(S1 u S 2 ) = span(SI) u span(S2).

Question 3.15 to Question 3.31 are exercises for Sections 3.3 and 3.4.

15. Determine which of the following are subspaces of JR 3 . Justify your answer.

(a) {(0,0,0)}. (b) { ( 1' 1' 1)}.

(c) {(0,0,0), (LL1)}. (d) { (0, 0, c) I cis an integer}. (e) { (O,O,c) I cis a real number}. (f) { (1, 1, c) I cis a real number}. (g) {(a, b. c) I a, b, care real numbers and abc= 0 }. (h) {(a, b, c) I a, b, care real numbers and a~ b ~ c }. (i) {(a, b, c) I a, bare real numbers and 4a = 3b }.

(j) { (a, b, b) I a, b are real numbers } . (k) {(a, b, ab) I a. bare real numbers}. 16. Determine which of the following are subspaces of JR 4 . Justify your answers. (a) {(w,x,y,z)lw+x=y+z}. (b) {(w,x,y,z)lwx=yz}.

(c) { (w, x, y, z) I w + x + y = z2 } . (d) { (w,x,y,z) I w = 0 and y = 0}.

" 3. Vector Spaces

Exercise 3

121

(e) { (w,x,y,z) I w = 0 or y = 0}. (f) { (w,x,y,z) I w = 1 and y = 0}. (g) { (w, x, y, z) I w

+z=

0 and x

+ y- 4z =

0 and 4w

+ y- z =

0 }.

(h) {(w,x,y,z)lw+z=O or x+y-4z=Oor4w+y-z=O}. 17. Give an example of a 2 x 3 matrix A, if possible, such that the solution space of the linear system Ax = 0 is

-w, u+w},

u+v

+w}.

ts above span 1R

3

(a) JR3.

(b) the plane { (x, y, z) I 2x

(c) the line { (t,2t,3t) It E JR}.

(d) the zero subspace.

·

+ 3y - z = 0 } .

·answer. 18. Let W be a subspace of JRn and let v E .!Rn. The set 2

[u,v} = 1R . n(S 1 ) n span(S2).

m(S1 ) U span(S2).

W+v={u+vluEW} is called a coset of Hi containing v. interpretation for the coset vV + v.

For each of the following, give a geometric

= { (x, y) I x + y = 0} and v = (1, 1). (b) W = { c(L 1, 1) IcE lR} and v = (0, 0, 1). (c) W = { (x, y, z) I x + y + z = 0} and v = (2, 0, -1). (a) W

md 3.4.

II

answer.

19. Let U, V and W be the three planes defined in Question 3.4. Is U IR 3 ? Is V n W a subspace of JR 3 ? Justify your answers. 20. Let V and W be subspaces of .!Rn. Define V

(a) Show that V

+ W = { v +wIvE V

and w

E W}.

+ W is a subspace of .!Rn.

(b) Write down the subspace V (i) V

n V a subspace of

= { (t, 0) I t

E

+W

lR} and W

explicitly if

= { (0. t) It E lR }.

(ii) V = { (t,t,t) It E IR} and W

= { (t, -t,O) It E .IR}.

2L (All vectors in this question are written as column vectors.) Let A be an m x n matrix. Define VA to be the subset {Au I u E .!Rn} of JRm. our answers.

(a) Show that VA is a subspace of .!Rm. (b) Write down the subspace VA explicitly if

(i) A=

(1 2 3). 0

1

1

Chapter 3. Vector Spaces

122

22. (All vectors in this question are written as column vectors.) Let A be ann x n matrix. Define WA to be the subset { u E JR.n I Au= u} of JR.n.

(a) Show that W A is a subspace of JR.n. (b) Let A=

(

1 0

0 1

0 0

-1) 0 -1

. Write down the subspace W A explicitly.

23. Determine which of the following statements are true. Justify your answer.

(a) JR. 2 is a subspace of JR. 3 . (b) The solution set of x

+ 2y -

z = 0 is a subspace of JR. 3 .

(c) The solution set of x

+ 2y -

z = 1 is a subspace of JR. .

3

(d) If 5 1 and 5 2 are two subsets of JR.n, then span(51 U 52)= span(51) (See Question 3.20.) 24. Let

+ span(52)·

V and W be subspaces of JR.n.

(a) Show that V n W is a subspace of JR.n. (Hint: Use Remark 3.3.8.) (b) Give an example of V and W in JR. 2 such that V U W is not a subspace. (c) Show that V U W is a subspace of JR.n if and only if V .. The scalar ).. is called an eigenvalue of A and u is said to be an eigenvector of A associated with the eigenvalue >.. Example 6.1.4 1. Let A be the 2 x 2 matrix in Example 6.1.1. Let u

=

(!)

and v

= (~

1 ). Then

Diagonalization

Section 6.1. Eigenvalues and Eigenvectors

179

Au= (0.96 0.04

0.01) (1) = 0.99 4

(1) = u 4

and

Av = (0.96 0.04

0.01) ( 1 ) 0.99 -1

= ( 0.95 ) = 0 _95 ( 1 ) = 0 _95 v. -0.95 -1

So 1 and 0.95 are eigenvalues of A, u is an eigenvector of A associated with the eigenvalue 1 and v is an eigenvector of A associated with the eigenvalue 0.95. 1 1 1

D

and let u

UJ

and w

~ ( ~2) Then

m~ (D ~ ~ o: DuJ m ~ ~J ~

Bu

B,

~ (D "~

~ G: DG)

3u

3

0

(

o,

and

20% of the total ay in the urban

So 3 and 0 are eigenvalues of B, u is an eigenvector of B associated with the eigenvalue 3 and v, w are eigenvectors of B associated with the eigenvalue 0. Note that 1 0

m is to express

the problem of eigenvalues and

-1

1 1 1

1) (1 1

1

1 1

01 -1

1) -2 1

(3 0 0) 0 0

0 0 0 0

(check it). vector u in JFg_n

e an eigenvector

( 1) . Then '-1

Remark 6.1.5 Let A be a square matrix of order n. Then

A is an eigenvalue of A ¢:?

Au= AU for some nonzero column vector u

¢:?

AU- Au= 0 for some nonzero column vector u

¢:?

(-\I- A)u = 0 for some nonzero column vector u

¢:?

the linear system (-\I- A)x = 0 has non-trival solution

¢:?

det(U- A)

= 0 (by Theorem 3.6.11).

If expanded, det(-\I- A) is a polynomial in A of degree n.

180

Chapter 6. Diagonalization

Definition 6.1.6 Let A be a square matrix of order n. The equation

det(.U- A) = 0 is called the characteristic equation of A and the polynomial det(U- A) is called the characteristic polynomial of A. Example 6.1.7

1. Let A be the 2 x 2 matrix in Example 6.1.1. The characteristic polynomial of A is

(>- (o1

det(U- A)= det

0)- (0.96 1 0.04

I),-

=

0.96 -0.04

-0.01 ), - 0.99

0.01)) 0.99

I

= (>.- 0.96)(>.- 0.99)- ( -0.01)( -0.04) = ), 2

1.95,\ + 0.95

-

= (>.- 1)(>.- 0.95). Hence det(>.J- A) = 0 if and only if), = 1 or 0.95. The eigenvalues of A are 1 and 0.95. 2. Let B be the 3 x 3 matrix in Example 6.1.4.2. The characteristic polynomial of B is ,\-1 det(U-B)= -1 -1 = ),3-

-1 >.-1 -1

3),2

=

-1 -1 ,\-1

(>.- 3)(,\- 0)2.

Hence det(>.J- B) = 0 if and only if),= 3 or 0. The eigenvalues of B are 3 and 0.

3. Let C

~ G~~

D.

The chamdeci,tic polynomial of C ;,

), det(>.J- C) = 0 -1

= >, 3

-

1 ), -1 >, 2

-

0 -2 >.-1

2,\ + 2 = (>.- 1)(>.- h)(>.+ h).

Hence det(>.J- C) = 0 if and only if), = 1. yl2 or yl2 and -V2.

-V2.

The eigenvalues of C are 1.

Diagonalization

Section 6.1. Eigenvalues and Eigenvectors

181

Theorem 6.1.8 (The Main Theorem on Invertible Matrices) Let A be ann x n matrices .. The following statements are equivalent. 1. A is invertible.

2. The linear system Ax

=

0 has only the trivial solution.

3. The reduced row-echelon form of A is an identity matrix. 4. A can be expressed as a product of elementary matrices.

10mial of A is

5. det(A)

:/= 0.

6. The rows of A form a basis for m;,n. 7. The columns of A form a basis for m;,n.

8. rank(A)

= n.

9. 0 is not an eigenvalue of A.

; of A are 1 and

Proof By Theorem 3.6.11, statements 1 to 7 are equivalent. By Remark 4.2.5.2, "5 {'} 8''. Since det(OJ- A)= det( -A)= (-l)n det(A),

by Remark 6.1.5, det(A)

:/= 0 if and

only if 0 is not an eigenvalue of A, i.e. ·'5 {'} g·•.

lynomial of B is Theorem 6.1.9 If A is a triangular matrix (either upper triangular matrix or lower triangular matrix), the eigenvalues of A are the diagonal entries of A. Proof Suppose A= (a;j)nxn is a triangular matrix. Then A.I- A is a triangular matrix with diagonal entries A.- an, A.- a22, ... , A.- ann· By Theorem 2.5.8,

det(U- A)= (A.- an)(A.- a22) ···(A.- ann)·

Bare 3 and 0.

Hence the diagonal entries, an, a22, ... , ann, of A are the eigenvalues of A. Example 6.1.10

1. The eigenvalues of

J2).

'!lues of C are 1,

2. The eigenvalues of

(

-~1

3.5 5 0

(~: ~ 10

-4.5

~) 10

are -2, 0 and 10.

Chapter 6. Diagonalization

182

Definition 6.1.11 Let A be a square matrix of order n and A an eigenvalue of A. Then the solution space of the linear system (>.I -A)x = 0 is called the eigenspace of A associated with the eigenvalue A and is denoted by E>.. Note that if u is a nonzero vector in E>., then u is an eigenvector of A associated with the eigenvalue A. Example 6.1.12 1. Let A be the 2 x 2 matrix in Example 6.1.1. The eigenvalues of A are 1 and 0.95. For A= 1, the linear system (>.I- A)x = 0 is -0.01 ) 1-0.99

(X) y

-

Solving the system, we have

(~)

= t(

1- 0.96 ( -0.04

0.04 ( -0.04

(0) 0 0

·~ 5 )

-0.01) 0.01

(X) y

=

(0) . 0

where tis an arbitrary parameter. Hence

the eigenspace of A associated with the eigenvalue 1 is

For A= 0.95, the linear system (>.I- A)x = 0 is (

0.95- 0.96 -0.04

-0.01 ) 0.95 - 0.99

Solving the system. we have

(X) = y

(~)

=

-0.01 ( -0.04

(0) 0

t( ~ ) 1

-0.01) -0.4

(X) = y

(0) . 0

where tis an arbitrary parameter. Hence

the eigenspace of A associated with the eigenvalue 0.95 is

E 0 .95 =span { (

~1 )}.

2. Let B be the 3 x 3 matrix in Example 6.1.4.2. The eigenvalues of B are 3 and 0. For A= 3, the linear system (>.I- B)x = 0 is 3 -1 -1 ( -1

Solving the

-1

3-1 -1

~y,tem, we have G) ~ t G) whec,e tie an acbitmry pammetec Hence

Diagonalization

1e of A. Then )fA associated

Section 6.1. Eigenvalues and Eigenvectors

183

the eigenspace of B associated with the eigenvalue 3 is

;iated with the

~

For A= 0, the linear system (.\I- B)x = 0 is

1 and 0.95.

=

(~).

rameter. Hence

0 -1 -1 ( -1

Solving the

-1 -1 ) (X) (0)0 0-1 -1 0-1 0 y z

'Y~tem, we have

¢}

G) 'G)

(-1-1 -1 -1) (X) (0) -1 -1 -1 -1 00 +t

0 1 )

y z

whece ,, t ace acbitmy

parameters. Hence the eigenspace of B associated with the eigenvalue 0 is

3. Let C be the 3 x 3 matrix in Example 6.1.7.3. The eigenvalues of C are L

V2 and

-V2.

:~)=G).

For.\= 1, the linear system (.\I- C)x = 0 is

rameter. Hence

Solving the eyetem, we have

are 3 and 0.

:trameter. Hence

G) ~

~) 2

t(

whece tie an acbitcary panunetec, Hence

the eigenspace of C associated with the eigenvalue 1 is

Similarly, the eigenspaces of C associated with the eigenvalues

V2 and -V2 are

184

Chapter 6. Diagonalization

4. Let M =

G~).

By Theorem 6.1.9, Jvf has only one eigenvalue 2. It is easy to

check that the eigenspace of M associated with the eigenvalue 2 is E 2 = span {

Section 6.2

G) }.

Diagonalization

Definition 6.2.1 A square matrix A is called diagonalizable if there exists an invertible matrix P such that p- 1 AP is a diagonal matrix. Here the matrix Pis said to diagonalize

A. Example 6.2.2 0 96 1. The matrix A= ( · 0.04

0 01 · ) in Example 6.1.1 is diagonalizable because 0.99 1

1 )- (0.96 -1 0.04

2. The matrix B

=

( 1~ 1~ 1~)

in Example 6.1.4.2 is diagonalizable because

1 0 -1

3. The matrix M

=

0.01) (1 0.99 4

1 1 1

(~ ~)

in Example 6.1.12.4 is not diagonalizable. i.e. there is no

invertible matrix that can diagonalize M.

Proof Assume the contrary. i.e. there exists an invertible matrix (: -1

G!) G~) G!) = G ~) ·

!)

such that

)iagonalization

It is easy to

Section 6.2. Diagonalization

Then

G~) (~ !)

185

(~

!) G~)

r:

which implies

=

Aa =f.Lb

2b

= Ac

+2c b

+ 2d =

f.Ld.

Solving the equations, we obtain a= 0 and b = 0. However,

!)

(~

(~ ~)

is not

invertible, a contradiction. (We shall introduce a systematic approach to test whether a square matrix is diagonalizable in Algorithm 6.2.4.) an invertible o diagonalize

Proof

( =?) Suppose A is diagonalizable. Let P be an invertible matrix such that

1 ,>,

where D =

>- 2

0).

(

0 Let P = (u1 have

lse

An

u2

un) where Ui is the ith column of P. Since AP =PD. we

c ] ,),2

=}

e. there is no

~)

Theorem 6.2.3 Let A be a square matrix of order n. Then A is diagonalizable if and only if A has n linearly independent eigenvectors.

A (u1

u2

(Au1

Au2

un) = (u1

u2

Aun) = (>-1u1

un) A2u2

O

Anun) ·

So Aui = Aiui for all i, i.e. u1, u 2 . ... , Un are eigenvectors of A. Since P is invertible, by Theorem 3.6.11. { u1, u2, ... , Un} is a basis for JRn and hence must be linearly independent. such that

(-,1 u1 A2U2

] c O

>-2

By Theorem 3.6.11, Pis invertible. So P- 1 AP = D, i.e. A is diagonalizable.

Algorithm 6.2.4 Given a square matrix A of order n, we want to determine whether A is 1 diagonalizable. Also, if A is diagonalizable, find an invertible matrix P such that P- AP is a diagonal matrix. Step 1: Find all distinct eigenvalues .>- 1 , .>- 2 , ... , Ak· (By Remark 6.1.5, eigenvalues can be obtained by solving the characteristic equation det(.>-1- A) = 0.) Step 2: For each eigenvalue Ai, find a basis S>-.i for the eigenspace E>-.,· Step 3: LetS= S>-. 1 U S>-. 2 U · · · U S>-.k·

lSI < n, then A is not diagonalizable. If lSI = n, say S = {u1, u2, ... , un},

(a) If (b)

then P

un) is an

invertible matrix that diagonalizes A.

Remark 6.2.5 The following are some remarks on Algorithm 6.2.4: 1. In Step 1, sometimes, the matrix A may have eigenvalues that are not real numbers but complex numbers, i.e. the characteristic equation det(.>-1- A) = 0 has complex solutions. We can still use the algorithm to diagonalize the matrix. (See Question 6.10.) However, to discuss the theory, we need the concept of vector spaces over complex numbers. 2. Suppose the characteristic polynomial of the matrix A can be factorized as

where .>- 1 , .>- 2 ,

... ,

Ak are distinct eigenvalues of A. Then for each eigenvalue Ai,

Furthermore, A is diagonalizable if and only if in Step 2, for each eigenvalue Ai, dim(E>-.J =

Ti,

i.e. IS>-., I =

Ti·

(The proof of this result requires some advanced knowledge of linear algebra which is beyond the scope of this book.)

Diagonalization

Section 6.2. Diagonalization

187

3. In Step 3, the set Sis always linearly independent. (See Question 6.22.)

) =PD

Example 6.2.6 1. Let B be the 3 x 3 matrix in Example 6.1.4.2. Step 1: By Example 6.1.4.2, the eigenvalues are 3 and 0. Step 2: By Example 6.1.12.2,

.alizable.

{(D }

ffi a

1e whether A is h that P~ 1 AP

envalues can be

b~i" foe E,,

{0!), G)};,

a

-1)

~G1

-1

Step 3, Let P

b~idocE0 , ~

. Then P~ 1 BP =

0

-1

un)

is an

In Step 3, if we let Q

=

~

(

1 1 1

2. Let C be the 3 x 3 matrix in Example 6.1.7.3.

Jt real numbers ' 0 has complex (See Question tor spaces over

Step 1: By Example 6.1.7.3, the eigenvalues are 1,

v'2 and -J2.

Step 2: By Example 6.1.12.3,

z;ed as { (

~)

} i " bMiefo;£

~ and

envalue A;, {

(--~)} ieabMi,fmE_~

1 eigenvalue A;,

algebra which is

Step 3: Let P

=

(

-2

-1

~

v'2 1

( ~ ~ -v'2~ ) .

0

0

188

Chapter 6. Diagonalization

3. Let A

~ ~ ~) .

= (

-3

5

2

Step 1: The eigenvalues are 1 and 2. (We can get the eigenvalues of A without solving the equation det(..U- A) = 0. Why?) Step 2: For A= 1, the linear system (.>.I- A)x = 0 is

(~~ ;~ ~J mm Solving the

So { (

~y~tem, we have

G) ~

t(

~1) whe" t i~ an rucbitcary pammetee

~~)} i'" hMidm E;

For A= 2, the linear system (..U- A)x = 0 is

(~~ ~, ~)(~) m G) ~ G) t

Solving tbe 'yetem, we have

So {

m} ;,

whece t

i~ an aebitmey panrmetec,

a Laei' foe E,,

Step 3: Since we only have two linearly independent eigenvectors, A is not diagonalizable. Theorem 6.2. 7 Let A be a square matrix of order n. If A has n distinct eigenvalues, then A is diagonalizable. Proof Suppose A has n distinct eigenvalues. In Step 2 of Algorithm 6.2.4, we can find one eigenvector for each eigenvalue and hence we have n eigenvectors. By Remark 6.2.5.3, these eigenvectors are linearly independent. So A is diagonalizable.

Example 6.2.8 Let A =

(0~1 2~1 2~1

2!1) .

Diagonalization

Section 6.2. Diagonalization

189

By Theorem 6.1.9, A has eigenvalues 1, 2, 3 and 4. Thus by Theorem 6.2.7, A is diagonalizable.

s of A without

Remark 6.2.9 The converse of Theorem 6.2.7 is not true, i.e. a diagonalizable matrix of order n may not need to have n distinct eigenvalues. For example, the matrix B in Example 6.2.6.1 is a 3 x 3 diagonalizable matrix but has only two eigenvalues 3 and 0. See also Remark 6.2.5.2. Discussion 6.2.10 As we have seen in Example 6.1.1, one of the applications of diagonalization is to compute powers of square matrices: Let A be a square matrix of order n and P an invertible matrix such that

rary parameter.

2. Suppose A is invertible. By Theorem 6.1.8, we know that .Ai

cJ 0 for

all i. Then

rary parameter.

is not diagonal-

igenvalues. then 2.4, we can find Remark 6.2.5.3,

Example 6.2.11

~ ~

!

0~ -~1)

such that p-l AP

4

L Let A

P =

( -~2

(

6 ). Following Algocithm 6.2.4, we find '"' invectible m,t ,B

a=f3

a-1 IRm be a linear transformation and A the standard matrix forT. Then Ker(T) =the nullspace of A which is a subspace of IRn.

Proof Ker(T) = { u

I

T(u) = 0} = { u

I

Au= 0} which is the nullspace of A.

Definition 7.2.10 Let T be a linear transformation. The dimension of Ker(T) is called the nullity ofT and is denoted by nullity(T). By Theorem 7.2.9, if A is the standard matrix forT then nullity(T) = nullity(A).

Example 7.2.11

is, we need

1. The nullity of the linear transformation in Example 7.2.8.1 is 0 while the nullity of the linear transformation in Example 7.2.8.2 is 1. 2. Let T : IR 4 --> IR 4 be a linear transformation defined by

Find a basis for the kernel of T and determine the nullity of T

Solution The kernel is found by solving

= 0, z = 0.

Chapter 7. Linear Transformations

218

The linear system has a general solution w = s, x = -3t, y = t. z = t where s, t are arbitrary parameters. Thus the kernel is the subspace

{(l,O,O,O)T, (0, -3,l,l)T} is a basis for the kernel and nullity(T) = dim(Ker(T)) = 2. Theorem 7.2.12 (Dimension Theorem for Linear Transformations) If T: IR:.n JR:.m is a linear transformation, then rank(T)

+ nullity(T)

--+

= n.

Proof The theorem is a restatement of Theorem 4.3.4, the Dimension Theorem for Matrices: If A is an m x n matrix, then rank( A)+ nullity(A) = n. If we define a linear transformation T : IR:.n --+ JR:.m by T(x) = Ax for x E IR:.n. rank(T) = rank(A) and nullity(T) = nullity(A). we have

rank(T)

Section 7.3

Since

+ nullity(T) =rank( A)+ nullity(A) = n.

Geometric Linear Transformations

Discussion 7.3.1 Several well-known geometric transformations on IR:. 2 and IR:. 3 such as scalings, reflections about lines and planes through the origin, and rotations about the origin are in fact linear transformations. In the following, we shall examine some examples of these transformations. Recall that every linear transformation T : IR:.n --+ JR:.m is completely described by the images of the vectors in a basis for JR:.n. In particular, we can use the image of the standard basis {e 1 , e 2 . . . . , en} for JR:.n to find the standard matrix for a linear transformation (see Discussion 7.1.8). Example 7.3.2 (Scalings in IR:. 2 ) Suppose S : IR:. 2

--+

IR:. 2 is a linear transformation such that

for some positive real numbers .A 1 and .A 2. The standard matrix for S is ( .Aol

\02) . So

/\

nsformations

Section 7.3. Geometric Linear Transformations

219

1ere s. t are The effect of this linear transformation is to scale by a factor of .\ 1 along the x-axis and by a factor of .\ 2 along the y-axis. For example, in the following, we show the graph of the image of the square with vertices A= (1,1)T, B = (-1,1)T, C = (-1,-1)T, D = (L-1)T under S with .\ 1 = 1.5 and .\2 = 0.5: y

y

B

[f T: JRn--+

A

s X

c

·or Matrices: JRn.

Since

(

D

10.5

S (B,...:..)----+---,S (A) 0) 0.5

X

S(C)

S(D)

In general, S is called a scaling along the x and y-axes by factors of .\ 1 and .\ 2 respectively. For the special case when .\ 1 = .\ 2 = .\, the scaling is known as a dilation if .\ > 1 and a contraction if 0 < .\ < 1. Remark 7.3.3 Let T : JR 2 --+ JR 2 be a linear transformation such that T(x) x E JR 2 where A is a 2 x 2 matrix. Suppose there exists a 2 x 2 invertible matrix P

JR 3 such as s about the ne examples s completely se the image for a linear

= (u 1

Ax for

u 2 ) such that

for some positive real numbers .\ 1 and .\ 2 . Then

Thus T can be regarded as a scaling that scales along axes in the directions of u 1 and u 2 by factors of .\ 1 and .\2 respectively. (In here, the new axes may not be perpendicular to each other.) The observation above gives us a geometric interpretation for diagonalizable matrices. Example 7.3.4 LetT : JR 2 . ( _1 forT IS 0 25

~J· So

--+

JR 2 be a linear transformation such that the standard matrix

It is easy to check that

-2)- 1 1

(

1

0.25

220

Chapter 7. Linear Transformations

Thus Tis a scaling that scales along axes in the directions of (2, 1)T and ( -2, 1)T by factors of 1.5 and 0.5 respectively. The following graph shows the image of the rectangle with vertices A C = (-2, -1)T, D = (2, -1)T under the transformation T:

= (2.l)T. B = (-2, l)T,

y

y

T(A), T X

(

1

0.25

X

D"T(C)

Example 7.3.5 (Scalings in JR;. 3 ) Similar to Example 7.3.2, the standard matrix for the scaling along the x, y and z-axes in

Ill.' by factom of A, A, and A,, ceepectively, is For the special case when A1 a contraction if 0 < A < 1.

0' ~

= A2 = A3 = A, the scaling is known as a dilation if A > 1 and

Example 7.3.6 (Reflections in JR;. 2 )

1. Reflections about the usual coordinate axes of JR;. 2 : Let F 1 : JR;. 2 ---> 1R;.2 be the reflection about the x-axis, i.e.

So the standard matrix for F 1 is given by

G ~1 )

y

X

and

Fl (G)) = G ~1) G) (:y) . Similarly, the reflection F 2

:

JR;. 2 ---> JR;. 2 about the y-axis has the standard matrix

1sformations

r

by factors

1d z-axes in

Section 7.3. Geometric Linear Transformations

2. The reflection about the line y = x : Let F 3 : IR 2 -+ IR 2 be the reflection about the line y = x. We have

y

So the standard matrix for F 3 is given by

(0 1) d 1

0

X

an

3. Reflections about any line in IR 2 that passes through the origin: First of all, recall that any line in IR 2 (except x = 0) that passes through the origin has an equation of the form y = mx where m is the gradient of the line. If 8 is the angle between the x-axis and the line y = mx, then m = tan(8). 2

Let F : IR -+ IR 2 be the reflection about such line. To obtain the standard matrix (and thus the formula) for F, we consider the images of the standard basis vectors e1 = (1, Of and e 2 = (0, 1)T for JR 2 .

fA> 1 and

lard matrix

221

y

y y

= x tan(8)

y

=

x tan(8)

8 X

We have F

X

((1)) _F(e _(IIF(el)ll IIF(e -

1 )-

0

1 )11

cos(28)) -_ (cos(28)) . sin(28) sm(28)

and

F((O)) = F(e 1

2

)

= (IIF(e2)ll cos(28-

IIF(e 2 )11 sin(28-

~)) = ( co~(~7r _

~)

28) ) - sm( 2- 28)

sin(28) ) - ( - cos(28) ·

Chapter 7. Linear Transformations

222

So the standard matrix for F is given by

(:~~~~:?

sin(2B) ) - cos(2B) ·

e

We check that, when = 0, ~ and i, the standard matrix gives us reflections about the x-axis, y-axis and the line y = x respectively. Remark 7.3.7 The formula of the reflection Fin Example 7.3.6.3 can also be written as

F(u) = u- 2(u · n)n

for u E IR.2

where n =(sin( B), - cos(B))T. (Check it.) Can you explain the formula geometrically? Example 7.3.8 (Reflections in IR. 3 ) Similar to Example 7.3.6.1, the standard matrices for reflections about xy-plane, xz-plane and yz-plane in IR.3 are

(

~ ~ ~1 1

0

0 )

~ ~1 ~

(1

0

0)

and

(-001

0~ ~1)

respectively. (See also Question 7.26.) Example 7.3.9 (Rotations in IR. 2 ) 1. Anti-clockwise rotations about the origin through angles ~,

1r

and

3 27[

:

Let R 1 : IR.2 -+ IR.2 be the anti-clockwise rotation about the origin through an angle ~- The following graph shows the image of the triangle with vertices A = (0, 1)T, B = (-1, -1)T, C = (1, -1f under the rotation R 1 :

y

y

A

X

B

X

c

To obtain the standard matrix for R 1 , we consider the images of the standard basis vectors e 1 = (1,0)T and e 2 = (0, 1)T for IR. 2 .

Jsformations

Section 7.3. Geometric Linear Transformations

223

y

y

tions about X

written as

rically?

We have

te, xz-plane

So the standard matrix for R 1 is

The image of A is (

~ ~) 1

and the image of Cis

(~ ~1 ).

G) = ( ~

1 } the image of B is

G~ ) 1

(

~1 )

=

G~ =~) (~ 1

) (

G).

Similarly, the standard matrices for anti-clockwise rotations through angles are ( -1 0

0 ) and ( 0 -1 -1

1}

11

and

3 ;

1) respectively. . 0

2. The anti-clockwise rotation through an angle 8:

gh an angle 4 = (0.1y,

To obtain the standard matrix for R, we consider the images of the standard basis vectors e 1 = (1,0)T and e 2 = (0, 1)T for ffi?. y

y

X

mdard basis

X

We have

R

((1)) -_ 0

_

R(e 1 ) - (IIR(ei)IIcos(B)) -_ (cos(B)) . IIR(ei)II sin(B) sm(B)

X

Chapter 7. Linear Transformations

224

and _ R (( 0)) -_ R(e2)1

(IIR(e2)iicos(~+8)) IIR(e 2 )ilsin(~+8)

cos( e) So the standard matrix for R is given by ( sin( 8)

-_ (-sin(e)) . cos(e)

e))

-sin( cos( 8)

·

Example 7.3.10 (Rotations in IR. 3 ) Let R : IR.3 ----" IR. 3 be the anti-clockwise rotation about the z-axis through an angle e. \Ve consider the image of the standard basis vectors e 1 = (1, 0, O)T, e 2 = (0. L O)T and e 3 = (0,0, 1)T for IR 3 .

z

-------7

R(e2)

/

/

y

/

~-------------------/ X

We have the following observations: L The z-axis is fixed under the rotation, i.e. R( e 3 ) = e 3 . 2. The xy-plane is rotated in the same manner 7.3.9.2. It is easier to check the images of (You are reminded that e 1 = (1, 0. O)T and

as the rotation in IR 2 discussed in Example e 1 and e 2 by drawing the xy-plane only. e 2 = (0, 1, O)T in the following graphs.) y

y

X

Thus R(e 1 )

=

[cos(8)] e 1 +[sin( e)] e 2 and R(e2)

X

=-

[sin(8)] e 1 + [cos(8)] e2.

sformations

Section 7.3. Geometric Linear Transformations

225

Summarizing the results above, we have -sin(())) cos(()) , ( 0

The standard matrix for R is

e.

m angle . 1, O)T and

(

cos( e)

-sin( e)

sin~ e)

co~(())

0)

~ .

Similarly, the standard matrices for rotations in JR 3 about x-axis and y-axis are 1

(~

0 cos(e) sin( B)

-

si~(e))

cos(()) and

(

cos( e)

-

si~(e)

0 1 0

sin(B)) 0 cos(e)

respectively.

Example 7.3.11 (Translations) 1. Let T : JR 2

----+

JR 2 be a translation in JR 2 such that

T( (xy)) __ (xy ++ ab)

for (x, yf E JR 2

where a and b are real constants. For example, the following graph shows the image of the square with vertices ( 1. 1) T. (0, 1)T, (O,O)T, (1,0)T under the translation with a= 1 and b = 2: y

y

in Example -plane only. graphs.)

3 2

T

1

X

X

D

a= 1, b = 2

2

X

Tis not a linear transformation except when a= b = 0. If a, b are not both zero, T ( (

~))

= (

~)

-1- ( ~) which violates Theorem 7 .1.4.1.

Chapter 7. Linear Transformations

226

2. Let T' : ~ 3

-->

~ 3 be a translation in ~ 3 such that

where a, b and c are real constants. Same as translations in ~ 2 , T' is not a linear transformation except when a= b = c = 0.

Example 7.3.12 (Shears) 1. A mapping H : ~ 2

--> ~ 2

is called a shear in the x-direction by a factor of k if

For example, the following graph shows the image of the square with vertices (1.1)T, (0, 1)T, (0, O)T, (1. O)T under the shear in the x-direction with k = 1:

y

y

1

H 1

Note that any point on the line y under the action of H.

X

k=1

1

2

X

= 1 is moved to the right by a distance of k = 1

We observe that for all (x,y)T in ~ 2 ,

Thus H is a linear transformation and the standard matrix for H is (

~ ~) .

Similarly, the standard matrix for the shear in the y-direction by a factor of k is

tsformations

Section 7.3. Geometric Linear Transformations

227

2. A mapping H' : JR. 3 ---> JR. 3 is called a shear in the x-direction by a factor of k 1 and in the y-direction by a factor of k 2 if

,ot a linear

(~ ~ ~:).

Same os Part I, H' ie a lineae tcanefonnation with tlie etandacd matcix

Note that any point on the plane z = 1 is translated by k 1 in the x-direction and by k 2 in the y-direction under the action of H'.

kif

Discussion 7.3.13 (2D Computer Graphic) In 2D (dimension two) computer graphic, a figure is drawn by connecting a set of points by lines. If a figure is drawn by connecting n points, we can store it by a 2 x n matrix. For example, the matrix

A=G

0 0 1 1 0 0

y

~)

1

X

gives us the square with vertices ( 1, 1) T,

(0, 1)T, (0, O)T, (1, O)T.

X

:e of k

We can transform a figure by changing the positions of the vertices and then redrawing the figure. If the transformation is linear, it can be carried out by pre-multiplying the standard matrix for the transformation to the matrix representing the figure.

=

1

:tor of k is

For example, if we want to double both the width and the height of the square above, we only need to pre-multiply

G~)G

0 0 1 1 0 0

(~ ~)

y 2 t------,

to A, i.e.

~) = G 2

0

0

2

0 0

2

X

which gives us the square with vertices

(2, 2f' (0, 2)T' (0, O)T' (2, O)T. There are four primary geometric transformations that are used in 2D computer graphics: scalings, reflections, rotations and translations. We know that scalings, reflections and rotations are linear transformations but translations are not. One method to handle translations

Chapter 7. Linear Transformations

228

is to use a new system of coordinates called homogeneous coordinates: The homogeneous coordinate system is formed by equaling each vector in JR1. 2 with a vector in JR1. 3 having the same first two coordinates and having 1 as its third coordinate. For example, the matrix A representing the square with vertices (L 1f, (0, 1)T. (0, O)T, (1, Of becomes

0

0

1

1 0 0 1

1

1

If we want to draw the figure, we simply ignore the third coordinate.

Suppose P is the standard matrix for a geometric linear transformation on scaling, a reflection or a rotation. Then the matrix

JR1.

2

such as a

will transform A' accordingly. For example, the matrix square:

(:

0 2 0

G ~) D(~ 0 2 0

0

1 1

can double both the width and the height of the

1

0 0

0

1

1

D (:

0 2

0 0

2 0

1

1

1

D

To do a translation, we need to use a shear defined in JR1. 3 . For example, if we want to translate the square by a distance of 2 in the x-direction and by a distance of 1 in the y-direction, the shear with the standard matrix

(~1

001

2~)

(H DG H~ D G: y

2

1

D 2

3

X

will do the job:

2 3 1 1

1 1

sformations

a vector in 'r example, r becomes

Exercise 7

229

Exercise 7 Question 7.1 to Question 7.17 are exercises for Sections 7.1 and 7.2. 1. Determine whether the following are linear transformations. vVrite down the standard

matrix for each of the linear transformations.

~ 2 ~ ~ 2 such that T1 ( (~))

=

(b)

T2:

~ 2 ~ ~ 2 such that T2 ( (~))

=

(c)

Te •~' · ~l eueh that T, ((;)) ~ (x ~ ") foe (:) ~'

' such as a

ight of the

1re want to )f 1 in the

G) ~2 . e;) C) ~2 .

:

(a) T 1

(d) T, •

(~ ~ ~)

for

for

E

E

E

>

~l ·~ ~' euch that T, (G)) ~ (~ ~ :) foe G) ~·. E

(e) T 5 : ~n ~~such that T5 (x) = x · y for x E ~n where y = (y 1 , Y2· fixed vector in ~n. (f) T5:

~n ~~such

that T5(x) = x · x for x E

... , Ynr

is a

~n.

(In Parts (e) and (f). ~ is regarded as ~ 1 .) 2. For each of the following linear transformations, (i) determine whether there is enough information for us to find the formula ofT; and (ii) find the formula and the standard matrix for T if possible. (a) T: ~ 3 ~ ~ 4 such that

(b) T: ~ 2 ~ ~ 3 such that

(c) T: ~ 2 ~ ~ 2 such that

Chapter 7. Linear Transformations

230

(d) T : IR 3

-+

IR 2 such that

(e) T : IR 3

_,

IR 3 such that

(f) T : JR 3

-+

IR such that

3. Let S and T be linear transformations as defined below. Determine the formulae of the compositions SoT and To S whenever they are defined.

(a) 8 • li