Matrix Decomposition and Its Application in Statistics NK

Share Embed Donate


Short Description

Matrix Decomposition and Its Application in Statistics NK...

Description

Matrix Decomposition and its Application in Statistics  Nishith Kumar  Lecturer  Department of Statistics Begum Rokeya University, Rangpur. Email nk.!ru"#$gmail.com 1

%vervie& ' ' ' ' ' ' ' '

(ntro)uction LU )ecomposition *R )ecomposition +holesky )ecomposition or)an Decomposition Spectral )ecomposition Singular value )ecomposition -pplications 2

Introduction /his Lecture covers relevant matri0 )ecompositions, !asic numerical metho)s, its computation an) some of its applications. Decompositions provi)e a numerically sta!le &ay to solve a system of linear euations, as sho&n alrea)y in 12ampler, 3#4"5, an) to invert a matri0. -))itionally, -))itionally, they provi)e an important tool for analy6ing the numerical sta!ility of a system. Some of most freuently use) )ecompositions are the LU, QR , Cholesky , Jordan, Spectral decomposition   an) Singular alue decompositions . 3

Easy to solve system (Cont.) Some linear system that can be easily solved

The solution:

 b3 7 a33  b 7 a   8 88     b 7 a   n nn 

4

Easy to solve system (Cont.) Lower triangular matrix:

Solution: This system is solved using forward substitution

5

Easy to solve system (Cont.) Upper Triangular Matrix:

Solution: This system is solved using Backward substitution



LU Decomposition LU )ecomposition &as originally )erive) as a )ecomposition of ua)ratic an) !ilinear forms. Lagrange, in the very first paper in his collecte) &orks9 34:#; )erives the algorithm &e call that is use) to solve the system of linear euation. Let A !e a m × m &ith nonsingular suare matri0. /hen there e0ists t&o matrices L an) U  such that, &here L is a lo&er triangular matri0 an) U  is an upper triangular matri0. u33 u38 " u 88 U  =     " "

   

u3m 

     u mm  u8m

2here,  A = LU 

an)

 l 33  l  83  L =   l   m3

"



l 88







l m 8



  "    l mm  "

?L Lagrange 934@A 3>3@;

-. C. /uring 93#38?3#:=; !

o& to )ecompose -LUF -LUF  9upper triangular;  A ≈… ≈U  9upper   E k  ⋅⋅⋅ E 3 A ⇒ U    E 3;−3 ⋅⋅⋅ 9 E   E k;−3 U  ⇒ A  9 E 

− 8 8 − > A  No&, − 3@ 8 8  3 " "  A − 8 8 8 =  − 8 3 " 38 − > A 3 − 3 7 8 " 3  @ − 3@ 8 " "  A − 8 8 A − 8 8  3 " "  3 ⇒ " − = 8  = " 3 "  − 8 3 " 38 − > A " " − : " − @ 3 − 3 7 8 " 3  @ − 3@ 8

A  A = 38   @ A − 8 " − =  " − 38

U

E 2

 E 1

 A

(f each such elementary matri0 E i is a lo&er triangular matrices, it can !e prove) that 9 E 3;−3, ⋅⋅⋅, 9 E   E k;−3 are lo&er triangular, an)  E 3;−3 ⋅⋅⋅ 9 E   E k;−3 is a lo&er triangular matri0. 9 E   E 3;−3 ⋅⋅⋅ 9 E   E k;−3 then  A=LU. Let L=9 E  "

Calculation o! L o! L and  and U (cont.) A −8   A = 38 − >   @ − 3@

8

 $  8

A

3 "  "

"

"  A

3

 3   @

"

"  38

−8 −> − 3@

8

A

 8 

 No& re)ucing the first column &e & e have − 8 8 " "  A A − 8 8  3 " − = 8 =  − 8 3 " 38 − > A      " − 38 3 − 3 7 8 " 3  @ − 3@ 8 " "  A −8  A − 8 8  3 " "   3 ⇒ " − = 8  = " 3 "  − 8 3 " 38 − > " " − : " − @ 3 − 3 7 8 " 3  @ − 3@

8

  8

A

#

Calculation o! L o! L and  and U (cont.)  No&  3  −8  − 37 8

" "

  3

3 " "

−3

3 " " 3  " − @

"

−3

  3 "

3 =  8 3 7 8

" "  3 " " 

" "

3 " " 3

3 "

"

  3 "

@

3   " = 8   3 3 7 8

@

  3

/herefore, A −8  A = 38 − >   @ − 3@

 3 " " A − 8 8    " − = 8  A $  8 3 "   =LU     8 3 7 8 @ 3 " " − : 8

(f - is a Non singular matri0 then for each L 9lo&er triangular matri0; the upper triangular matri0 is uniue !ut an LU  )ecomposition  )ecomposition is not uniue. /here can  !e more than one such LU  )ecomposition  )ecomposition for a matri0. Such as A   A = 38   @

 A " "  3 − 8 7 A 8 7 A 8   A $ 38 3 " " − = 8   @ @ 3 " 3@ 8 " : 

−8 −>

=LU 

1%

alculation L and U (cont.) (cont.) Calculationof o! L o! L and  and U   /hus LU )ecomposition is not uniue. Since &e compute  LU  )ecomposition !y elementary transformation so if &e change L then U &ill !e change) such that -LU /o fin) out the uniue LU  )ecomposition,  )ecomposition, it is necessary to  put some restriction on L an) U  matrices.  matrices. Gor e0ample, &e can reuire the lo&er triangular matri0  L to !e a unit one 9i.e. set all the entries of its main )iagonal to ones;. LU Decomposition in R"

' li!rary9Catri0; ' 0H?matri09c9@,8,3, #,@,=,=,8,: ;,ncol@,nro&@; ' e0pan)9lu90;;

11

alculation of L and U (cont.) '

#ote there are also generali6ations of LU to non?suare an) singular

matrices, such as rank revealing LU factori6ation. ' 1Ian, +./. 98""";. %n the e0istence an) computation of rank revealing LU factori6ations. Linear Algebra and its Applications, @3A 3##?888. ' Ciranian, L. an) :" 3#3A;

Girstly *R )ecomposition originate) &ith >@;. Later Erhar) Schmi)t 93#"4;  prove) the *R Decomposition /heorem

(rhard Schmidt

93>4A?3#:#;

(f A is a m×n matri0 &ith linearly in)epen)ent columns, then  A can !e )ecompose) as ,  A = ! &here * is a m×n matri0 &hose columns form an orthonormal !asis for the column space of  A an) R is an nonsingular upper triangular matri0. 1

*R?Decomposition 9+ont.; /heorem  (f A is a m×n matri0 &ith linearly in)epen)ent columns, then  A can !e )ecompose) as ,  A = ! &here * is a m×n matri0 &hose columns form an orthonormal !asis for the column space of  A an) R is an nonsingular upper triangular matri0. Iroof Suppose -1u3  u8 . . .  un5 an) rank 9 A;  n. -pply the 8#. CAUC;?/)>-=2

Let A !e a m × m real symmetric matri0. /hen there e0ists an orthogonal matri0  9  such that  9   A9  = Λ or  A =  9 Λ 9  , &here : is a )iagonal matri0. ( 



34

Spectral Decomposition and &rincipal component Analysis 0Cont12 By using spectral )ecomposition &e can &rite

 A = 9 Λ 9 ( 

(n multivariate analysis our )ata is a matri0. Suppose our )ata is  )  matri0. Suppose  is mean centere) i.e.,  )  → 9 )  − µ ; an) the variance covariance matri0 is V. /he variance covariance matri0 V is real an) symmetric. Using spectral )ecomposition &e can &rite ;=9:9 ( . 2here : is a )iagonal matri0. Λ = diag 9λ 3, λ 8 ,, λ  ; -lso λ  ≥ λ  ≥  ≥ λ  n

3

8

n

tr9;;  /otal variation of Data tr9 :; 35

Spectral Decomposition and &rincipal component Analysis 0Cont12  /he Irincipal component transformation is the transformation 9?W; 9  2here,  E9i;"  X9i;i  +ov9i , $;" if i Y $  X93; Z X9 8; Z . . . Z X9n; n

∑  = r  .r 3 7 8 !r ( 

 9  3



=  2  , m



 9 8

=

2 n

= U Λ" appro0imation "

Lo& rank -ppro0imation to flo&ers image

Rank?3"" appro0imation

Rank?38" appro0imation #

Lo& rank -ppro0imation to flo&ers image

an>15% a**ro?imation

/rue (mage

!%

Butlier Detection Using S9D  Nishith an) Nasser 98""4,CSc. /hesis; propose a graphical metho) of outliers )etection using SXD. (t is suita!le for !oth general multivariate )ata an) regression )ata. Gor this &e construct the scatter plots of first t&o I+[s, an) first I+ an) thir) I+. 2e also make a !o0 in the scatter  plot &hose range lies median93 st9D ; K @ × mad 93 st9D ; in the ) ?a0is an) median98nd9D @rd9D ; K @ × mad 98nd9D @rd9D ; in the Y? a0is. 2here mad  me)ian a!solute )eviation. /he points that are outsi)e the !o0 can !e consi)ere) as e0treme outliers. /he points outsi)e one si)e of the !o0 is terme) as outliers. -long &ith the !o0 &e may construct another smaller !o0 !oun)e) !y 8.:78 C-D line !1

Butlier Detection Using S9D 0Cont12 -2K(NS?BR-DU?K-SS 93#>=; D-/Data set containing 4: o!servations &ith 3= influential o!servations. -mong them there are ten high leverage outliers 9cases 3?3";  an) for high leverage points  9cases 33?3=; ?(mon 98"":;. Scatter plot of a&kins, Bra)u an) kass )ata 9a; scatter plot of first t&o I+[s an) 9!; scatter plot of first an) thir) I+. !2

Butlier Detection Using S9D 0Cont12 C%D(G(ED BR%2N D-/Data set given !y Bro&n 93#>";. Ryan 93##4; pointe) out that the original )ata on the :@ patients &hich contains 3 outlier 9o!servation num!er 8=;.

Scatter plot of mo)ifie) Bro&n )ata 9a; scatter plot of first t&o I+[s an) 9!; scatter plot of first an) thir) I+.

(mon an) a)i98"":; mo)ifie) this )ata set !y putting t&o more outliers as cases := an) ::. -lso they sho&e) that o!servation 8=, := an) :: are outliers !y using generali6e) stan)ar)i6e) Iearson resi)ual 9. Daily mean &in) spee) 92S; m7sec #. ours of !right sunshine as percentage of ma0imum possi!le sunshine hours 9CIS; 3". Solar ra)iation 9SR; cal7cm87)ay

!!

+onseuences of SXD 21 &at'ix Alo'itm*5 9ol )1 Easic Decompositions5 Siam5 &hiladelphia1

Catri0 Decomposition. http77fe)c.&i&i.hu?!erlin.)e70plore7e!ooks7html7csa7no)[email protected]

"1

View more...

Comments

Copyright ©2017 KUPDF Inc.
SUPPORT KUPDF