Data Virtualization Business Intelligence Systems Van Der Lans Book en US

Data Virtualization for Business Intelligence Systems Revolutionizing Data Integration for Data Warehouses

Rick F. van der Lans

([FHUSWFUHDWHGIRU,QIRUPDWLFD&RUSRUDWLRQ&KDSWHU7KH)XWXUHRI'DWD9LUWXDOL]DWLRQ_  



7KH)XWXUHRI'DWD9LUWXDOL]DWLRQ$FFRUGLQJWR-DPHV0DUNDULDQ&72RI ,QIRUPDWLFD&RUSRUDWLRQ  ([SRQHQWLDO JURZWK LQ WUDGLWLRQDO WUDQVDFWLRQDO GDWD SOXV QHZ LQIRUPDWLRQ IURP VRFLDO PHGLD FDOO GHWDLO UHFRUGV VHQVRUV DQG GHYLFHV DQG JHRORFDWLRQ V\VWHPV  KDV PDGH LW LPSHUDWLYH WKDW EXVLQHVVHV WDNH WKH OHDG LQ KDUQHVVLQJ GDWD WR GHULYH QHZ LQVLJKWV IRU FRPSHWLWLYH DGYDQWDJH 7R WXUQ WKHVH QHZ LQVLJKWV LQWR EXVLQHVV RSSRUWXQLWLHV ERWK EXVLQHVV DQG ,7 H[HFXWLYHV DUH EHLQJ FKDOOHQJHGWRUHWKLQNWKHLULQIRUPDWLRQPDQDJHPHQWSUDFWLFHVEUHDNGRZQRUJDQL]DWLRQDODQGGDWD VLORVDQGLPSURYHEXVLQHVV,7FROODERUDWLRQ  :KLOHPDQ\FRPSDQLHVNQRZWKDWGDWDLVNH\WRGULYLQJFRPSHWLWLYHDGYDQWDJHWKH\DOVRNQRZWKDW WKH\PD\QRWEHDVHIIHFWLYHDVWKH\FRXOGEHDWWUDQVIRUPLQJLWLQWRDXVHIXODVVHW7KHGDWDLVWKHUH EXWLW·VIUDJPHQWHGDFURVVV\VWHPVLW·VLQWKHFORXGLW·VRQWKHGHVNWRSLW·VRQPRELOHGHYLFHVLW·VLQ DYDULHW\RIGDWDVRXUFHV,QIDFWLW·VHYHU\ZKHUHDQGLW·VGXSOLFDWHGDJDLQDQGDJDLQ,W·VRIWHQQRW DYDLODEOHZKHQLW·VQHHGHGDQGZKHQLWLVDYDLODEOHLWPLJKWEHIXOORIHUURUVRUGRHVQRWGHOLYHUWKH YDOXHGHVLUHGWRLPSURYHRSHUDWLRQVDQGLQFUHDVHUHYHQXH$QGDVGDWDJURZVWKHFRVWWRPDQDJH LWRQO\LQFUHDVHV  7KLVLVSDUWLFXODUO\WUXHLQWKHFDVHRIEXVLQHVVLQWHOOLJHQFHWKHWRSSULRULW\IRUWKH&,2$FFRUGLQJWR )RUUHVWHU 5HVHDUFK ,QF ´$V GDWD YROXPHV DQG LQIRUPDWLRQ FRPSOH[LW\ FRQWLQXH WR VN\URFNHW WUDGLWLRQDO EXVLQHVV LQWHOOLJHQFH WRROV WU\ KDUG WR NHHS XS ZLWK HYHU LQFUHDVLQJ DQG FKDQJLQJ GHPDQGV %XW LW·V DQ XSKLOO EDWWOH  DQG EXVLQHVV LQWHOOLJHQFH WRROV DQG DSSOLFDWLRQV GR QRW DOZD\V NHHS XS WKH ULJKW OHYHO RI SDFH DQG DGYDQFHPHQW $V D UHVXOW WKH ULIW EHWZHHQ EXVLQHVV UHTXLUHPHQWV IRU RQGHPDQG LQIRUPDWLRQ DQG UHDOWLPH GHFLVLRQV DQG EXVLQHVV LQWHOOLJHQFH DSSOLFDWLRQVDQG,7VXSSRUWVWDII VDELOLW\WRVXSSRUWWKHPFRQWLQXHVWRJURZµ  $FFRUGLQJWRWKHUHSRUW • RI%,UHTXLUHPHQWVFKDQJHRQEHWZHHQDGDLO\DQGPRQWKO\EDVLV •  RI WKH UHVSRQGHQWV VDLG WKH\ KDYH WR DVN GDWD DQDO\VWV WR FUHDWH FXVWRP UHSRUWV IRU WKHP • RIFXVWRPUHSRUWUHTXHVWVUHTXLUHDFXVWRPFXEHRUGDWDPDUWWRDQVZHUWKHUHTXHVW •  RI UHVSRQGHQWV FLWHG WKDW LW WDNHV EHWZHHQ GD\V DQG PRQWKV WR JHW WKHLU %, UHTXHVWV IXOILOOHG  ,W LV FOHDU ² LQ RUGHU WR WXUQ GDWD LQWR QHZ LQVLJKWV DQG WKXV RSSRUWXQLWLHV EXVLQHVVHV QHHG WR SD\ JUHDWHUDWWHQWLRQWRGDWDLQWHJUDWLRQWRSURYLGHPRUHDFWLRQDEOHWUXVWZRUWK\DQGWLPHO\LQIRUPDWLRQ LQWKHLU%,SURMHFWVDQGLQLWLDWLYHV7KH\QHHGWRSXWGDWDLQWHJUDWLRQDQGEXVLQHVVXVHUGULYHQVHOI VHUYLFH WR ZRUN IRU XQFRYHULQJ QHZ LQVLJKWV WKDW WKH EXVLQHVV QHHGV DQG FDQ WUXVW 'RLQJ WKLV ZLOO GULYH JURZWK UHGXFH FRVWV DQG GHOLYHU LQQRYDWLRQV WKURXJK VSRWWLQJ SDWWHUQV DQG WUHQGV ZKLOH PHHWLQJFRPSOLDQFHDQGULVNPDQGDWHVPRUHHIIHFWLYHO\%XWKRZGR\RXDFFRPSOLVKWKLV"

 +RZWR0D[LPL]H5HWXUQRQ'DWDZLWK'DWD9LUWXDOL]DWLRQ  +HUHDUHVRPHFULWLFDOIDFWRUVWKDWRUJDQL]DWLRQVPXVWFRQVLGHULQRUGHUWRPD[LPL]HWKHLUUHWXUQRQ GDWD  • 7KH\ PXVW EH PRUH EXVLQHVVIRFXVHG DQG WKLQN IURP D SHUVSHFWLYH RI DQ HQGWRHQG EXVLQHVVXVHRILQIRUPDWLRQDQGDVVRFLDWHGSURFHVVHV • 7KH\ PXVW HPSOR\ VHOIVHUYLFH DQG EXVLQHVV,7 FROODERUDWLRQ EHVW SUDFWLFHV WR HQDEOH WKH EXVLQHVVWRRZQWKHLUGDWDZKLOH,7UHWDLQVFRQWURODQGJRYHUQDQFH • 7KH\ PXVW HPSOR\ DJLOH GDWD LQWHJUDWLRQ WHFKQLTXHV WKDW WKH FXW ZDLW DQG ZDVWH LQ WKH SURFHVVZKLOHFRPSOHPHQWLQJWUDGLWLRQDODSSURDFKHVWR%, • 7KH\ QHHG WR GHILQH XVHUGULYHQ VHUYLFHOHYHO DJUHHPHQWV 6/$V  DURXQG GDWD ODWHQF\ DFFXUDF\FRQVLVWHQF\DYDLODELOLW\DQGXSWLPHDQGXQGHUVWDQGV\VWHPLPSOLFDWLRQV

'DWD9LUWXDOL]DWLRQIRU%XVLQHVV,QWHOOLJHQFH6\VWHPV²5HYROXWLRQL]LQJ'DWD,QWHJUDWLRQIRU'DWD:DUHKRXVHV &RS\ULJKW‹(OVHYLHU,QF$OOULJKWVUHVHUYHG



([FHUSWFUHDWHGIRU,QIRUPDWLFD&RUSRUDWLRQ&KDSWHU7KH)XWXUHRI'DWD9LUWXDOL]DWLRQ_  

 •

7KH\ PXVW XVH GDWD WR FRPSOHPHQW WKHLU EXVLQHVV VWUDWHJ\ DQG WHFKQLFDO LQIUDVWUXFWXUH LQVWHDGRIUHLQYHQWLQJWKHHQWLUHZKHHO

 2IODWHGDWDYLUWXDOL]DWLRQKDVHYROYHGDVDQDJLOHGDWDLQWHJUDWLRQFRQFHSWWRHQDEOHPRUHDJLOH%, +RZHYHU WUDGLWLRQDO %, DSSURDFKHV LQFOXGLQJ GDWD LQWHJUDWLRQ GDWD ZDUHKRXVLQJ DQG RWKHU FRPSOH[ GDWD SURFHVVLQJ LQLWLDWLYHV DUH QRW JRLQJ DZD\ DQ\WLPH VRRQ 7KLV LV EHFDXVH GDWD ZLOO FRQWLQXH WR EH KHWHURJHQHRXV GLUW\ DQG VHPDQWLFDOO\ GLIIHUHQW DFURVV V\VWHPV 7R VXFFHHG GDWD YLUWXDOL]DWLRQPXVWFRH[LVWUHXVHDQGFRPSOHPHQWH[LVWLQJLQIUDVWUXFWXUHDQGLQYHVWPHQWVPDGHWR VROYHWKHVHSUREOHPVUDWKHUWKDQEHD%DQG$LGIRUDVPDOOVXEVHWRIVSHFLDOXVHFDVHV,WPXVWDOVR LQYROYHWKHEXVLQHVVXVHUHDUO\DQGRIWHQWRHQVXUHWKDWWKHGDWDLVWUXVWZRUWK\  %H\RQG/RRNLQJ8QGHUWKH+RRG  7KHWULFNLVWRJDLQFRPSHWLWLYHDGYDQWDJHE\DFFHOHUDWLQJWKHGHOLYHU\RIFULWLFDOGDWDDQGUHSRUWV DQG EH DEOH WR WUXVW DQG FRQVXPH WKHP LQVWDQWO\ %XW GDWD YLUWXDOL]DWLRQ PXVW EH GRQH ULJKW WR VXSSRUW WKH FULWLFDO VXFFHVV IDFWRUV 9HU\ RIWHQ GDWD YLUWXDOL]DWLRQ ERUURZV KHDYLO\ IURP LWV GDWD IHGHUDWLRQOHJDF\7KHSULPDU\XVHFDVHGDWDIHGHUDWLRQGRHVZHOOLVWRDFFHVVDQGPHUJHDOUHDG\ FOHDQHGDQGFRQIRUPHGGDWDLQUHDOWLPHOHDYLQJWKHKHDY\OLIWLQJIRURWKHUSURFHVVLQJORJLFWRPDNH WKLV SRVVLEOH 6R WKH WLPH DGYDQWDJH JDLQHG LV ORVW DV RQH UHDOL]HV WKH IHGHUDWHG GDWD KDG WR EH SUHSSHGIRUIHGHUDWLRQ$VDUHVXOWWKH52,VLPSO\GLVDSSHDUV  6R GR JR EH\RQG ORRNLQJ XQGHU WKH KRRG DQG DVN D IHZ KDUG TXHVWLRQV 7R ZKDW H[WHQW GRHV WKH VROXWLRQ VXSSRUW GDWD WUDQVIRUPDWLRQ" ,V LW QRPLQDO OLPLWHG WR ZKDW \RX FDQ GR SURJUDPPDWLFDOO\ WKURXJK 64/ RU ;4XHU\" ,V WKHUH DQ\ GDWD SURILOLQJ RU ZLOO \RX UHTXLUH VWDJLQJ DQG IXUWKHU SURFHVVLQJ",VLWSURILOLQJRIERWKORJLFDQGVRXUFHVMXVWVRXUFHVRUQHLWKHU",VGDWDFOHDQVLQJDQG FRQIRUPLQJ VLPSOLVWLF KDQGFRGHG RUQRQH[LVWHQW" +RZ DERXW UHXVH" &DQ \RX TXLFNO\ DQG HDVLO\ UHXVH YLUWXDO YLHZV IRU DQ\ XVH FDVH LQFOXGLQJ EDWFK" 7R GR GDWD YLUWXDOL]DWLRQ ULJKW LW UHTXLUHV D GHHS DQG WKRURXJK XQGHUVWDQGLQJ RI YHU\ FRPSOH[ SUREOHPV WKDW H[LVW LQ WKH GDWD LQWHJUDWLRQ GRPDLQ  6RZKDW·VRXUSHUVSHFWLYH"6LPSO\SXWGDWDYLUWXDOL]DWLRQPXVWWDNHDSDJHIURPWKHZRUOGRIYLUWXDO PDFKLQHV 'DWD YLUWXDOL]DWLRQ PXVW GR WKH KHDY\ OLIWLQJ RI DFFHVVLQJ SURILOLQJ FOHDQVLQJ WUDQVIRUPLQJDQGGHOLYHULQJIHGHUDWHGGDWDWRDQGIURPDQ\DSSOLFDWLRQRQGHPDQG,WPXVWKDQGOH DOOWKHXQGHUO\LQJGDWDFRPSOH[LW\LQRUGHUWRSURYLGHFRQIRUPHGDQGWUXVWHGGDWDUHXVLQJWKHORJLF IRUHLWKHUEDWFKRUUHDOWLPHRSHUDWLRQZKHWKHUWKURXJK64/62$5(67-621RUQHZDFURQ\PV \HW WR EH VSHFLILHG 'DWD YLUWXDOL]DWLRQ PXVW EH EXLOW JURXQGXS RQ WKH FXW WKH ZDLW DQG ZDVWH EHVW SUDFWLFHVGLVFXVVHGLQWKHERRNRQ/HDQ,QWHJUDWLRQE\6FKPLGWDQG/\OH  %\VWDUWLQJZLWKDORJLFDOGDWDPRGHOJLYLQJ EXVLQHVVDQG ,7UROHEDVHGYLVLELOLW\LQWRWKHGDWDHDUO\ LQ WKH SURFHVV HQDEOLQJ GDWD SURILOLQJ RQ IHGHUDWHG GDWD WR VKRZ DQG UHVROYH WKH GDWD LQWHJULW\ LVVXHVDSSO\LQJDGYDQFHGWUDQVIRUPDWLRQVLQFOXGLQJGDWDTXDOLW\LQUHDOWLPHWRIHGHUDWHGGDWDDQG FRPSOHWHO\DQGLQVWDQWO\UHXVLQJWKHLQWHJUDWLRQORJLFRUYLUWXDOYLHZVIRUEDWFKRUUHDOWLPH\RXFDQ FXWWKHZDLWDQGZDVWHWKURXJKRXWWKHSURFHVV%\OHYHUDJLQJRSWLPL]DWLRQVSDUDOOHOLVPSLSHOLQLQJ LGHQWLW\ UHVROXWLRQ DQG RWKHU FRPSOH[ WUDQVIRUPDWLRQDO FDSDELOLWLHV \RX FDQ RQO\ ILQG LQ D PDWXUH GDWDLQWHJUDWLRQSODWIRUPGDWDYLUWXDOL]DWLRQFDQHQDEOHPRUHDJLOHEXVLQHVVLQWHOOLJHQFH  )LQDOO\ZLWKHQWHUSULVHVJHQHUDWLQJKXJHYROXPHVRIGDWDWKHW\SHVRIGDWDFKDQJLQJHQRUPRXVO\ DQGQHHGIRUVKRUWHUGDWDSURFHVVLQJVSHHGVGDWDYLUWXDOL]DWLRQFDQPD[LPL]HWKHUHWXUQRQGDWD