Time Series Analysis R

Analysis of Time Series Data Using R1 ZONGWU CAIa,b,c E-mail address: a

[email protected]

Department of Mathematics & Statistics and Department of Economics, University of North Carolina, Charlotte, NC 28223, U.S.A.

b c

Wang Yanan Institute for Studies in Economics, Xiamen University, China

College of Economics and Management, Shanghai Jiaotong University, China

July 30, 2006

c 2006, ALL RIGHTS RESERVED by ZONGWU CAI ! 1 This

manuscript may be printed and reproduced for individual or instructional use, but may not be printed for commercial purposes.

Preface The purpose of this lecture notes is designed to provide an overview of methods that are useful for analyzing univariate and multivariate phenomena measured over time. Since this is a course emphasizing applications with both theory and applications, the reader is guided through examples involving real time series in the lectures. A collection of simple theoretical and applied exercises assuming a background that includes a beginning level course in mathematical statistics and some computing skills follows each chapter. More importantly, the computer code in R and datasets are provided for most of examples analyzed in this lecture notes. Some materials are based on the lecture notes given by Professor Robert H. Shumway, Department of Statistics, University of California at Davis and my colleague, Professor Stanislav Radchenko, Department of Economics, University of North Carolina at Charlotte. Some datasets are provided by Professor Robert H. Shumway, Department of Statistics, University of California at Davis and Professor Phillips Hans Franses at University of Rotterdam, Netherland. I am very grateful to them for providing their lecture notes and datasets.

Contents 1 Package R and Simple Applications 1.1 Computational Toolkits . . . . . . . . . . . . . . . . . 1.2 How to Install R ? . . . . . . . . . . . . . . . . . . . . 1.3 Data Analysis and Graphics Using R – An Introduction 1.4 CRAN Task View: Empirical Finance . . . . . . . . . . 1.5 CRAN Task View: Computational Econometrics . . . . 2 Characteristics of Time Series 2.1 Introduction . . . . . . . . . . . . . . . . 2.2 Stationary Time Series . . . . . . . . . . 2.2.1 Detrending . . . . . . . . . . . . 2.2.2 Differencing . . . . . . . . . . . . 2.2.3 Transformations . . . . . . . . . . 2.2.4 Linear Filters . . . . . . . . . . . 2.3 Other Key Features of Time Series . . . 2.3.1 Seasonality . . . . . . . . . . . . 2.3.2 Aberrant Observations . . . . . . 2.3.3 Conditional Heteroskedasticity . . 2.3.4 Nonlinearity . . . . . . . . . . . . 2.4 Time Series Relationships . . . . . . . . 2.4.1 Autocorrelation Function . . . . . 2.4.2 Cross Correlation Function . . . . 2.4.3 Partial Autocorrelation Function 2.5 Problems . . . . . . . . . . . . . . . . . . 2.6 Computer Code . . . . . . . . . . . . . . 2.7 References . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . . . . . (109 . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

3 Univariate Time Series Models 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Least Squares Regression . . . . . . . . . . . . . . . . . . . 3.3 Model Selection Methods . . . . . . . . . . . . . . . . . . . 3.3.1 Subset Approaches . . . . . . . . . . . . . . . . . . 3.3.2 Sequential Methods . . . . . . . . . . . . . . . . . . 3.3.3 Likelihood Based-Criteria . . . . . . . . . . . . . . 3.3.4 Cross-Validation and Generalized Cross-Validation ii

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . pages) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

1 1 3 4 4 8

. . . . . . . . . . . . . . . . . .

15 15 17 21 24 26 28 30 30 33 34 36 38 39 40 45 49 53 66

. . . . . . .

69 69 74 79 79 83 85 87

CONTENTS

3.4 3.5

3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13

3.14 3.15 3.16

iii

3.3.5 Penalized Methods . . . . . . . . . . . . . . . . . . . . . . . Integrated Models - I(1) . . . . . . . . . . . . . . . . . . . . . . . . Autoregressive Models - AR(p) . . . . . . . . . . . . . . . . . . . . 3.5.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . Moving Average Models – MA(q) . . . . . . . . . . . . . . . . . . . Autoregressive Integrated Moving Average Model - ARIMA(p, d, q) Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . Regression Models With Correlated Errors . . . . . . . . . . . . . . Estimation of Covariance Matrix . . . . . . . . . . . . . . . . . . . Long Memory Models . . . . . . . . . . . . . . . . . . . . . . . . . . Periodicity and Business Cycles . . . . . . . . . . . . . . . . . . . . Impulse Response Function . . . . . . . . . . . . . . . . . . . . . . 3.13.1 First Order Difference Equations . . . . . . . . . . . . . . . 3.13.2 Higher Order Difference Equations . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Non-stationary Processes and Structural Breaks 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Inappropriate Detrending . . . . . . . . . . . . . . . . 4.2.2 Spurious (nonsense) Regressions . . . . . . . . . . . . . 4.3 Unit Root and Stationary Processes . . . . . . . . . . . . . . . 4.3.1 Comparison of Forecasts of TS and DS Processes . . . 4.3.2 Random Walk Components and Stochastic Trends . . . 4.4 Trend Estimation and Forecasting . . . . . . . . . . . . . . . . 4.4.1 Forecasting a Deterministic Trend . . . . . . . . . . . . 4.4.2 Forecasting a Stochastic Trend . . . . . . . . . . . . . . 4.4.3 Forecasting ARMA models with Deterministic Trends . 4.4.4 Forecasting of ARIMA Models . . . . . . . . . . . . . . 4.5 Unit Root Tests . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 The Dickey-Fuller and Augmented Dickey-Fuller Tests 4.5.2 Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Structural Breaks . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 Testing for Breaks . . . . . . . . . . . . . . . . . . . . 4.6.2 Zivot and Andrews’s Testing Procedure . . . . . . . . . 4.6.3 Cautions . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

88 90 93 93 99 102 106 108 120 130 133 136 141 142 146 150 157 182

. . . . . . . . . . . . . . . . . . . . . .

185 185 188 189 190 190 191 193 194 194 195 195 196 197 197 200 200 201 203 205 205 209 213

CONTENTS

iv

5 Vector Autoregressive Models 5.1 Introduction . . . . . . . . . . . . 5.1.1 Properties of VAR Models 5.1.2 Statistical Inferences . . . 5.2 Impulse-Response Function . . . 5.3 Variance Decompositions . . . . . 5.4 Granger Causality . . . . . . . . . 5.5 Forecasting . . . . . . . . . . . . 5.6 Problems . . . . . . . . . . . . . . 5.7 References . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

215 215 218 220 222 225 226 229 229 233

6 Cointegration 6.1 Introduction . . . . . . . . 6.2 Cointegrating Regression . 6.3 Testing for Cointegration . 6.4 Cointegrated VAR Models 6.5 Problems . . . . . . . . . . 6.6 References . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

234 234 235 236 239 242 243

7 Nonparametric Density, Distribution & Quantile Estimation 7.1 Mixing Conditions . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Density Estimate . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Asymptotic Properties . . . . . . . . . . . . . . . . . . . 7.2.2 Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.3 Boundary Correction . . . . . . . . . . . . . . . . . . . . 7.3 Distribution Estimation . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Smoothed Distribution Estimation . . . . . . . . . . . . 7.3.2 Relative Efficiency and Deficiency . . . . . . . . . . . . . 7.4 Quantile Estimation . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Value at Risk . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 Nonparametric Quantile Estimation . . . . . . . . . . . . 7.5 Computer Code . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

244 244 245 246 249 251 254 254 257 258 258 260 261 264

. . . . . . . . . .

267 267 267 268 270 271 273 274 277 279 279

. . . . . .

. . . . . .

. . . . . .

. . . . . .

8 Nonparametric Regression Estimation 8.1 Bandwidth Selection . . . . . . . . . . 8.1.1 Simple Bandwidth Selectors . . 8.1.2 Cross-Validation Method . . . . 8.2 Multivariate Density Estimation . . . . 8.3 Regression Function . . . . . . . . . . 8.4 Kernel Estimation . . . . . . . . . . . . 8.4.1 Asymptotic Properties . . . . . 8.4.2 Boundary Behavior . . . . . . . 8.5 Local Polynomial Estimate . . . . . . . 8.5.1 Formulation . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

CONTENTS

8.6

8.7

8.8 8.9

8.5.2 Implementation in R . . . . . . . . . . . . 8.5.3 Complexity of Local Polynomial Estimator 8.5.4 Properties of Local Polynomial Estimator 8.5.5 Bandwidth Selection . . . . . . . . . . . . Functional Coefficient Model . . . . . . . . . . . . 8.6.1 Model . . . . . . . . . . . . . . . . . . . . 8.6.2 Local Linear Estimation . . . . . . . . . . 8.6.3 Bandwidth Selection . . . . . . . . . . . . 8.6.4 Smoothing Variable Selection . . . . . . . 8.6.5 Goodness-of-Fit Test . . . . . . . . . . . . 8.6.6 Asymptotic Results . . . . . . . . . . . . . 8.6.7 Conditions and Proofs . . . . . . . . . . . 8.6.8 Monte Carlo Simulations and Applications Additive Model . . . . . . . . . . . . . . . . . . . 8.7.1 Model . . . . . . . . . . . . . . . . . . . . 8.7.2 Backfitting Algorithm . . . . . . . . . . . 8.7.3 Projection Method . . . . . . . . . . . . . 8.7.4 Two-Stage Procedure . . . . . . . . . . . . 8.7.5 Monte Carlo Simulations and Applications Computer Code . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . .

v . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

280 281 284 289 292 292 293 294 296 296 299 301 311 311 311 315 317 319 322 322 326

List of Tables 3.1

AICC values for ten models for the recruits series . . . . . . . . . . . . . . .

4.1 4.2 4.3

Large-sample critical values for the ADF statistic . . . . . . . . . . . . . . . 198 Summary of DF test for unit roots in the absence of serial correlation . . . . 199 Critical Values of the QLR statistic with 15% Trimming . . . . . . . . . . . 203

5.1 5.2

Sims variance decomposition in three variable VAR model . . . . . . . . . . 228 Sims variance decomposition including interest rates . . . . . . . . . . . . . . 228

6.1

Critical values for the Engle-Granger ADF statistic . . . . . . . . . . . . . . 238

8.1

Sample sizes required for p-dimensional nonparametric regression to have comparable performance with that of 1-dimensional nonparametric regression using size 100 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

vi

98

List of Figures 2.1 2.2 2.3 2.4

2.5 2.6 2.7 2.8 2.9

2.10 2.11 2.12 2.13 2.14

2.15 2.16 2.17

Monthly SOI (left) and simulated recruitment (right) from a model (n=453 months, 1950-1987). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simulated MA(1) with θ1 = 0.9. . . . . . . . . . . . . . . . . . . . . . . . . . Log of annual indices of real national output in China, 1952-1988. . . . . . . Monthly average temperature in degrees centigrade, January, 1856 - February 2005, n = 1790 months. The straight line (wide and green) is the linear trend y = −9.037 + 0.0046 t and the curve (wide and red) is the nonparametric estimated trend. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Detrended monthly global temperatures: left panel (linear) and right panel (nonlinear). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Differenced monthly global temperatures. . . . . . . . . . . . . . . . . . . . . Annual stock of motor cycles in the Netherlands, 1946-1993. . . . . . . . . . Quarterly earnings for Johnson & Johnson (4th quarter, 1970 to 1st quarter, 1980, left panel) with log transformed earnings (right panel). . . . . . . . . . The SOI series (black solid line) compared with a 12 point moving average (red thicker solid line). The top panel: original data and the bottom panel: filtered series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . US Retail Sales Data from 1967-2000. . . . . . . . . . . . . . . . . . . . . . . Four-weekly advertising expenditures on radio and television in The Netherlands, 1978.01 − 1994.13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . First difference in log prices versus the inflation rate: the case of Argentina, 1970.1 − 1989.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Japanese - U.S. dollar exchange rate return series {yt }, from January 1, 1974 to December 31, 2003. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Quarterly unemployment rate in Germany, 1962.1 − 1991.4 (seasonally adjusted and not seasonally adjusted) in the left panel. The scatterplot of unemployment rate (seasonally adjusted) versus unemployment rate (seasonally adjusted) one period lagged in the right panel. . . . . . . . . . . . . . . . . . Multiple lagged scatterplots showing the relationship between SOI and the present (xt ) versus the lagged values (xt+h ) at lags 1 ≤ h ≤ 16. . . . . . . . . Autocorrelation functions of SOI and recruitment and cross correlation function between SOI and recruitment. . . . . . . . . . . . . . . . . . . . . . . . Multiple lagged scatterplots showing the relationship between the SOI at time t + h, say xt+h (x-axis) versus recruits at time t, say yt (y-axis), 0 ≤ h ≤ 15. vii

18 21 22

23 24 25 26 27

29 31 32 33 36

37 39 41 42

LIST OF FIGURES 2.18 Multiple lagged scatterplots showing the relationship between the SOI at time t, say xt (x-axis) versus recruits at time t + h, say yt+h (y-axis), 0 ≤ h ≤ 15. 2.19 Partial autocorrelation functions for the SOI (left panel) and the recruits (right panel) series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.20 Varve data for Problem 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.21 Gas and oil series for Problem 6. . . . . . . . . . . . . . . . . . . . . . . . . 2.22 Handgun sales (per 10,000,000) in California and monthly gun death rate (per 100,00) in California (February 2, 1980 -December 31, 1998. . . . . . . . . . Autocorrelation functions (ACF) for simple (left) and log (right) returns for IBM (top panels) and for the value-weighted index of US market (bottom panels), January 1926 to December 1997. . . . . . . . . . . . . . . . . . . . . 3.2 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) for the detrended (top panel) and differenced (bottom panel) global temperature series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 A typical realization of the random walk series (left panel) and the first difference of the series (right panel). . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Autocorrelation functions (ACF) (left) and partial autocorrelation functions (PACF) (right) for the random walk (top panel) and the first difference (bottom panel) series. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Autocorrelation (ACF) of residuals of AR(1) for SOI (left panel) and the plot of AIC and AICC values (right panel). . . . . . . . . . . . . . . . . . . . . . 3.6 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) for the log varve series (top panel) and the first difference (bottom panel), showing a peak in the ACF at lag h = 1. . . . . . . . . . . . . . . . . . . . . 3.7 Number of live births 1948(1) − 1979(1) and residuals from models with a first difference, a first difference and a seasonal difference of order 12 and a fitted ARIMA(0, 1, 1) × (0, 1, 1)12 model. . . . . . . . . . . . . . . . . . . . . . . . 3.8 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) for the birth series (top two panels), the first difference (second two panels) an ARIMA(0, 1, 0)×(0, 1, 1)12 model (third two panels) and an ARIMA(0, 1, 1)× (0, 1, 1)12 model (last two panels). . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) for the log J&J earnings series (top two panels), the first difference (second two panels), ARIMA(0, 1, 0) × (1, 0, 0)4 model (third two panels), and ARIMA(0, 1, 1) × (1, 0, 0)4 model (last two panels). . . . . . . . . . . . . . . 3.10 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) for ARIMA(0, 1, 1)×(0, 1, 1)4 model (top two panels) and the residual plots of ARIMA(0, 1, 1) × (1, 0, 0)4 (left bottom panel) and ARIMA(0, 1, 1) × (0, 1, 1)4 model (right bottom panel). . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11 Monthly simple return of CRSP Decile 1 index from January 1960 to December 2003: Time series plot of the simple return (left top panel), time series plot of the simple return after adjusting for January effect (right top panel), the ACF of the simple return (left bottom panel), and the ACF of the adjusted simple return. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

viii

42 46 50 51 53

3.1

72

77 91

92 99

104

111

112

115

116

119

LIST OF FIGURES 3.12 Autocorrelation functions (ACF) and partial autocorrelation functions (PACF) for the detrended log J&J earnings series (top two panels)and the fitted ARIMA(0, 0, 0) × (1, 0, 0)4 residuals. . . . . . . . . . . . . . . . . . . . . . . 3.13 Time plots of U.S. weekly interest rates (in percentages) from January 5, 1962 to September 10, 1999. The solid line (black) is the Treasury 1-year constant maturity rate and the dashed line the Treasury 3-year constant maturity rate (red). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.14 Scatterplots of U.S. weekly interest rates from January 5, 1962 to September 10, 1999: the left panel is 3-year rate versus 1-year rate, and the right panel is changes in 3-year rate versus changes in 1-year rate. . . . . . . . . . . . . 3.15 Residual series of linear regression Model I for two U.S. weekly interest rates: the left panel is time plot and the right panel is ACF. . . . . . . . . . . . . . 3.16 Time plots of the change series of U.S. weekly interest rates from January 12, 1962 to September 10, 1999: changes in the Treasury 1-year constant maturity rate are in denoted by black solid line, and changes in the Treasury 3-year constant maturity rate are indicated by red dashed line. . . . . . . . . . . . . 3.17 Residual series of the linear regression models: Model II (top) and Model III (bottom) for two change series of U.S. weekly interest rates: time plot (left) and ACF (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.18 Sample autocorrelation function of the absolute series of daily simple returns for the CRSP value-weighted (left top panel) and equal-weighted (right top panel) indexes. The log spectral density of the absolute series of daily simple returns for the CRSP value-weighted (left bottom panel) and equal-weighted (right bottom panel) indexes. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.19 The autocorrelation function of an AR(2) model: (a) φ1 = 1.2 and φ2 = −0.35, (b) φ1 = 1.0 and φ2 = −0.7, (c) φ1 = 0.2 and φ2 = 0.35, (d) φ1 = −0.2 and φ2 = 0.35. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.20 The growth rate of US quarterly real GNP from 1947.II to 1991.I (seasonally adjusted and in percentage): the left panel is the time series plot and the right panel is the ACF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.21 The time-series yt is generated with wt ∼ N (0, 1), y0 = 5. At period t = 50, there is an additional impulse to the error term, i.e. w!50 = w50 + 1. The impulse response function is computed as the difference between the series yt without impulse and the series y!t with the impulse. . . . . . . . . . . . . . . 3.22 The time-series yt is generated with wt ∼ N (0, 1), y0 = 3. At period t = 50, there is an additional impulse to the error term, i.e. w!50 = w50 + 1. The impulse response function is computed as the difference between the series yt without impulse and the series y!t with the impulse. . . . . . . . . . . . . . . 3.23 Example of impulse response functions for first order difference equations. . . 3.24 The time series yt is generated with wt ∼ N (0, 1), y0 = 3. For the transitory impulse, there is an additional impulse to the error term at period t = 50, i.e. w!50 = w50 + 1. For the permanent impulse, there is an additional impulse for period t = 50, · · ·, 100, i.e. w!t = wt + 1, t = 50, 51, · · ·, 100. The impulse response function (IRF) is computed as the difference between the series yt without impulse and the series y!t with the impulse. . . . . . . . . . . . . . .

ix

123

125

125 126

127

127

135

139

139

143

144 146

147

LIST OF FIGURES

x

3.25 Example of impulse response functions for second order difference equation. . 149

Chapter 1 Package R and Simple Applications 1.1

Computational Toolkits

When you work with large datasets, messy data handling, models, etc, you need to choose the computational tools that are useful for dealing with these kinds of problems. There are “menu driven systems” where you click some buttons and get some work done - but these are useless for anything nontrivial. To do serious economics and finance in the modern days, you have to write computer programs. And this is true of any field, for example, empirical macroeconomics - and not just of “computational finance” which is a hot buzzword recently. The question is how to choose the computational tools. According to Ajay Shah (December 2005), you should pay attention to four elements: price, freedom, elegant and powerful computer science, and network effects. Low price is better than high price. Price = 0 is obviously best of all. Freedom here is in many aspects. A good software system is one that doesn’t tie you down in terms of hardware/OS, so that you are able to keep moving. Another aspect of freedom is in working with colleagues, collaborators and students. With commercial software, this becomes a problem, because your colleagues may not have the same software that you are using. Here 1

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

2

free software really wins spectacularly. Good practice in research involves a great accent on reproducibility. Reproducibility is important both so as to avoid mistakes, and because the next person working in your field should be standing on your shoulders. This requires an ability to release code. This is only possible with free software. Systems like SAS and Gauss use archaic computer science. The code is inelegant. The language is not powerful. In this day and age, writing C or fortran by hand is “too low level”. Hell, with Gauss, even a minimal thing like online help is tawdry. One prefers a system to be built by people who know their computer science - it should be an elegant, powerful language. All standard CS knowledge should be nicely in play to give you a gorgeous system. Good computer science gives you more productive humans. Lots of economists use Gauss, and give out Gauss source code, so there is a network effect in favor of Gauss. A similar thing is right now happening with statisticians and R. Here I cite comparisons among most commonly used packages (see Ajay Shah (December 2005)); see the web site at http://www.mayin.org/ajayshah/ COMPUTING/mytools.html. R is a very convenient programming language for doing statistical analysis and Monte Carol simulations as well as various applications in quantitative economics and finance. Indeed, we prefer to think of it of an environment within which statistical techniques are implemented. I will teach it at the introductory level, but NOTICE that you will have to learn R on your own. Note that about 97% of commands in S-PLUS and R are same. In particular, for analyzing time series data, R has a lot of bundles and packages, which can be downloaded for free, for example, at http://www.r-project.org/.

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

3

R, like S, is designed around a true computer language, and it allows users to add additional functionality by defining new functions. Much of the system is itself written in the R dialect of S, which makes it easy for users to follow the algorithmic choices made. For computationally-intensive tasks, C, C++ and Fortran code can be linked and called at run time. Advanced users can write C code to manipulate R objects directly. 1.2

How to Install R ?

(1) go to the web site http://www.r-project.org/; (2) click CRAN; (3) choose a site for downloading, say http://cran.cnr.Berkeley.edu; (4) click Windows (95 and later); (5) click base; (6) click R-2.3.1-win32.exe (Version of 06-01-2006) to save this file first and then run it to install. The basic R is installed into your computer. If you need to install other packages, you need to do the followings: (7) After it is installed, there is an icon on the screen. Click the icon to get into R; (8) Go to the top and find packages and then click it; (9) Go down to Install package(s)... and click it; (10) There is a new window. Choose a location to download the packages, say USA(CA1), move mouse to there and click OK; (11) There is a new window listing all packages. You can select any one of packages and click OK, or you can select all of them and then click OK.

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

1.3

4

Data Analysis and Graphics Using R – An Introduction (109 pages)

See the file r-notes.pdf (109 pages) which can be downloaded from http://www.math.uncc.edu/˜ zcai/r-notes.pdf. I encourage you to download this file and learn it by yourself. 1.4

CRAN Task View: Empirical Finance

This CRAN Task View contains a list of packages useful for empirical work in Finance, grouped by topic. Besides these packages, a very wide variety of functions suitable for empirical work in Finance is provided by both the basic R system (and its set of recommended core packages), and a number of other packages on the Comprehensive R Archive Network (CRAN). Consequently, several of the other CRAN Task Views may contain suitable packages, in particular the Econometrics Task View. The web site is http://cran.r-project.org/src/contrib/Views/Finance.html

1. Standard regression models: Linear models such as ordinary least squares (OLS) can be estimated by lm() (from by the stats package contained in the basic R distribution). Maximum Likelihood (ML) estimation can be undertaken with the optim() function. Non-linear least squares can be estimated with the nls() function, as well as with nlme() from the nlme package. For the linear model, a variety of regression diagnostic tests are provided by the car, lmtest, strucchange, urca, uroot, and sandwich packages. The Rcmdr and Zelig packages provide user interfaces that may be of interest as well.

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

5

2. Time series: Classical time series functionality is provided by the arima() and KalmanLike() commands in the basic R distribution. The dse packages provides a variety of more advanced estimation methods; fracdiff can estimate fractionally integrated series; longmemo covers related material. For volatily modeling, the standard GARCH(1,1) model can be estimated with the garch() function in the tseries package. Unit root and cointegration tests are provided by tseries, urca and uroot. The Rmetrics packages fSeries and fMultivar contain a number of estimation functions for ARMA, GARCH, long memory models, unit roots and more. The ArDec implements autoregressive time series decomposition in a Bayesian framework. The dyn and dynlm are suitable for dynamic (linear) regression models. Several packages provide wavelet analysis functionality: rwt, wavelets, waveslim, wavethresh. Some methods from chaos theory are provided by the package tseriesChaos. 3. Finance: The Rmetrics bundle comprised of the fBasics, fCalendar, fSeries, fMultivar, fPortfolio, fOptions and fExtremes packages contains a very large number of relevant functions for different aspect of empirical and computational finance. The RQuantLib package provides several optionpricing functions as well as some fixed-income functionality from the QuantLib project to R. The portfolio package contains classes for equity portfolio management. 4. Risk Management: The VaR package estimates Value-atRisk, and several packages provide functionality for Extreme Value Theory models: evd, evdbayes, evir, extRremes, ismec, POT. The mvtnorm package provides code for mul-

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

6

tivariate Normal and t-distributions. The Rmetrics packages fPortfolio and fExtremes also contain a number of relevant functions. The copula and fgac packages cover multivariate dependency structures using copula methods. 5. Data and Date Management: The its, zoo and fCalendar (part of Rmetrics) packages provide support for irregularlyspaced time series. fCalendar also addresses calendar issues such as recurring holidays for a large number of financial centers, and provides code for high-frequency data sets. CRAN packages: * ArDec * car * copula * dse * dyn * dynlm * evd * evdbayes * evir * extRemes * fBasics (core) * fCalendar (core) * fExtremes (core) * fgac

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

* fMultivar (core) * fOptions (core) * fPortfolio (core) * fracdiff * fSeries (core) * ismev * its (core) * lmtest * longmemo * mvtnorm * portfolio * POT * Rcmdr * RQuantLib (core) * rwt * sandwich * strucchange * tseries (core) * tseriesChaos * urca (core) * uroot * VaR * wavelets * waveslim * wavethresh

7

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

8

* Zelig * zoo (core) Related links: * CRAN Task View: Econometrics. The web site is http://cran.cnr.berkeley.edu/src/ contrib/Views/Econometrics.html or see the next section. * Rmetrics by Diethelm Wuertz contains a wealth of R code for Finance. The web site is http://www.itp.phys.ethz.ch/econophysics/R/ * Quantlib is a C++ library for quantitative finance. The web site is http://quantlib.org/ * Mailing list: R Special Interest Group Finance 1.5

CRAN Task View: Computational Econometrics

Base R ships with a lot of functionality useful for computational econometrics, in particular in the stats package. This functionality is complemented by many packages on CRAN, a brief overview is given below. There is also a considerable overlap between the tools for econometrics in this view and finance in the Finance view. Furthermore, the finance SIG is a suitable mailing list for obtaining help and discussing questions about both computational finance and econometrics. The packages in this view can be roughly structured into the following topics. The web site is

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

9

http://cran.r-project.org/src/contrib/ Views/Econometrics.html 1. Linear regression models: Linear models can be fitted (via OLS) with lm() (from stats) and standard tests for model comparisons are available in various methods such as summary() and anova(). Analogous functions that also support asymptotic tests (z instead of t tests, and Chi-squared instead of F tests) and plug-in of other covariance matrices are coeftest() and waldtest() in lmtest. Tests of more general linear hypotheses are implemented in linear.hypothesis() in car. HC and HAC covariance matrices that can be plugged into these functions are available in sandwich. The packages car and lmtest also provide a large collection of further methods for diagnostic checking in linear regression models. 2. Microeconometrics: Many standard micro-econometric models belong to the family of generalized linear models (GLM) and can be fitted by glm() from package stats. This includes in particular logit and probit models for modelling choice data and poisson models for count data. Negative binomial GLMs are available via glm.nb() in package MASS from the VR bundle. Zero-inflated count models are provided in zicounts. Further over-dispersed and inflated models, including hurdle models, are available in package pscl. Bivariate poisson regression models are implemented in bivpois. Basic censored regression models (e.g., tobit models) can be fitted by survreg() in survival. Further more refined tools for microecnometrics are provided in micEcon. The package bayesm implements a Bayesian approach to microeconometrics and marketing. Inference for relative distributions is contained in package reldist.

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

10

3. Further regression models: Various extensions of the linear regression model and other model fitting techniques are available in base R and several CRAN packages. Nonlinear least squares modelling is available in nls() in package stats. Relevant packages include quantreg (quantile regression), sem (linear structural equation models, including two-stage least squares), systemfit (simultaneous equation estimation), betareg (beta regression), nlme (nonlinear mixed-effect models), VR (multinomial logit models in package nnet) and MNP (Bayesian multinomial probit models). The packages Design and Hmisc provide several tools for extended handling of (generalized) linear regression models. 4. Basic time series infrastructure: The class ts in package stats is R’s standard class for regularly spaced time series which can be coerced back and forth without loss of information to zooreg from package zoo. zoo provides infrastructure for both regularly and irregularly spaced time series (the latter via the class “zoo” ) where the time information can be of arbitrary class. Several other implementations of irregular time series building on the “POSIXt” time-date classes are available in its, tseries and fCalendar which are all aimed particularly at finance applications (see the Finance view). 5. Time series modelling: Classical time series modelling tools are contained in the stats package and include arima() for ARIMA modelling and Box-Jenkins-type analysis. Furthermore stats provides StructTS() for fitting structural time series and decompose() and HoltWinters() for time series filtering and decomposition. For estimating VAR models, several methods are available: simple models can be fitted by ar() in stats, more

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

11

elaborate models are provided by estVARXls() in dse and a Bayesian approach is available in MSBVAR. A convenient interface for fitting dynamic regression models via OLS is available in dynlm; a different approach that also works with other regression functions is implemented in dyn. More advanced dynamic system equations can be fitted using dse. Unit root and cointegration techniques are available in urca, uroot and tseries. Time series factor analysis is available in tsfa. 6. Matrix manipulations: As a vector- and matrix-based language, base R ships with many powerful tools for doing matrix manipulations, which are complemented by the packages Matrix and SparseM. 7. Inequality: For measuring inequality, concentration and poverty the package ineq provides some basic tools such as Lorenz curves, Pen’s parade, the Gini coefficient and many more. 8. Structural change: R is particularly strong when dealing with structural changes and changepoints in parametric models, see strucchange and segmented. 9. Data sets: Many of the packages in this view contain collections of data sets from the econometric literature and the package Ecdat contains a complete collection of data sets from various standard econometric textbooks. micEcdat provides several data sets from the Journal of Applied Econometrics and the Journal of Business & Economic Statistics data archives. Package CDNmoney provides Canadian monetary aggregates and pwt provides the Penn world table. CRAN packages:

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

* bayesm * betareg * bivpois * car (core) * CDNmoney * Design * dse * dyn * dynlm * Ecdat * fCalendar * Hmisc * ineq * its * lmtest (core) * Matrix * micEcdat * micEcon * MNP * MSBVAR * nlme * pscl * pwt * quantreg

12

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

* reldist * sandwich (core) * segmented * sem * SparseM * strucchange * systemfit * tseries (core) * tsfa * urca (core) * uroot * VR * zicounts * zoo (core) Related links: * CRAN Task View: Finance. The web site is http://cran.cnr.berkeley.edu/src/contrib/ Views/Finance.html or see the above section. * Mailing list: R Special Interest Group Finance * A Brief Guide to R for Beginners in Econometrics. The web site is http://people.su.se/˜ ma/R−intro/.

13

CHAPTER 1. PACKAGE R AND SIMPLE APPLICATIONS

* R for Economists. The web site is http://www.mayin.org/ajayshah/KB/R/ R−for−economists.html.

14

Chapter 2 Characteristics of Time Series 2.1

Introduction

The very nature of data collected in different fields as as diverse as economics, finance, biology, medicine, and engineering leads one naturally to a consideration of time series models. Samples taken from all of these disciplines are typically observed over a sequence of time periods. Often, for example, one observes hourly or daily or monthly or yearly data, even tick-by-tick trade data, and it is clear from examining the histories of such series over a number of time periods that the adjacent observations are by no means independent. Hence, the usual techniques from classical statistics, developed primarily for independent identically distributed (iid) observations, are not applicable. Clearly, we can not hope to give a complete accounting of the theory and applications of time series in the limited time to be devoted to this course. Therefore, what we will try to accomplish, in this presentation is a considerably more modest set of objectives, with more detailed references quoted for discussions in depth. First, we will attempt to illustrate the kinds of time series analyses that can arise in scientific contexts, particularly, in economics and finance, and give examples of applications using real data. This necessarily 15

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

16

will include exploratory data analysis using graphical displays and numerical summaries such as the autocorrelation and cross correlation functions. The use of scatter diagrams and various linear and nonlinear transformations also will be illustrated. We will define classical time series statistics for measuring the patterns described by time series data. For example, the characterization of consistent trend profiles by dynamic linear or quadratic regression models as well as the representation of periodic patterns using spectral analysis will be illustrated. We will show how one might go about examining plausible patterns of cause and effect, both within and among time series. Finally, some time series models that are particularly useful such as regression with correlated errors as well as multivariate autoregressive and state-space models will be developed, together with unit root, co-integration, and nonlinear time series models, and some other models. Forms of these models that appear to offer hope for applications will be emphasized. It is recognized that a discussion of the models and techniques involved is not enough if one does not have available the requisite resources for carrying out time series computations; these can be formidable. Hence, we include a computing package, called R. In this chapter, we will try to minimize the use of mathematical notation throughout the discussions and will not spend time developing the theoretical properties of any of the models or procedures. What is important for this presentation is that you, the reader, can gain a modest understanding as well as having access to some of the principal techniques of time series analysis. Of course, we will refer to Hamilton (1994) for additional references or more complete discussions relating to an application or principle and will discuss them in detail.

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

2.2

17

Stationary Time Series

We begin by introducing several environmental and economic as well as financial time series to serve as illustrative data for time series methodology. Figure 2.1 shows monthly values of an environmental series called the Southern Oscillation Index (SOI) and associated recruitment (number of new fish) computed from a model by Pierre Kleiber, Southwest Fisheries Center, La Jolla, California. Both series are for a period of 453 months ranging over the years 1950 − 1987. The SOI measures changes in air pressure that are related to sea surface temperatures in the central Pacific. The central Pacific Ocean warms up every three to seven years due to the El Ni˜no effect which has been blamed, in particular, for foods in the midwestern portions of the U.S. Both series in Figure 2.1 tend to exhibit repetitive behavior, with regularly repeating (stochastic) cycles that are easily visible. This periodic behavior is of interest because underlying processes of interest may be regular and the rate or frequency of oscillation characterizing the behavior of the underlying series would help to identify them. One can also remark that the cycles of the SOI are repeating at a faster rate than those of the recruitment series. The recruit series also shows several kinds of oscillations, a faster frequency that seems to repeat about every 12 months and a slower frequency that seems to repeat about every 50 months. The study of the kinds of cycles and their strengths will be discussed later. The two series also tend to be somewhat related; it is easy to imagine that somehow the fish population is dependent on the SOI. Perhaps there is even a lagged relation, with the SOI signalling changes in the fish population. The study of the variation in the different kinds of cyclical behav-

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

18

ior in a time series can be aided by computing the power spectrum which shows the variance as a function of the frequency of oscillation. Comparing the power spectra of the two series would then give valuable information relating to the relative cycles driving each one. One might also want to know whether or not the cyclical variations of a particular frequency in one of the series, say the SOI, are associated with the frequencies in the recruitment series. This would be measured by computing the correlation as a function of frequency, called the coherence. The study of systematic periodic variations Recruit

0

−1.0

20

−0.5

40

0.0

60

0.5

80

1.0

100

Southern Oscillation Index

0

100

200

300

400

0

100

200

300

400

Figure 2.1: Monthly SOI (left) and simulated recruitment (right) from a model (n=453 months, 1950-1987).

in time series is called spectral analysis. See Shumway (1988) and Shumway and Stoffer (2001) for details. We will need a characterization for the kind of stability that is exhibited by the environmental and fish series. One can note that the two series seem to oscillate fairly regularly around central values (0 for SOI and 64 for recruitment). Also, the lengths of the cycles and their orientations relative to each other do not seem to be changing drastically over the time histories. In order to describe this in a simple mathematical way, it is con-

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

19

venient to introduce the concept of a stationary time series. Suppose that we let the value of the time series at some time point t be denoted by {xt}. Then, the observed values can be represented as x1, the initial time point, x2, the second time point and so forth out to xn, the last observed point. A stationary time series is one for which the statistical behavior of xt1 , xt2 , . . . , xtk is identical to that of the shifted set xt1+h, xt2+h, . . . , xtk +h for any collection of time points t1, t2, . . ., tk and for any shift h. This means that all of the multivariate probability density functions for subsets of variables must agree with their counterparts in the shifted set for all values of the shift parameter h. This is called strictly strong stationary, which can be regarded as a mathematical assumption. The above version of stationarity is too strong for most applications and is difficult or impossible to be verified statistically in applications. Therefore, to relax this mathematical assumption, we will use a weaker version, called weak stationarity or covariance stationarity, which requires only that first and second moments satisfy the constraints. This implies that E(xt) = µ and E[(xt+h − µ)(xt − µ)] = γx(h),

(2.1)

where E denotes expectation or averaging over the population densities and h is the shift or lag. This implies, first, that the mean value function does not change over time and that γx(h), the population covariance function, is the same as long as the points are separated by a constant shift h. Estimators for the population covariance are important diagnostic tools for time correlation as we shall see later. When we use the term stationary time series in the sequel, we mean weakly stationary as defined by (2.1). The autocorrelation function (ACF) is defined as a scaled version of (2.1) and is written as ρx(h) = γx(h)/γx(0),

(2.2)

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

20

which is always between −1 and 1. The denominator of (2.2) is the mean square error or variance of the series since γx(0) = E[(xt −µ)2]. Exercise: For given time series {xt}nt=1, how do you check whether the time series {xt} is weakly or strong stationary? Thank about this problem. Example 1.1: We introduce in this example a simple example of a time domain model to be considered in detail later. A simple moving average model assumes that the series xt is generated from linear combinations of independent or uncorrelated “shocks” wt, sometimes called white noise1 (WN), to the system. For example, the simple first order moving average series xt = wt − 0.9 wt−1 is stationary when the inputs {wt} are assumed independent with E(wt) = 0 and E(wt2) = 1. It can be easily verified that E(xt) = 0 and γx(h) = 1 + 0.92 if h = 0, −0.9 if h = ±1, 0 if h > 1 (please verify this). We can see what such a series might look like by drawing random numbers wt from a standard normal distribution and then computing the values of xt. One such simulated series is shown in Figure 2.2 for n = 200 values; the series resembles vaguely the real data in the bottom panel of Figure 2.1. Many of our techniques are based on the idea that a suitably modified time series can be regarded as stationary (weakly). This requires first that the mean value function be constant as in (2.1). Several simple commonly occurring nonstationary time series can be illustrated by letting this assumption be violated. For example, the 1

white noise is defined as a sequence of uncorrelated ransom variables with mean zero and same variance.

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

21

−3 −2 −1

0

1

2

3

Simulated MA(1)

0

50

100

150

200

Figure 2.2: Simulated MA(1) with θ1 = 0.9.

series yt = t + xt, where xt is the moving average series of Example 1.1, will be nonstationary because E(xt) = t and the constant mean assumption of (2.1) is clearly violated. Four techniques for modifying the given series to improve the approximation to stationarity are detrending, differencing, transformations, and linear filtering as discussed below. A simple example of a nonstationary series is also given later. 2.2.1

Detrending

One of the dominant features on many economic and business time series is the trend. Such a trend can be upward or downward, it can be steep or not, and it can be exponential or approximately linear. Since a trend should be definitely somehow be incorporated in a time series model, simply because it can be exploited for out-ofsample forecasting, an analysis of trend behavior typically requires quite some research input. The discussion later will show that the type of trend has an important impact on forecasting. The general version of the nonstationary time series given above is

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

22

to assume a general trend of the form yt = Tt + xt, particularly, the linear trend Tt = β1 + β2 t. If one looks for a method of modifying the above series to achieve stationarity, it is natural to consider the residual x" t = yt − T#t = yt − β#1 − β#2 t

as a plausible stationary series where β#1 and β#2 are the estimated intercept and slope of the least squares line for yt as a function of t. The use of the residual or detrended series is common and the process of constructing residual is known as detrending.

8

Example 1.2: To illustrate the presence of trends in economic data, consider the five graphs in Figure 2.3, which are the annual indices of

5

6

7

agriculture commerce consumption industry transport

1955

1960

1965

1970

1975

1980

1985

Figure 2.3: Log of annual indices of real national output in China, 1952-1988.

real national output (in logs) in China in five different sectors for the sample period 1952 − 1988. These sectors are agriculture, industry, construction, transportation, and commerce. From this figure, it can be observed that the five sectors have grown over the year at different rates, and also that the five sectors seem to have been affected by the, likely, exogenous shocks to the Chinese economy around 1958 and 1968. These shocks roughly correspond

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

23

to the two major political movements in China: the Great-LeapForward around 1958 until 1962 and the Cultural Revolution from 1966 to 1976. It also appears from the graphs that these political movements may not have affected each of the five sectors in a similar fashion. For example, the decline of the output in the construction sector in 1961 seems much larger than that in the industry sector in the same year. It also seems that the Great-Leap-Forward shock already had an impact on the output in the agriculture sector as early as 1959. To quantify the trends in the five Chinese output series, one might consider a simple regression model with a linear trend as mentioned earlier or some more complex models. Example 1.3: As another more interesting example, consider the global temperature series given in Figure 2.4. There appears to be an increasing trend in global temperature which may signal global Original Data with Linear and Nonlinear Trend

−1.0

−0.5

0.0

0.5

o o o o o o o o o oo o o o o o o oo o oo o o o o o o ooo oo o o oo o o o o oo o o o o o o o o o o o o oooo o o o oo o o oo o o o o o o o o o oo o o o o o o o oo o o o o o o o o o o o o o o oo oo o o o o o oo o o o o o o o o o o oo oo o o o o o o o o o o o o o o o o o o o oo o o o oo o o oo o o o o o o o o o o o o o o o oo oo o o o oo o oo o o o o oo o oo o o o o oo o o o o o o oo o o o o ooo o o oo o oo o o o o o o o o o o o o o o o o o oo o o o o o o o o o oo o oo o o o o o o o o o o oo oo o o o o o o o o o o o oo o o o o o o oo o o oo oo oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o ooo o o o o o o o o oo o o o ooo o o o o o o o o o o o o o o oo o o oo o o o oo o o o oo o o oo o o o o o o o o oo o o o o o oo oo o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o oo o o o oo oo o oo o o o o o o o o o o o o o o o o o o o o o o o o oo oo o o o o oo o o oo o o o o o o oo o o o o o o o o o o o o o o oo o oo o oo o o oo oooo o o o oo o o o o o o o oo o o o o o oo o oo oo o o o o oo o o o oo o o oo o o o o o o o o oo oo o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o oo o oo oo o o o o o o o o o o o o o ooooo oo o o o o o o o o o o oo o o o o o o o oo o o o o o o o o o oo o oo o o o o o o o o o o o oo oo o oo o oo oo o o o o o o o o o oo o o o o o o o o o o o oo o o o o o oo o o o o o o o o o o oo o o o o o oo o o o o o o oo o o o o o o ooo o o oo o o o o o o o o oo o o o o o o o o o ooo oo o o o o o o o o o o o o o o o oo o o o o oo o ooo o o o o oo o o o o oo o o o o o o o o ooo o o o oo o o o o o o o oo o oo o o oo o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o oo oo oo o o o oo o o o o o o o o o o oo o oo o oo o o o o o oo o o o o o o o o ooo o o o o o o o o o o o o o o oo o o o o o o oo o o o o o ooo oo o o o oo o o o o oo oo o o oo oo oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo oo o oo ooo o o o o oo o o o o o o oo oo o oo o o o o o o o o o o o o oo o o oo o o o o o o o oo o o o o o o o o o oo o o o o oo o o o o o o o o o o o o o o o o ooo o o o o o o o o o o oo o o oo o o oo oo oo o o o o o o o o o oo o o o oo oo o o o o o o oooo o o oo o o o oo o o o o o o oo o o o o o oo o o o o o oo o o o o o o o o o oo oo o o o o o o o ooo oooo o o o o o o o o o o oo o oo o oo o o ooo oo o o o o o o o o o o oo o o o oo o o o o oooo o oo o oo o o o o oo o o o o o o o oo oo o oo o o o o o oo o o oo o oo o oo o o oo o o oo o o o o o o o o o o o oo o oo oo o o o o o oo o o oo o o oo o o o o oo o o o o o oo oo oo o o o o oo ooo o o o o o oo oo oo o o oo o o o o o o o o o oo o o o oo o o o o o o o o o o oo o o o o o o o o oo o o o o o o o o o oo o o o o o o o o oooo oo o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o oo oo o oo o o oo o o oo o oo o o o o o o oo o o o o oo oo o o o oo o o oo o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o

o

o

1900

1950

2000

Figure 2.4: Monthly average temperature in degrees centigrade, January, 1856 - February 2005, n = 1790 months. The straight line (wide and green) is the linear trend y = −9.037 + 0.0046 t and the curve (wide and red) is the nonparametric estimated trend.

warming or it may be just a normal fluctuation. Fitting a straight line relating time t to temperature in degrees Centigrade by simple least squares leads to β#1 = −9.037, β#2 = 0.0046 and a detrended series shown in the left panel of Figure 2.5. Note that the detrended series

CHAPTER 2. CHARACTERISTICS OF TIME SERIES Detrended: Nonlinear

−0.5

−0.6 −0.4 −0.2

0.0

0.0

0.2

0.4

0.5

0.6

Detrended: Linear

24

1900

1950

2000

1900

1950

2000

Figure 2.5: Detrended monthly global temperatures: left panel (linear) and right panel (nonlinear).

still contains a trend like bulge that is highest at about t = 60 years. In this case the slope of the line is often used to argue that there is a global warming trend and that the average increase is approximately 0.83 degrees F per 100 years. It is clear that the residuals in Figure 2.5 still contain substantial correlation and the ordinary least squares model may not be appropriate. There may also be other functional forms that do a better job of detrending; for example, quadratic or logarithmic representations are common or nonparametric approach can be used (We will discuss this approach in detail later); see the detrended series shown in the right panel of Figure 2.5. Detrending is particularly essential when one is estimating the covariance function and power spectrum. 2.2.2

Differencing

A common method for achieving stationarity in nonstationary cases is with the first difference ∆yt = yt − yt−1,

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

25

where ∆ is called the differencing operator. The use of differencing as a method for transforming to stationarity is common also for series with trend. For example, in the trend in Example 1.3, the differenced series would be ∆yt = b + xt − xt−1, which is stationary because the difference xt − xt−1 can be shown to be stationary. Example 1.4: The first difference of the global temperature series is shown in Figure 2.6 and we see that the upward linear trend has disappeared as has the trend like bulge that remained in the

−0.5

0.0

0.5

1.0

Differenced Time Series

1900

1950

2000

Figure 2.6: Differenced monthly global temperatures.

detrended series. Higher order differences are defined as successive applications of the operator ∆. For example, the second difference is ∆2yt = ∆∆yt so that∆2yt = yt − 2 yt−1 + yt−2. If the model also contains a quadratic trend term c t2, it is easy to show that taking the second difference reduces the model to a stationary form. The trends in Figures 2.3 and 2.4 are all of the familiar type, that is, many economic time series display an upward moving trend. It is however not necessary for a trend to move upwards to be called a trend. It is also that a trend is less smooth and may display slowly changing tendencies which once in a while change directions.

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

26

250

60

Example 1.5: An example of such a trending pattern is given in the top left panel of Figure 2.7 and the first difference in the top right

40

Data

100

0

150

20

200

First difference

60

70

80

90

0

10

20

30

40

30

50

−20 −10

0

10

20

Second order difference

0

10

20

30

40

Figure 2.7: Annual stock of motor cycles in the Netherlands, 1946-1993.

panel of Figure 2.7 and the second order difference in the bottom left panel of Figure 2.7, where the annual stock of motor cycles in The Netherland is displayed, for 1946 − 1993, with the first order and second order differencing time series. From the figures in the right top and bottom panels, we can see that the differencing might not work well for this example. One way to describe this changing trends is to allow the parameters to change over time, driven by some exogenous shocks (macroeconomic variables)2, for example, oil shock in 1974. 2.2.3

Transformations

A transformation that cuts down the values of larger peaks of a time series and emphasizes the lower values may be effective in reducing 2

see the paper by Cai (2006)

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

27

nonstationary behavior due to changing variance. An example is the logarithmic transformation yt = log(xt), where log denotes the exponential-base logarithm. Example 1.6: For example, the data shown in Figure 2.8 represent quarterly earnings per share for the American Company Johnson & transformed log(earnings)

0

0

5

1

10

2

15

J&J Earnings

0

20

40

60

80

0

20

40

60

80

Figure 2.8: Quarterly earnings for Johnson & Johnson (4th quarter, 1970 to 1st quarter, 1980, left panel) with log transformed earnings (right panel).

Johnson from the from the fourth quarter of 1970 to the first quarter of 1980. It is easy to note some very nonstationary behavior in this series that cannot be eliminated completely by differencing or detrending because of the larger fluctuations that occur near the end of the record when the earnings are higher. The right panel of Figure 2.8 shows the log-transformed series and we note that the latter peaks have been attenuated so that the variance of the transformed series seems more stable. One would have to eliminate the trend still remaining in the above series to obtain stationarity. For more details on the current analyses of this series, see the later analyses and the papers by Burman and Shumway (1998) and Cai and Chen (2006). A general transformation is the well-known Box-Cox transformation; see Hamilton (1994, p.126), Shumway (1988), and Shumway

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

28

and Stoffer (2000), defined in terms of arbitrary power xαt for some α in a certain range, which can be chosen based on some optimal criterion such as the smallest mean squared error. 2.2.4

Linear Filters

The first difference is a linear combination of the values of the series at two lags, say 0 and 1 and has the effect of retaining the faster oscillations and attenuating or reducing the slower oscillations. We may define more general linear filters to do other kinds of smoothing or roughening of a time series to enhance signals and attenuate noise. Consider the general linear combination of past and future values of a time series given as yt =

∞ $

j=−∞

aj xt−j

where aj , j = 0, ±1, ±2, . . ., define a set of fixed filter coefficients to be applied to the series of interest. An example is the first difference where a0 = 1, a1 = −1, aj = 0 otherwise. Note that the above {yt} is also called a linear process in probability literature. Example 1.7: To give a simple illustration, consider the twelve month moving average aj = 1/12, j = 0, ±1, ±2, ±3, ±4, ±5, ±6 and zero otherwise. The result of applying this filter to the SOI index is shown in Figure 2.9. It is clear that this filter removes some higher oscillations and produces a smoother series. In fact, the yearly oscillations have been filtered out (see the bottom panel in Figure 2.9) and a lower frequency oscillation appears with a cycling rate of about 42 months. This is the so-called El Ni˜no effect that accounts for all kinds of phenomena. This filtering effect will be examined further later on spectral analysis since it is extremely important to know exactly how one is influencing the periodic oscillations by filtering.

29

−1.0

−0.5

−0.5

0.0

0.0

0.5

0.5

1.0

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

0

100

200

300

400

0

100

200

300

400

Figure 2.9: The SOI series (black solid line) compared with a 12 point moving average (red thicker solid line). The top panel: original data and the bottom panel: filtered series.

To summarize, the graphical examination of time histories can point the way to further analyses by noting periodicities and trends that may be present. Furthermore, looking at time histories of transformed or filtered series often gives an intuitive idea as to whether one series could also be associated with another. Figure 2.1 indicates that the SOI series tends to precede or lead the recruit series. Naturally, one can ask for a more detailed specification of the leading lagging relation. In the following sections, we will try to show how classical time series methods can be used to provide partial answers to these kinds of questions. Before doing so, we spend some space to introduce some other features, such as seasonality, outliers, nonlinearity, and conditional heteroscedasticity common seen in the economic and financial as well as environmental data.

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

2.3 2.3.1

30

Other Key Features of Time Series Seasonality

When time series (particularly, economic and financial time series) are observed each day or month or quarter, it is often the case that such as a series displays a seasonal pattern (deterministic cyclical behavior). Similar to the feature of trend, there is no precise definition of seasonality. Usually we refer to seasonality when observations in certain seasons display strikingly different features to other seasons. For example, when the retail sales are always large in the fourth quarter (because of the Christmas spending) and small in the first quarter as can be observed from Figure 2.10. It may also be possible that seasonality is reflected in the variance of a time series. For example, for daily observed stock market returns the volatility seems often highest on Mondays, basically because investors have to digest three days of news instead of only day. For mode details, see the book by Taylor (2005, §4.5) and Tsay (2005). Example 1.8: In this example we consider the monthly US retail sales series (not seasonally adjusted) from January of 1967 to December of 2000 (in billions of US dollars). The data can be downloaded from the web site at http://marketvector.com. The U.S. retail sales index is one of the most important indicators of the US economy. There are vast studies of the seasonal series (like this series) in the literature; see, e.g., Franses (1996, 1998) and Ghysels and Osborn (2001) and Cai and Chen (2006). From Figure 2.10, we can observe that the peaks occur in December and we can say that retail sales display seasonality. Also, it can be observed that the trend is basically increasing but nonlinearly. The same phenomenon can be observed from Figure 2.8 for the quarterly earnings for Johnson & Johnson.

31

50000 100000

200000

300000

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

0

100

200

300

400

Figure 2.10: US Retail Sales Data from 1967-2000.

If simple graphs are not informative enough to highlight possible seasonal variation, a formal regression model can be used, for example, one might try to consider the following regression model with seasonal dummy variables ∆ yt = yt − yt−1 =

s $

j=1

βj Dj,t + εt,

where Dj,t is a seasonal dummy variable and s is the number of seasons. Of course, one can use a seasonal ARIMA model, denoted by ARIMA(p, d, q) × (Q, D, Q)s, which will be discussed later. Example 1.9: In this example, we consider a time series with pronounced seasonality displayed in Figure 2.11, where logs of fourweekly advertising expenditures on ratio and television in The Netherlands for 1978.01 − 1994.13. For these two marketing time series one can observe clearly that the television advertising displays quite some seasonal fluctuation throughout the entire sample and the radio advertising has seasonality only for the last five years. Also, there seems to be a structural break in the radio series around observation 53. This break is related to an increase in radio broadcasting minutes in January 1982. Furthermore, there is a visual evidence that the trend

32

11

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

9

10

television

8

radio

0

50

100

150

200

Figure 2.11: Four-weekly advertising expenditures on radio and television in The Netherlands, 1978.01 − 1994.13.

changes over time. Generally, it appears that many time series seasonally observed from business and economics as well as other applied fields display seasonality in the sense that the observations in certain seasons have properties that differ from those data points in other seasons. A second feature of many seasonal time series is that the seasonality changes over time, like what studied by Cai and Chen (2006). Sometimes, these changes appear abrupt, as is the case for advertising on the radio in Figure 2.11, and sometimes such changes occur only slowly. To capture these phenomena, Cai and Chen (2006) proposed a more general flexible seasonal effect model having the following form: yij = α(ti) + βj (ti) + eij ,

i = 1, . . . , n,

j = 1, . . . , s,

where yij = y(i−1)s+j , ti = i/n, α(·) is a (smooth) common trend function in [0, 1], {βj (·)} are (smooth) seasonal effect functions in [0, 1], either fixed or random, subject to a set of constraints, and the error term eij is assumed to be stationary. For more details, see Cai and Chen (2006).

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

2.3.2

33

Aberrant Observations

Possibly distorting observations do not necessarily come in a sequence as in the radio advertising example (which might have socalled regime shifts). It may also be that only few observations have a major impact on time series modeling and forecasting. Such data points are called aberrant observations (outliers in statistics).

5

Example 1.10: As an illustration example, we consider the differenced yt, that is ∆ yt = yt − yt−1, where yt = log(wt), with wt the price level, and the inflation rate ft = (wt −wt−1)/wt−1 in Argentina, for the sample 1970.1 − 1989.4 in Figure 2.12. From the figure, it

0

1

2

3

4

difference inflation

70

75

80

85

90

Figure 2.12: First difference in log prices versus the inflation rate: the case of Argentina, 1970.1 − 1989.4.

is obvious that in the case where the quarterly inflation rate is high (as is the case in 1989.3 where it is about 500 percent), ∆ yt series is not a good approximation to the inflation rate (since the 1989.3 observation would not now correspond to about 200 percent). Also, we can observe that the data in 1989 seem to be quite different from those observations the year before. In fact, if there is any correlation between ∆ yt and ∆ yt−1, such a correlation may be affected by these

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

34

observations. In other words, if a simple regression is used to model the correlation between ∆ yt and ∆ yt−1, we would expect that an estimate of the coefficient ρ is influenced by the data points in the last year. The ordinary least square estimates of ρ are ρ" = 0.561(0.094) for the entire sample and ρ" = 0.704(0.082) for the sample without the data points in the last year, where the estimated standard error is in parentheses. It now turns to the question on how to handle with these aberrant observations. See Chapter 6 of Franses (1998) for details on discussion of methods to delete several types of aberrant observations, and also methods to take account of such data for forecasting. 2.3.3

Conditional Heteroskedasticity

An important feature of economic time series, and in particular, of financial time series is that aberrant observations tend to emerge in clusters (persistence). The intuitive interpretation is that if a certain day news arrives on a stock market, the reaction to this news is to buy or sell many stocks, while the day after the news has digested and valued properly, the stock market returns back to the level of before the arrival of the news. This pattern would be reflected by (a possibly large) increasing or decreasing (usually, the negative impact is larger than the positive impact, called asymmetric) in the returns on one day followed by an opposite change on the next day. As a result, we can regard them as aberrant observations in a row and two sudden changes in returns are correlated since the second sharp change is caused by the first. This is called conditional heteroskedasticity. To characterize this phenomenon, one might use the so called the autoregressive conditional heteroscedasticity (ARCH) model of Engle (1982) and the generalized autoregressive

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

35

conditional heteroscedasticity (GARCH) model of Bollerslev (1986) or other GARCH type models; see Taylor (2005) and Tsay (2005). Example 1.11: This example concerns the closing bid prices of the Japanese Yen (JPY) in terms of U.S. dollar. There is a vast amount of literature devoted to the study of the exchange rate time series; see Sercu and Uppal (2000) and the references therein for details. Here we explore the possible nonlinearity feature (see the next section), heteroscedasticity, and predictability of the exchange rate series (We will discuss this later). The data is a weekly series from January 1, 1974 to December 31, 2003. The daily noon buying rates in New York City certified by the Federal Reserve Bank of New York for customs and cable transfers purposes were obtained from the Chicago Federal Reserve Board (www.frbchi.org). The weekly series is generated by selecting the Wednesdays series (if a Wednesday is a holiday then the following Thursday is used), which has 1566 observations. The use of weekly data avoids the so-called weekend effect as well as other biases associated with nontrading, bid-ask spread, asynchronous rates and so on, which are often present in higher frequency data. We consider the log return series yt = 100 log(ξt/ξt−1), plotted in Figure 2.13, where ξt is an exchange rate level on the t-th week. Around the 44th week of 1998 (the period of the Asian fiance crisis), the returns on the Japanese and U.S. dollar exchange rate decreased by 9.7%. Immediately after that observation, we can find several data points that are large in the absolute value. Additionally, in other parts of sample, we can observe “bubbles”, i.e., clusters of observations with large variances. This phenomenon is called volatility clustering (persistence) or, conditional heteroskedasticity. In other words, the variance changes over time. To allow for the possibility that high volatility is followed by high

36

−10

−5

0

5

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

0

500

1000

1500

Figure 2.13: Japanese - U.S. dollar exchange rate return series {yt }, from January 1, 1974 to December 31, 2003.

volatility, and that low volatility will be followed by low volatility, where volatility is defined in terms of returns themselves, one can consider the presence of so-called conditional heteroskedasticity or its variants such as ARCH or GARCH type models by using (∆ yt)2 as the variance of the returns. Also, once can replay (∆ yt)2 by |∆ yt|, the absolute value of return. The main purpose of exploiting volatility clustering is to forecast future volatility. Since this variable is a measure of risk, such forecasts can be useful to evaluate investment strategies or portfolio selection or risk management. Furthermore, it can be useful for decisions on buying or seeling options or derivatives. See Hamilton (1994, Chapter 21), Taylor (2005), and Tsay (2005) for details. 2.3.4

Nonlinearity

The nonlinear feature of time series can be seen often in economic and financial data as well as other applied fields; see the popular books by Tong (1990), Granger and Ter¨asvirta (1993), Franses and van Dijk (2000), and Fan and Yao (2003). Beyond linear domain, there are

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

37

infinite many nonlinear forms to be explored. Early development of nonlinear time series analysis focused on various nonlinear (sometimes non-Gaussian) parametric forms. The successful examples include, among others, the ARCH-modeling of fluctuating structure for financial time series, and the threshold modeling for biological and economic data, as well as regime switches or structural change modeling for economic and financial time series.

10

Example 1.12: Consider the example of the unemployment rate in Germany for 1962.1 to 1991.4 in Figure 2.14. From the graph in the

0

2

2

4

4

6

6

8

8

unadjusted seasonally adjusted

65

70

75

80

85

90

2

4

6

8

Figure 2.14: Quarterly unemployment rate in Germany, 1962.1 − 1991.4 (seasonally adjusted and not seasonally adjusted) in the left panel. The scatterplot of unemployment rate (seasonally adjusted) versus unemployment rate (seasonally adjusted) one period lagged in the right panel.

left panel, it is clear that unemployment rate sometimes rises quite rapidly, usually in the recession years 1967, 1974 − 1975, and 1980 − 1982, while it decreases very slowly, usually in times of expansions. This asymmetry can be formalized by estimating the parameters in the following simple regression ∆ yt = yt − yt−1 = β1 It(E) + β2 It(R) + εt,

where It(·) is the indicator variable, which allows the absolute value of the rate of change to vary across the two state, say, “decreasing

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

38

yt” and “increasing yt” from β1 to β2, where β1 may be different from −β2. For the German seasonally adjusted unemployment rate, we find that β#1 = −0.040 and β#2 = 0.388, indicating that when the unemployment rate increases (in recessions), it rises faster than when it goes down (in expansions). Furthermore, from the graph in the right panel, the scatterplot of yt versus yt−1, where yt is the seasonally adjusted unemployment rate, we can observe that this series displays cyclical behavior around points that shift over time. When these shifts are endogenous, i.e., caused by past observations on yt themselves, this can be viewed as a typical feature of nonlinear time series. For the detailed analysis of this dataset using a nonlinear methods, the reader is referred to the book by Franses (1998) for nonlinear parametric model and the paper by Cai (2002) for nonparametric model. 2.4

Time Series Relationships

One can identify two basic kinds of association or correlation that are important in time series considerations. The first is the notion of self correlation or autocorrelation introduced in (2.2). The second is that series are somehow related to each other so that one might hypothesize some causal relationship existing between the phenomena generating the series. For example, one might hypothesize that the simultaneous oscillations of the SOI and recruitment series suggest that they are related. We introduce below three statistics for identifying the sources of time correlation. Assume that we have two series xt and yt that are observed over some set of time points, say t = 1, . . . , n.

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

2.4.1

39

Autocorrelation Function

Correlation at adjacent points of the same series is measured by the autocorrelation function defined in (2.2). For example, the SOI series in Figure 2.1 contains regular fluctuations at intervals of approximately 12 months. An indication of possible linear as well as nonlinear relations can be inferred by examining the lagged scatterplots, defined as plots that put xt on the horizontal Figure 2.15. Multiple lagged scatterplots showing the relations between SOI and the present (xt) and lagged values (xt+h) of SOI at lags h = 0, 1, . . . , 12 axis and xt+h on the vertical axis for various values of the lag h = 1, 2, 3, . . ., 12.

14

o o oo oo o oo o o o o o oo o o o o o o o o o o o o o o o o oo o oo o o o o o o oo o oo o o o o o o o o o o o oo o o o o o o o o oo o o o oo o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o oo o o o o oo o o o o oo oo o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o ooo o oo o o o o o o o o o o oo o o o o o oo o oo o o o o o o o oo o o o o o o o o o o o o o o ooo o o oo o o o o o ooo o o oo o o o o o ooo o o o o o o o o o o o o o o o o o oo oo o oo oo oo o o o o oo o o oo ooo o o oooo o oo o o o o oo ooo ooo o o o oo oo o o o oo oo −1.0 0.0 0.5 1.0

15

1.0 0.0 −1.0 1.0 0.0 −1.0 1.0

8

0.0

o o o o o o o o oo oo o oooo oo o oo o oo oooo oo oo ooo o o o o o o o o o o o o o o o o oo o oo o o oo o o o o o o o o o o o oo o o oo o o o o o o o o o o o o o o oo o o o o o o o o o o o o oo o o o ooo o o o o o o o o oo o o o o o o o o o o o o o o o oo oo o oo o o o o o o o o o oo o oo o o o o o o o o o ooo o o o o o o oo o o o o o o o o o o o o oo o o o o o o o o o o o o o oo oo oo o o o o o o oo o o o o o o o o oooo o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o oo o oo oo o o o o o o o o o o o o o o oo o o oo o oo o o o o oo o o oo o oo o o o oo o o oo o o −1.0 0.0 0.5 1.0

−1.0

11

o o o o o o o o o ooo o o o oo o o o o o o o o o oo oo o oo o oo o oo o o oo oo o o o oo o o o o o o o o oo oo o o o o o o o o o oo o o o o o o o o o o o o o o o oo o oo o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o ooo o o o o o o o oooo o oo o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o oo o o o o o o o o o o o oo oo o o o o o o o o o o o o o o o o o o o ooo o ooo o o oo oo o o o oo oo oo o o oo o o o o o oo o o oo oo ooo o o o o oo o oo o oo o o o o o ooo o o o o ooooo o oo o o o o oo o o ooo o o o o o o oo oo o −1.0 0.0 0.5 1.0

12

1.0

1.0 0.0 −1.0 1.0 0.0 −1.0 1.0 −1.0

0.0

o o o o o o o o o o o oo o o oo o oo o o o oo oo o o o ooo oo oo o oo oo oo o o o o o o o o o oo o o o o o ooo o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o oo o oo o oo o o o o o o o o o oo o oo o o o o o o o o o o o o o oo o o o o o oo oo o oo o o o o o o oo oo oo o o o o o o o o ooo o o o o o o o o o o o o o o o o o o o o o o o oo o o o o oo oo oo o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o oo o o oo ooo o o oo o o o o o o o oo oo o −1.0 0.0 0.5 1.0

4

o o o ooo oo o oo oo o o ooo o oo o o o oo o o oo o o o o oo o o o o o o o o ooo o o o o o o o oo o oo o ooo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o oo o o o o o oooo o o o o ooo o o oo o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o ooo o o o oo oo o o o o o o oo o o o o o o o o o o oo oo o o o o o o oo o o oo o o o o o o o o o o o o o o o o o o o oo o oooo o o o o oo oo o ooo o o o oo o o o o o oo o o o o o oo oooo oo o o oo o ooo oo o ooo oo o oo −1.0 0.0 0.5 1.0

0.0

o o oo o o o o o o o ooo o o oo o oo o o o o o o o o o o o o oo o o o o oo o o o oo o o o o o oooo oo o o o o o o o oo o o o o o oo o o o o o o o o o o o o o o o o o o oo o oo o o ooo o o o o o o o o o o o o o o oooo oo ooo o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o o o o oo o o o o o o o o o o o o o o o o oo oo o o o o o o o o o o o ooo o o o o o o o o o oo o o o o o oo o o o o oo o o o o o o o o o o o o o o o o ooo oo o o o o o o oo oo o o o oo o o o o o o o oo oo o o o oo ooo o oo ooo oo oo oo oo o oo oo o oooo ooo oo o o o o oo o oo o o ooo o o oo o o −1.0 0.0 0.5 1.0

7

1.0

10

ooo o oo ooooo o o o o o o o o o o o oo o ooo oo ooo oo o o o o o o o o o o o o oo o o o o o oo o o o o o o o o o oo o o o o o o o o oo o oo o oo oo o o o o o o oo o oo o o o oo o o o o oo ooo o o o o o o o o o o o oo o ooo o o o o o o o o o o ooo o o oo o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo oo o o o o oo o oooo o oo o oooo o o o o o o oo o o o o o o oo o o o o o o o oo oo o o o o o o o oo o o o o o o o o oo o o o o oo o oo oo oo oooo oo oo o o oo oo o o o o oooo oo o oo ooo o oo o o o oo oo o o o −1.0 0.0 0.5 1.0

0.0

1.0 0.0 −1.0 1.0 0.0 −1.0 1.0 0.0

o o o o o oo o o oo o ooo oo o oo o oo o o oo o oo ooo o oo o o o o o o o oooo o oo o oo o oo o oo o o oo o o o o o oo o o o o o o o o o o o oo o o o o o o o o o o o o o o oo oo oooo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo oo o o o o o o o o o o oo oo o o o o o o oo o oo o o oooo oo o o o o o o o o oo o o o o o o o o oooo o o o oo o o o o o o o o o o o o o o o o oo o o o o o o o o o o oooo ooo o o o o oo o o o o o o oo o o o o oo o o oo o oo o o o o o o o o o oo oo o o oo oo oo ooo o o o o ooo oo o o oo o o o o o o o o o o o o ooo o o o o o −1.0 0.0 0.5 1.0

3

o oo o oo o o o ooo ooo o o o o o oo o o oo oo o o oo oo o o oo o o oo o o oo oo o o o oo o o o oo o o o o o o o o o ooo o o o ooo oo o ooo oo o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o o ooo o o o o o o o o o oo o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o ooo o o o ooo oo o o o o o o oo o o o o o o o oo oo o o o o o o o o o o oo o o o o o o o o o o o o o ooo o ooo o ooo oo o oo o oo oo o oo ooo o o o oo o o o o o o o o oo o oo o o o oo ooo o o o o ooo o o o o o o oo o o o ooo o oo o o o o o o ooo o o oo o o oo −1.0 0.0 0.5 1.0

−1.0

13

6

o o oo o o ooo o o o o o o o o o ooo o o o o o o o oo o o o o o oo o o o oo o o o oo o o o o o o o o o o oo o oo o o o o o o o o o o o o o o o o o o o o o o o ooo o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o o o o oo oo o o o o o o o o o oo ooo oo o o o o oo o o o o o oo o o o oo o o o oo o o o o oo o oo o o o o o o o o o o o o oo o o o o o oo o oo o o o o o o o oo o o o o o o o o ooo o o oo o o o o oo o o o o o o o o o o o ooo oo o o o o o o oo o o o o o o −1.0 0.0 0.5 1.0

−1.0

−1.0

0.0

o oo o oo o o o o ooo o o o o oo o o oo oo o o o o o o o o o o o o o o o oo o o o o o o o o o ooo o o oo o o o o o o oo o o o o o oooo o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o ooo o o o o o oo o o oo o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo oo o o o o o o o o o o o oooo o o o o o o o o o o o o o o o o o o o o oooo o o o o o o o o oo o o oo oo oo o o oo o o o o o o oo o o o o o o o o o o o oo oo o o oo o oo o o oo oo o o o o o o o o o o o o oo o o o ooo o o o o o o o o oooo o o o o o o o o o o o o oo o o oo oo o o −1.0 0.0 0.5 1.0

oo ooo ooo oo oo oo o o oo o o oo o o ooo oo o o o o o o o o o o o o o o o o o o oo oo o o o o o o oo o o o o o o o o o o o o o ooo o o o o o o o o o o o o oo o o oo o o o o o o o o o o ooo oo o o o o o o o o ooo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o ooo o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o oo o o o o o o o o o oo o o o o oo o o o o oo o o o o o o oo o o o o o o o o o o o o o o o o o o ooo o o o oo oo o o o oo oo o o o o o o o o o o o o o oo o oo o o oo o o o o o o o oo oo o o o o o o o o o o ooo oo o ooo oooo oo −1.0 0.0 0.5 1.0

−1.0

9

1.0

−1.0

0.0

o oooo o o o o o o o o o oo o oo ooo oo o oo oo o o ooo o o oo o o o oo oo o o o o o o o o o o o oo o o o o o oo o o o o o o o o o o o o o oo oo o o o o o o o oo o o o o o o o o o o ooo o oo o o ooo o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o ooo oo o o o ooo o oo o o o o ooo o o o o oo o o oo o o o oo o o o oo oo oo o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o o o o o oo ooo o o oo o o oo o o o o o o ooo o oo oo o o oo o oooooo o o o oo ooo o o ooo oo o o o oo oo o ooo −1.0 0.0 0.5 1.0

2

1.0

5

1.0

−1.0

0.0

o o o o o oo o ooo o o o o o o o o o o o oo o oo oo o oo o o oo oo o oo o o o oo o o oo o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o o o o o o o ooo oo o o o o o o ooo o o o o o o o o o o o o o oo oo o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o oo o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o oo oo o o oo o oo o o oo oo o oo o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o ooooo ooo oooo o oo o o o o o o o oo o o o o oo o ooo o oo oo o oooo oo o oo o o o o o o o o o o o oo o o o o o o o o o o o oo −1.0 0.0 0.5 1.0

o oo o oo o o oo oo o o o o o o o o o o o o oooo o o oo o o o o oo o o o o o o oo o o o o o o o o o oo o o o oo o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o ooo oo o o o o oo o o o o oo o o o o o ooo o o o o o o o o o o o o o o o oo oo o o o o o o o oo o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo o oo o o o o oo o o oo oo oo o oo o o o o o ooo oooo oo o o oo o o ooo oo oo o o oo o oo o ooo oo o oo oo o oo oo o ooo o oo o o oo o o o −1.0 0.0 0.5 1.0

0.0

1

1.0

−1.0

0.0

o o o o o o o o o o oo o o o o oo o o o o oo o oo oo o o o o o o o o o o o oo oo o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo oo o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o oo o oo oo o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o oo oo o o o o o o o o o o o o o o o o o o o o o oo o o o o o o oo oo oo o o o o oo o o o o o oo oo o o o o o oo oo o oo o ooo o o o o o o o o o o oo o ooo o oo oooo o ooo oo o oo oo o o −1.0 0.0 0.5 1.0

−1.0

1.0

Example 1.13: In Figure 2.15, we have made a lagged scatterplot of the SOI series at time t + h against the SOI series at time t and obtained a high correlation, 0.412, between the series xt+12 and the

16

Figure 2.15: Multiple lagged scatterplots showing the relationship between SOI and the present (xt ) versus the lagged values (xt+h ) at lags 1 ≤ h ≤ 16.

series xt shifted by 12 years. Lower order lags at t − 1, t − 2 also

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

40

show correlation. The scatterplot shows the direction of the relation which tends to be positive for lags 1, 2, 11, 12, 13, and tends to be negative for lags 6, 7, 8. The scatterplot can also show no significant nonlinearities to be present. In order to develop a measure for this self correlation or autocorrelation, we utilize a sample version of the scaled autocovariance function (2.2), say ρ"x(h) = γ" x(h)/γ" x(0),

where

1 n−h $ γx(h) = (xt+h − x)(xt − x), n t=1 % which is the sample counterpart of (2.2) with x = nt=1 xt/n. Under the assumption that the underlying process xt is white noise, the approximate standard error of the sample ACF is 1 σρ = √ . (2.3) n That is, ρ"x(h) is approximately normal with mean 0 and variance 1/n. "

Example 1.14: As an illustration, consider the autocorrelation functions computed for the environmental and recruitment series shown in the top two panels of Figure 2.16. Both of the autocorrelation functions show some evidence of periodic repetition. The ACF of SOI seems to repeat at periods of 12 while the recruitment has a dominant period that repeats at about 12 to 16 time points. Again, the maximum values are well above two standard errors shown as dotted lines above and below the horizontal axis. 2.4.2

Cross Correlation Function

The fact that correlations may occur at some time delay when trying to relate two series to one another at some lag h for purposes of

0.0 10

20

30

40

50

0

10

20

30

40

50

1.0

0

−0.5

0.0

0.5

ACF of Recruits

0.5

ACF of SOI Index

−0.5

41

1.0

1.0

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

−0.5

0.0

0.5

CCF of SOI and Recruits

−40

−20

0

20

40

Figure 2.16: Autocorrelation functions of SOI and recruitment and cross correlation function between SOI and recruitment.

prediction suggests that it would also be useful to plot xt+h against yt . Example 1.15: In order to examine this possibility, consider the lagged scatterplot matrix shown in Figures 2.17 and 2.18. Figure 2.17 plots the SOI at time t + h, xt+h, versus the recruitment series yt at lag 0 ≤ h ≤ 15 in Figure 2.17. There are no particularly strong linear relations apparent in this plots, i.e. future values of SOI are not related to current recruitment. This means that the temperatures are not responding to past recruitment. In Figure 2.18, the current SOI values, xt are plotted against the future recruitment values, yt+h for 0 ≤ h ≤ 15. It is clear from Figure 2.18 that the series are correlated negatively for lags h = 5, . . . , 9. The correlation at lag 6, for example, is −0.60 implying that increases in the SOI lead decreases in number of recruits by about 6 months. On the other hand, the series are hardly correlated (0.025) at all in the conventional sense,

0.0

0.5

1.0

14

15

0.5

1.0

100 60 0 20 100

0.0

8

100

0 20

60

o o o oo o o o o ooo o o o o o o o oo o oo o o o o o o o oo o o o o oo o o o o o o o o o oo o o oo o o o o o o oo o o o oo o oo o oo o o o oo o o oo o oo o o o o o o o o oo o o o o o o o o o o o o o o oo o o o o o oo o ooo o o o o o o o o o o o o oooo o o o o o o o o o o o o o o o o oo o o o o o o o oo o o o o o oo o oooo o o oooo oo o o o o ooo o oo oo oo o o o oo o o oo o o o oo o o oooo oo o ooo o oo o o o o oo o o o o oo o o o o o o o oo o o o oo o o o o o o o o oo o oo o oo o o o o o o o o o o o oo o o o o o o oo o o ooo oo oo ooo o o o o o o o o o o o o o o o o o o o ooo oo o oo o o o o o oo o o oo o o −1.0 0.0 0.5 1.0

60

o o oo o o o o o oo o o o oo o o o oo o o o oo o o o o o o o o o oo o o o o o o o o o o o o o o o o o oo o o o o ooo o o o o oo o o o o o o o o o o o o o o o o o oo o o oo o o o o o o o oo o o o o o o o o oo o o o oo o o oo o o o o o o oo oooo o o o o o o o o o o o o o oo o o o o o oo o oo o o o oo o o o o o o o o o o o o oo oo o ooooo o ooo oo o o o o oo o oo o o o oo o oo o o o oo oo o o o o o o o o o o o o o o o oo o o o o oo o o o o oo oo o ooo o o o oo oo o o oo o o o o o o o o o o oo oo o o o o o o o o o o oo o oo o oo o oo o o oo o ooo o o o o oo oo o o o o o o o o o o o oo o oo o o o o o oo o o o o oo o o o oo o o o −1.0 0.0 0.5 1.0

0 20

100 60 0 20 100 60 0 20 100 60

o o o oo oo oo oo o o o oo oo o o o o o o o o o o o o o o o o o o o o oo o o o o o o o oo oo o oo oo o o o o o oo o o o o o o o o oo oo o o ooo o o o o o o oooo o o o o o o o o o o o oo o o o o o o o oo o oo o o oo o o o oo o o oo o oo o o o o o o o o o o o o o oo o o o oo o ooo o o o o o o o o oooo ooo o o o oo o o o o o oo o oo o o oo o oooo o oo oo o o o oo oo o o o o o ooo oo o o o o oooo o o oo o o o o o o o o o o o o o o o o oo oo oooo o o o o ooo ooo o o o oo oo o o o o o o oo oo oo o o o oo o oo o o o o o o o o oo o o o oooo o oo o o o o o o o oo oo o o o o oo o o o o o o o o o o oo o o o oo o o o −1.0 0.0 0.5 1.0

4

−1.0

12

100

oo o o o o o o o o o o o o o o oo o o o o o o o o oo o oo o o o o o o o o o oo o o o o o o oo oo o o o o o o ooo oo oo oo o o o o o oo o oo oo o o o o o o o o oo o oo o o o oo o o o o oo oo o o oo oo o o o o o o o o o o o o o oo o o o o o o o o o o o oo o ooo o o o o o o o ooo o oo o oo o o o o o o o o o o o o oo o oo o ooo oo o o ooo oo o o oo o o o o o o oo o o o o oo ooo oo o oo oo o oo o o oo o o oo oo o o o ooo o o oooo o o o o oo o o oo o oo o o oo o o o o o o o o oo o o o oo o oo o o o o o o o o o o o o o o o o o o o o o o o oo o o o o ooo o oo o o o oo oo o o oo oo o o o o o oo o oo −1.0 0.0 0.5 1.0

11

o o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o o o oo o o o o o o o o o o o oo o o o o o o o oo o o o o o o oo o o o oo o oo ooo o oo oo o o o o o oo o oo o o o o o o o o o o o o o o oo o oo o o o oo o o o oo o o o oo o o o oo o o o o o o o o oo o o o o o o o o o o o o o o o o o o oo ooo o o oo o oo ooo o oo o oo o o o o o o o o o o o o o o o o oo ooo o o o o o o ooo o o o o o o o o o o o o o o oo o o oo o o o o o o o o o oo o o o oo o o o oo ooo oo o o o o ooo oo o o o o o o o o o o o oo o o o o ooo o ooo o o o oooo oo o o o o o o o o oo o o o ooooo oo oo o oo o oo o o o o o o o o o o o o o o

oo o o oo o o o o o o o oo o o o o o o o oo o o o o ooo o o o o o oo o o o o o oo o o o oo o o ooo o o o o o o o o oo o o o o oo o o o o o oo o oo o o oo o o o o ooo o oo o o o o o o oo o o o ooo o o o o o o o o o o o o o o o o o o o o o o ooo o o o oo o o oo o ooo o o oo o o o o o o oo o o o o o o o o o o o o ooo o ooo oo oo o o oo o o ooo oo o oo ooo o o o o o oo o oo o o o o o o o o oo o o o oo o o o o oo ooo oo o ooo o oo o oo o o o o o o oo oo o ooo o oo o oooo o oo o o o o o o oo oo o o oo o o o o o o o o oo o o o o o o o o o o o oo o oo o o oo o oo o o o o o o oo o o ooo oo oo o −1.0 0.0 0.5 1.0

60

10

7

o o o oo o o o oo o o o o o o o o o o oo o o o o o o o o o oo o o o o oo o o o o o o o o o o o o o o oo o o o o o o o o o o o o oo o o o o o o oo o o o o o o o oo o oo oo o o o o o o o o o o oo o o o o oo o o o oo oo o o o o o oo o o o o o o o o oo o oo o o oo o o ooo o o oo o o o o o oo o o ooo oo ooo o o o oo oo o o o o o o o o o o o oo oo o o oo o o oo o o oo o o o o o o o oo o oo ooo oo o o o oo o o o o oo o o o o o ooooo o o oo o oo o o o oo o o o oo o o o o o o ooo o o o ooo o o o oo o o o oo o oo o o o ooo oo oo o o o o o o o o ooo o o o o oo o o o o o o o oo ooo o o o o o o o oo o oo o oo ooo o o o o ooooo o oo o o −1.0 0.0 0.5 1.0

0 20

60

o o o oo o ooo o o o o o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o o o o o o ooo o o o o o o o o o o o o o o o o o o o oooo o o oo o oo o o o o o o o o oo o o o o o o o o oo o o oo oo o o o o o o o oo o o o oo o o o o o o o o o o oo o o o oo o o o o ooo o o o ooo o o o o o o o o ooooo o o o o o o o o o o oo o oo oo o o oo oo o oo oo o o o o o o o o o o oo oooo ooo o ooo o ooo oo o oo oo o o oo o o o o o oo o o o o o oo o o o o o oo o o oo o o o o ooo ooo o o ooo o o o oo o o o oo o oooo oo o o o o o o o oo o ooo o ooo oooo o oo o o o o o o o o o oo o o o oo o o o o o o o o o ooo o o o o o −1.0 0.0 0.5 1.0

oo o o o o o o o o o o o o o o oo o o o o o o o o o o oooo o o o o o o oo o o o o o o oo o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo ooo oo o o o o o o o o oo o o o o o oo oo o oo ooo o oo o o oo oo o ooo o o o o o o o o o oo o o o oo o o o o ooo o o o o o o oooo o o o o o o o o o oo oo o oo o o o o oo o o o oooo o o oooo o oo oooo o oo o o oo o o oo ooo o o o o o o o o oo o o oo oo o ooo o o o o o ooo o o o oo o o o o ooo o o o o o oo o o oo oo o o o o o o o o oo o o o o o o o o o o o oo o oo o o o o oo o o oo o oo o oo o o o o o o o o o o oo o oo oo o o o o oo o o o o o oo o o oo o −1.0 0.0 0.5 1.0

100

100 60 0 20 100 60 0 20 100

6

3

42

0 20

13

−1.0

1.0

60

ooo o o o o o o o oo o o o o o o o o o o o o oo o o o o o o o o o o o o o o o oo o o o o o o o o o o oo o o o o o o o o o oo oo o o o o o o o o o o o oo oo o o o o o oo o o o o o o o o o oo o o o o o o oo o o o o o o o oo o o o o o o o o o o o o ooo o o o o ooo o o o o o o o o o o oo ooo oo o o o o o o o o oooo o o o oo o o o oo o o o o o o o oo o o o ooo o oo oo o o o o o oo ooo oo oo o o o ooo o ooooo ooo o o o o oo o o o o oo o o o oo o oo oo o oo o o oo o o o o oo oooo oo oo o o o o o o o o o o oo oo o o o o o o o o o o o o o o o o o oo o oo o o o o o o o oo o o oo o oooo o o o o ooo o o oo o oo oo o o o o o o oo o o o o o

0.5

o oo o o oo o o o o o o o o o o o o oo o o o o o o o o o o o o o o oo o o o o o oo o o o o o o oo o o o o oooo o o oooo o o o o o o o o o o o o o o o o o o o o oo o o oooo o o oo o o o o ooo o o o o oo ooo oo o o o o o o oo o o o o o o o o o o o o oo o o o o o o o o oo o oo o o o o o ooo o o o o o o o o o o o o o o o o o o o o o oo oooo ooo o o o o o o o o o o o o o o oo oo ooo o o oo o ooo ooo o o o o o o o o o oo o o o o o o o oo o oo o o oo oo o o o o o o o o o o oo oo o oo oo o o o o o o oooo o o o oo oo o o o o oo oo o o o o o o o o o o ooo o o o o o oo oo o oo o o o oo o o o oo oo oo oo o o o o o oo o −1.0 0.0 0.5 1.0

0 20

9

0.0

o o o o o o oo o oo o o oo oo o o o oo o o oo oo o o o o o oo o oo o o o o o o o o o o o o o o o o o o o oo o o oo o oo o o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o o oo o o o o o o o o o o o o oo o o o o o o oo oo oo o o o o o o ooo o o o o o o o o o o o o o o o o o o o o o oo o oo o oo o oo oo o o o o o o o o o oooo o o o o o o o o o o oo ooo o oo o o o o o o o o o o o o o o o o o o o o oo ooo ooo oo o oo o o oo o o o o o o o o oo o o o o o o oo oo o oo o oo o o o oo ooo o o oo oo o o o o o o o oo o o oo o o o oo oo o o oo oo o o o ooo o o o o o oo o o o oo o o o o o oo oo o o o o o o o o o o o o o o o o oo −1.0 0.0 0.5 1.0

0 20

0 20

60

100

0 20

60

oo o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o oo o ooo o o o o o o o o o o o o o o o oo oo o o o o o oooo o o o o oo oo oo oo o o o o o o o oo o o o o o o o o o o o o o oo o o o o o ooo o o o o o o o o o oo oo o o o oo o o o o o o o o o o o o o o o o o o o o o ooo o oo o o o o o o o o o o o o o o o o o oo o o o o oo oo o o oo o o o o o o o o oo oo ooo o o o oo o o o o o o oo o o ooo oo o o oo o o o o o o o o o o oo o o oo o oo oo o o o oo oo o o oo oo o o o o o o o oo o o o o o o o o ooo o o ooo o o o o o o o o o oooo o o o o o o ooo oo o oo o o o oo oo o o o o o o ooo o o o oo o o o oo o o o o o o o ooo o o o o o −1.0 0.0 0.5 1.0

2

−1.0

100

5

100

0 20

60

o o o o o o o o o o o o o o o o o ooooo o o o o o o o o o o o o o oo o o o o o o o o o o o oo o oo o o o o o o o o o oo oo o o o oo o o oo oooo o oo o o o oo o o o o o o o oo o oo o ooo o oo o o o o o o o o oo o oo o oo o o o o o o o o o o o o oo o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o ooo o o oo o o oo oo o o o o o o o o o o o o o o o o o o o o o ooooo ooo o o oo oo oo oo oooo ooo o o o o o oo o o o o o ooo o oo o o oo o o o o o o o oo o o o o o o oo oo o oo oo o o o o o o o oo o o o o o o oo o o o o o oo o o o o oo o oo oo o oo o o o o ooo oo o o oooo o oo ooo o o oo oo oo o ooo o oo o −1.0 0.0 0.5 1.0

o oo o o o o o o o o o o o o oo o o o o o o o oo o oo o o o oo o oo o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o ooo o o o o o o o o o o o o o o ooo o o o oo o o o o o oooooo o o o o o oo o o o o o o o o o oo o o o oo o o o o ooo o o oo o o o o o ooo oo o o o o o oo o o o o o ooo o o o o o oo o o o o o o o o o o o o o o oo o o oo o o o o o o oo o o oo o o oooo o o o o o o o ooo oo o ooo o o oo oo o o o o o oo oo o o oo o o oo oo o o o o o ooo o oo o oo o o o o oooo oo o o oo o o o o oo o o o o o oo o oo ooo oo o o o oo oooo o ooo o o oo o o oo o o o o o o o o o o o o

60

1

100

0 20

60

oo o o o o o o o o o o o o o o o oo o ooo o o o o o o o o o oo o o o o oo o o o o oo o o o o o o o oo o o o o o o oo o o ooo o oo o ooo o oo o o o o oo o o o o o o o o o o o o o o o o o o o oo oo o o oo oo o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o oo o o o oo o o o o oo o oo oooo o o oo oo o o o o o o oo o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o ooo oo o ooo o ooo o o o o o o oo o o o oooo o o o o o oo o ooooo oo oo oo oo oo o o o o oo oo o oo o oo oo o o o oo o o o oo o o o oo o oo o o o oo o oo oo o oo o oo oo oo oo ooooo o o o o oo o o o oo o −1.0 0.0 0.5 1.0

0 20

100

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

16

13

14

o oo o o o o o o oo o o o o o o oo o o o oo o oo o o o o o oo o o o o o o o o o oo o oo o o o o o o o o o oo oo o o o o o o oo oo o o o o o o oo o o o o o o o o o o o o oo oo o oo o o oo o oo o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o oo o o o o o o o o oo o o o o oo o oo oo o o o o o o o o o o o o o o oooo o o o o o oo o o oo oo oo o o oo o o o o o o o oo o o oo o oo oo o o o oo o oo oo oo o o o oo oo oo o oo o ooo o oo o ooo o o o o oo ooo oo o ooo o ooo o o ooo o o oo o oo o o o o o o o oo o o o o o o o o oo o oo oo oo ooo o o oo o o o oo oo o o o o o o o o o o ooo o o ooooo −1.0 0.0 0.5 1.0

15

100 60 0 20 100 60 0 20 100

8

60

o o o o o o o o o o o o o o o o o o oo o o o o oo o o o o o o o o o o o o o o o o o o oo o o o oo o o oo o o oo o o o ooo o o o o o oo o o o o oo o o ooo o oo o oo o o o o o o o o o o o o o o o oo o o o o oo o ooo oo oo o oo o oo o oo o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o ooo o oo oo o o oo oo o o o ooo o o o oo o o oo o oo o o oooo o o o ooooo oooo o o o o o o o o ooo oo oo o o o oo o o o oo o o o o o o o o o o o o o o o o o o o o o o o ooo oo o o oo oo o o oo oo oooo o oo o o o o o o oo o o o o o o o o o o o o oo oo o o o o oo ooo oo o oo o oo oo oo o o o o o oo oo −1.0 0.0 0.5 1.0

0 20

100 60 0 20 100 60 0 20 100

11

o o o o o oo o o o o o oo o o o o oo o o o o o o oo o o o oo o o o ooo o o o o o o oo o o o o o o o o o o oo o oo oo o o oo o o o o o o o oo o o o o o oo o o o o oo o o o o o o o o o o o o o o o oo o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o ooo oo o o o o o o o o o o o o oo ooo o o o o o oo o o o o o o o o o o o o o oo oo o o oo o o o oo o o oo o o oo o o o o ooo o o o o o ooo o o o oo o o o o o o o o o o o o o o o o oo o oooo o o o o o oo o o o o o o o o o o o o o o o o o oo oo ooo o o o o o o o oo o o o o oo o o o o o o o o o o o o oo o o o o o o o o o o oo o oo o o −1.0 0.0 0.5 1.0

12

100

o o o oo oo ooo o o o o oooo o o o o o o o o o oo o o o o o o o o o o o o oo ooo o o o o o o o o o o o oo oo o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o o o oo o o oo o o o o oo o o o o oo oo o o o o o o o oo o o o o o o o o oo o o o o o o o o oo o ooo o o o o o o o o o o o o o o o o o o o o oo o ooo o o oo o o ooo ooo oo o o o o o o o o o o oo o oo o o oo o o oooo oo o oo oo oo o o o o o o oo o o oo o o o ooo oo oo o oo o o oo o ooo o o o o oo o o o o o o oo o o o o o o o o o o o o o o o o o o o o oo o o oo o o o oo o o oo o o o o o o o o o ooo o o oo o o oo oo −1.0 0.0 0.5 1.0

o o o oo oo o o oooo o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o oo o o o oo o o o o oo o o o oo o o o o o o o o o o oo o o o o o o o o o o o o o o o oo o o o o o o o oo o o o o o oo o oo o o oo o ooo o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o oo oo o ooo oo o o o oo oo o oo ooo o o o oo o ooo o o o o o oo o o o o oo o o oo oo o oo oo oo o o o o ooo o o o o o o o o o o o o o o o o o o o o o o oo o oo oo o o o o oo o oo o o oo oo o o o o o oo o o o oo o oo o o o oo o o ooo o o o o oo o o o oo o oo oo o o o o oo o o o o o oo o o o oo o −1.0 0.0 0.5 1.0

4

oo o ooo o oo o o o oo oo o oo o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o oo oo o o o o oo o o o o o oo ooo o o o oo o o o o o o o o o o oo ooo o o o o o o o o o o oo o o o o o o o o o o o oo o o o oo o o o o oo o o o o oo o o o o o o o o o o o o oo o oo o oo o o o o ooo oo o o o o o o o oooo o ooo o o oo o o o ooo o o o o o o o o o o o o oo o oo o oo oo ooo o oooo o o o o o o oo o o oo o o o o o o o oo o oo o o o o o o o o oo o o o oo o o o o oo o o o o ooo o o o o o o o oo oo o o o o o o o oo o o o o o ooo o o o oo oo oo o oo o o o o o o o o o oo ooo o o oo oo oo o oo o oo o o −1.0 0.0 0.5 1.0

60

10

7

60

o o o o o o o oo o o o o oo ooo oo o oo o o o o o o o o o o o o o oo o o o o o o o o o o o o oo o o o oo o o o o oo o o o o o o o o o o o o o o o o o o o o o o oo o oo o o o oo o o o o o o o oo o o o o o o oo o o o o o oo o ooo o o o o o o o oo o o o o o o o oo o oo o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o oo oo o o o oo o ooo o o o o o o o oo o o o o o oo o ooo oo o o o o oo o o oo o o oo oo o o oo o o o o oo o o o o o o o o o oo o oo o oo o o oo o o oo o o o o o o o o o oo o o oo o o o oo o o o o o o o oo o oo o o o o o o o o o oo oo ooo o o oo oo o o o oo o o o o o o o oo o o o o oo o o o oo o o −1.0 0.0 0.5 1.0

0 20

6

o o o o oo oo o oo o o o o o o o o oo o o o o o o o o o o o o o oo o o o o o o o o o o oo o o o o o o o o o oo o o o o o o o o o o o o oo o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o oo oo oo o o o o oo o oo o o o oo o o o oo o o o o ooo o oo o o o o oo o oo o o o o o o o o o o o o o o ooo oo o oo o o o o o o o o o o o o oooo o o o o o o o o o o o o oo o ooo o o o o o o o o oo o o o o o o o o o oo oo o o o oo o o o o o oo o o o oo ooo o o o oo o o o o o o oo o o o oo o o o o o ooo o o oo o oo o o o o −1.0 0.0 0.5 1.0

100

0 20

60

o o o o o oo o o oo o o o o o o o o o o oo o oo ooo o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o oo o o o o oo o o o oo o o o o o o o o o o o o o o o o oo o o o o o oo o o oo o o o o o oo o o o o o o o o o o o o o o o o o oo o o o o o o oo o oo o o oo o o o oo o o o o o o oo oo o o o oo o ooo o o o o o o o o o o o o o o oo oo o o o oo ooo o oo o o o oo o oo o o o o o o o o o o o o o oo o oo o o oo o o o o o o o oo o o o o o o o o o o o o oo o o o o o o o oo o o o o oo oo o o o o o ooo o oo o o o o o o o oo o o o o o o o o oo oo o o ooo o o o oo o oo o o o o o o o o o oo −1.0 0.0 0.5 1.0

3

60

100 60 0 20 100

1.0

o o o o o o oo o o oo o o oo o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o oo oo o ooo o o o o oo o o o o o o oo oo o o oo oo o o o oo o o oo oo o o o o o o o o o o o o o o o o o o o o o ooo o o o o o o oo oo o oo o o o o o o o o o o o o o o oo o o oo o oo ooo o o o o o o o o o o o o o o o o o oo o oo o ooo o o ooo oo o oo o o o o o o o o o oo o o o oo oo oo o oo o oo o o oo o o o o o o oo ooo o o o oo oo o ooo o ooo o oo o o o o o o o oo oo ooo o o o o o o oo oo o o oo o o o o o o o o o o o o o o o o o o o o o o o oo o ooooo oo o o oo o oo o oo ooo o oo o o oo o o o oo o −1.0 0.0 0.5 1.0

0 20

0 20

60

o o o o o o o o o o o o o o oo o o o o o o o o oo o o o o o o o o o o o o o o oo o o o o o o o o oo o o o o o ooo o o o o o ooo o o oooo o o o o o o o ooo oo oo o o o oo o o o o o o o o o o ooo o o o o o o o o ooo o o o o o oo o oo o o o o oo o o o o o o o o o o o o oo oo o o o o o o o oo o o o oo o oo o o o o o o o o o o o o o o o o ooo oo ooo o oo o o o o oo o o oo o o o o oo oo o oo oo o o o oo o o oo o o o o o oo o o o o o o o o ooo o o o o o oo ooo o o o o o o oo o o o o o o o o o o o o o o ooo ooo o oo o o o o oo oo o o o o o o o o o o o o o ooo ooo o oo o o o o o o o o o o o o oo o oo oo ooo oo o o o o o o o o −1.0 0.0 0.5 1.0

0.5

o o oo oo oo o o o oo o o o o oo o o o o o o o oo o oooo o oo o o o ooo o o o o o oo oo o o o o o o o o o o o o o o oo o o o o o o o o oo o o o oo o oo o o oo o o o o o o o o o o o o o o o o o oo o o oo o o o o o o o o o o o o o oo oo oo o o o o o o ooo o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o oo o oo o o o o oo ooo o o o o o oo o o o o o o o o o oo oo oo o o o o oo o o o ooo o o o oo oo oo o oo o o o o o o oo o o o oo oo o o oooo oo o o o ooo o o o oo o o o oo ooo o o o o oo o oo o o o o o o o o o o o o o o o o o o o o o oo oo o ooo o oo oo o o ooo o o oo o oo ooo oo o o o oo o o ooo o −1.0 0.0 0.5 1.0

0 20

9

100

0 20

60

oo o ooo o o o o o o o o o oo oo oo o o o o o o o o o o o o o o o o o o o o o oo ooo o o o o o o oo oo o o o o o o o o o o oo o o o o o o oo o o oo o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o ooo ooo o o o o o o oo o o o o o o oo oo o o o o o o o o o o o o o oo oo o o oo oo o oo o oo o o o o o ooo o o o o o o oo o oo o o o o o o o o ooo o o o o oo oo oo o o o oo o o oo o o o o o o oo o o o o oo o o o o o o o o oo o oo o o o o o oo o o oo o o o o o o o o o o o o o o o o o ooo oo o o o o oo o o o o o o oo oo oo o o oo o o o o o ooo o o o o o o o o o o ooo o o o o o o o −1.0 0.0 0.5 1.0

0.0

100

5

100

0 20

60

o o o o oo o o o o o o o o oo o oo o o oo o o o oo o o o o o o oo o oo o o o oo o o o o oo o oo o o o o o o o o o o o o o o oo oo o o o o o oo ooo o o o o o oo o oo oo o o o o o o o o o o o o o o o oo o o o o o o o o o oo o oo o o o o o oo o o o o o o o o o o o o o o o o o o o o oo o o o o ooo o o oo o o o o oo oo o oo o o ooo o o oo o o o o oo o o o o o oo ooo o o o o o o o o o oo oo oo o o o o oo o o oo o o o o oo oo o o oo oo oo oo o o o o o oo o oo o o o o o o o ooo oo o oo o o o o o o o o o o o oo o o o ooo o o o o o o o o oo o o oo oo oo oo o o o o o o o o o oo oooo ooo oo o o o o o o oo o o o o o o o o o oo o o −1.0 0.0 0.5 1.0

2

−1.0

60

1.0

0 20

0.5

100

0.0

100

1

−1.0

o o o o oo o o o o o o o o o o o o oo o ooo o o o o o o o oo o o o o o o o o o o o o o o o o o o o o ooo o o ooo o o o o o oo o o oo o o o o oo oo o oo o o o o o o o oo oo o o oo o o o o o o o o o o o ooo oo o o o o o ooo o o o o o oo o oo o o o o o o o oo o o o oo o o o o o oooo o oo o o oo o o o o o o o o o o o o o o o o o o o o oo o oo o oo oo o oo o o oo oo oo oo o o oo oo o o oo o o o o o o oo oo oo o o o o o o o o o o o o o o o o oo o o oo o ooooooo ooo o o o o o o o o o o o oo oo o o oo o o o o ooo o o o o o oo oo o o o ooo oooo oo oo o o o o o ooo oo o o o o oo oo oo o o oooo o o o o oo oo o

60

oo o o o o o o o o o o o o o o o oo o ooo o o o o o o o o o oo o o o o o oo o o o o oo o o o o o oo o o o o o o oo o o ooo o oo o ooo o oo o o o o oo o o o o o o o o o o o o o o o o o o oo oo o o oo oo o o o o o o o o o o o o o o o o o o o oo o oo o o o o o o o o o oo ooo o o o o o o oo o o o oo o o o o o o o o o o o o o oo o o oo oo o ooo oo o ooo o o o o o o o o o o o oo oo o o oo oo o o o o o o o oo o o o o o o o o o o oo oo ooo o o o o o o o o o o ooooo o oo oo oo oo oo oo o o oo o oo o oo oo o o o oo o o o oo o o oo o oo o o o oo o o oo oo o o o o o o o o o o oo oo ooooo o o oo oo o o oo o

0 20

0 20

60

100

Figure 2.17: Multiple lagged scatterplots showing the relationship between the SOI at time t + h, say xt+h (x-axis) versus recruits at time t, say yt (y-axis), 0 ≤ h ≤ 15.

16

Figure 2.18: Multiple lagged scatterplots showing the relationship between the SOI at time t, say xt (x-axis) versus recruits at time t + h, say yt+h (y-axis), 0 ≤ h ≤ 15.

measured at lag h = 0. The general pattern suggests that predicting recruits might be possible using the El Ni˜no at lags of 5, 6 ,7, . . .

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

43

months. A measure of the correlation between several series, xt and yt is the cross covariance function (CCF), defined in terms of the counterpart covariance to (2.2), E{(xt+h − µx)(yt − µy )} = γxy (h). The cross correlation is the scaled (to lie between −1 and 1) version of above, say & ρxy (h) = γxy (h)/ γx(0) γy (0). The above quantities are expressed in terms of population averages and must generally be estimated from sample data. The estimated sample cross correlation functions can be used to investigate the possibility of the series being related at different lags. In order to investigate cross correlations between two series xt, yt, t = 1, . . . , n, we note that a reasonable sample version of ρxy (h) might be computed as 1 n−h $ " γxy (h) = (xt+h − x)(yt − y), n t=1 where y is the sample mean of {yt}nt=1. The estimated &sample cross correlation function then becomes ρ"xy (h) = γ" xy (h)/ γ" x(0) γ" y (0), where h denotes the amount that one series is lagged relative to the other. Generally, one computes the function ρ"xy (h) for a number of positive and negative values, h = 0, ±1, ±2, up to about 0.3 n, say and displays the results as a function of lag h. The maximum value, attained for h = 0, is ρxy (0) = 1 and the function takes values between −1 and 1. This fact makes it easy to compare values of the cross correlation with each other. Furthermore, under the hypothesis that there is not relation at time h and that at least one of the two series are independent and identically distributed, the distribution of ρ"xy (h) is approximately normal with mean 0 and standard

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

44

deviation given again by (2.3). Hence, one can compare values of the sample cross correlation with some appropriate number of sample standard deviations based on normal theory. Generally, values within ±1.96 σρ might be reasonable if one is willing to live with each test at a signi.cance level of 0.05. Otherwise, broader limits would be appropriate. In general, if m tests are made, each at level a the overall level of significance is bounded by m α. Example 1.16: To give an example, consider the cross correlations between the environmental series and recruitment shown in the bottom panel of Figure 2.16. The cross correlation between the SOI, xt+h and recruitment yt shows peaks at h = −6, −7 which implies that lagged products involving xt−6 and yt as well as those involving xt−7 and yt match up closely. The value shown on the graph is ρ"xy (−6) = −0.6. This means, in practice that the values of the SOI series tend to lead the recruitment series by 6 or 7 units. Also, one may note that, since the value is negative, lower values of SOI are associated with higher recruitment. The standard error in this case is approximately σρ = 0.047 and the −0.6 easily exceeds two standard deviations, shown as lines above and below the axis in Figure 2.16. Hence, we can reject the hypothesis that the correlation is zero at that lag. It is clear also that there are some periodic fluctuations apparent in the cross correlations. For example, in the SOI Recruitment example, there seems to be systematic fluctuation with a period (1 full cycle) of about 12 months. This produces a number of secondary peaks in the cross correlation function. The analysis of this periodic behavior and the accounting for periodicities in the series is considered in later sections or the books by Franses (1996) and Ghysels and Osborn (2001).

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

2.4.3

45

Partial Autocorrelation Function

A third kind of time series relationship expresses what is essentially the self predictability of a series through the partial autocorrelation function (PACF). There are several ways of thinking about this measure. One may regard the PACF as the simple correlation between two points separated by a lag h, say xt and xt−h, with the effect of the intervening points xt−1, xt−2, . . ., xt−h+1 conditioned out, i.e. the pure correlation between the two points. This interpretation is often given in more practical statistical settings. For example, one may get silly causal inferences by quoting correlations between two variables, e.g. teachers’ income and wine consumption, that may occur simply because both are correlated with some common driving factor, in this case, the gross domestic product, GDP, or some other force influencing disposable income. In time series analysis we are really more interested in the prediction or forecasting problem. In this case we might consider the problem of predicting xt based on observation observed h units back in the past, say xt−1, xt−2, . . ., xt−h. Suppose we want to predict xt from xt−1, . . ., xt−h using some linear function of these past values. Consider minimizing the mean square prediction error M SE = E[(xt − x" t)2]

using the predictor x" t = a1 xt−1 + a2 xt−2 + · · · + ah xt−h over the possible values of the weighting coefficients a1, . . ., ah where we assume, for convenience that xt has been adjusted to have zero mean. Consider the result of minimizing the above mean square predic-

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

46

tion error for a particular lag h. Then, the partial autocorrelation function is defined as the value of the last coefficient a" h, i.e. Φhh = ah. As a practical matter, we minimize the sample error sum of squares n $

SSE =

t=h+1



(xt − x) −



h $

k=1

2

ak (xt−k − x)

# " . with the estimated partial correlation defined as Φ hh = a h

The coefficients, as defined above, are also between −1 and 1 and have the usual properties of correlation coefficients. In particular, its standard error under the hypothesis of no partial autocorrelation is still (2.3). The intuition of the above argument is that the last coefficient will get very small once the forecast horizon or lag h is large enough to give good prediction. In particular, the order of the autoregressive model in the next chapter will be exactly the lag h beyond which Φhh = 0.

1.0

1.0

Example 1.17: As an example, we show in the right panels panel of Figure 2.19 the partial autocorrelation functions of the SOI series

0.0 −0.5

−0.5

0.0

0.5

PACF of Recruits

0.5

PACF of SOI

0

5

10

15

20

25

30

0

5

10

15

20

25

30

Figure 2.19: Partial autocorrelation functions for the SOI (left panel) and the recruits (right panel) series.

(left panel) and the recruits series (right panel). Note that the PACF

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

47

of the SOI has a single peak at lag h = 1 and then relatively small values. This means, in effect, that fairly good prediction can be achieved by using the immediately preceding point and that adding further values does not really improve the situation. Hence we might try an autoregressive model with p = 1. The recruits series has two peaks and then small values, implying that the pure correlation between points is summarized by the first two lags. A major application of the PACF is to diagnosing the appropriate order of an autoregressive model for the series under consideration. Autoregressive models will be studied extensively later but we note here that they are linear models expressing the present value of a series as a linear combination of a number of previous values, with an additive error. Hence, an autoregressive model using two previous values for prediction might be written in the form xt = φ1 xt−1 + φ2 xt−2 + wt where φ1 and φ2 are fixed unknown coefficients and wt are values from an independent series with zero mean and common variance σw2 . For example, if the PACF at lag h is roughly zero beyond a fixed value, say h = 2 as observed for the recruits series in the left panel of Figure 2.19, then one might assume a model of the form above for that recruits series. To finish the introductory discussion, we note that the extension of what has been said above to multivariate time series is fairly straightforward if one restricts the discussion to relations between two series at a time. The two series xt and yt can be laid out in the vector (xt, yt) which becomes a multivariate series of dimension two (bivariate). To generalize, consider the p series (xt1, xt2, . . . , xtp) = xt as the row vector defining the multivariate series xt. The autocovariance matrix of the vector xt can then be defined as the p × p matrix

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

48

containing as elements γij (h) = E{(xt+h,i − µi)(xtj − µj )} and the analysis of the cross covariance structure can proceed on two elements at a time. The discussion of possible multiple relations is deferred until later. To recapitulate, the primary objective of this chapter has been to define and illustrate with real examples three statistics of interest in describing relationships within and among time series. The autocorrelation function measures the correlation over time in a single series; this correlation can be exploited for prediction purposes or for suggesting periodicities. The cross correlation function measures the correlation between series over time; it may be, for example, that a series is better related to the past of another series than it is to its own past. The partial autocorrelation function gives a direct measure of the lag length necessary to predict a series from itself, i.e. to forecast future values. It is also critical in determining the order of an autoregressive model satisfied by some real data set. It should be noted that all three of the measures given in this section can be distorted if there are significant trends in the data. It is obvious that lagged products of the form (xt+h − x)(xt − x) will be artificially large if there is a trend present. Since the correlations of interest are usually associated with the stationary part of the series, i.e., the part that can be thought of as being superimposed on trend, it is usual to evaluate the correlations of the detrended series. This means, in effect, that we replace x in the equations by a" + b" t if the trend can be considered to be linear. If the trend is quadratic or logarithmic, the appropriate alternate nonlinear predicted value is subtracted before computing the lagged products. Note also that

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

49

differencing the series, as discussed in Section 2.2.2, can accomplish the same result. 2.5

Problems

1. Consider a generalization of the model given in Example 1.1, namely, xt = µ + wt − θ wt−1, where {wt} are independent zeromean random variables with variance σw2 . Prove that E(xt) = µ, γx(0) = (1 + θ2)σw2 , γx(1) = −θ σw2 , γx(h) = 0 if |h| > 1, and finally show that xt is weakly stationary. 2 Consider the time series generated by x1 = µ + w1 and xt = µ + xt−1 + wt for t ≥ 2. Show that xt is not stationary no matter µ = 0 or not and find γx(h). 3. Suppose that xt is stationary with mean µx and covariance function given by γx(h). Find the mean and covariance function of (a) yt = a + b xt, where a and b are constants, (b) zt = xt − xt−1. 4. Consider the linear process yt =

∞ $

j=−∞

aj wt−j ,

where wt is a white noise process with variance σw2 and aj , j = 0, ±1, ±2, ldots, are constants. The process yt will exist (as a % limit in mean square) if j |aj | < ∞; you do not need to prove this. Show that the series yt is stationary, with autocovariance function ∞ $ aj+h aj . γy (h) = σw2 j=−∞

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

50

Apply the result to calculating the autocovariance function of the 3-point moving average (xt−1 + xt + xt+1)/3. For the following problems, you need to use a computer package. 5. Melting glaciers deposit yearly layers of sand and silt during the spring melting seasons which can be reconstructed yearly over a period ranging from the time de-glaciation began in New England (about 12,600 years ago) to the time it ended (about 6, 000 years ago). Such sedimentary deposits, called varves, can be used as a proxy for paleoclimatic parameters such as temperature. The file mass2.dat contains yearly records for 634 years beginning 11, 834 years ago, collected from one location in Mas-

100 0

50

varve thickness

150

Varve thickness from Massachusetts (n=634)

0

100

200

300

400

500

600

year

Figure 2.20: Varve data for Problem 5.

sachusetts. For further information, see Shumway and Verosub (1992). (a) Plot the varve records and examine the autocorrelation and partial autocorrelation functions for evidence of nonstationarity. (b) Argue that the transformation yt = log xt might be useful for stabilizing the variance. Compute γx(0) and γy (0) over two time

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

51

intervals for each series to determine whether this is reasonable. Plot the histograms of the raw and transformed series. (c) Plot the autocorrelation of the series yt and argue that a first difference produces a reasonably stationary series. Can you think of a practical interpretation for ut = yt −yt−1 = log xt −log xt−1? (d) Compute the autocorrelation function of the differenced transformed series and argue that a generalization of the model given by Example 1.1 might be reasonable. Assume that ut = wt − θ wt−1 is stationary when the inputs wt are assumed independent with E(wt) = 0 and (wt2) = σw2 . Using the sample ACF and the printed autocovariance γ" u(0), derive estimators for θ and σw2 . 6. Two time series representing average wholesale U.S. gasoline and oil prices over 180 months, beginning in July, 1973 and ending in December, 1987 are given in the file oil-gas.dat. Analyze the data using some of the techniques in this chapter with the idea that one should be looking at how changes in oil prices influence 700

Gas and oil prices (n=180 months)

400 100

200

300

price

500

600

GAS OIL

0

50

100

150

month

Figure 2.21: Gas and oil series for Problem 6.

changes in gas prices. For further reading, see Liu (1991). In

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

52

particular, consider the following options: (a) Plot the raw data and look at the autocorrelation functions to argue that the untransformed data series are nonstationary. (b) It is often argued in economics that price changes are important, in particular, the percentage change in prices from one month to the next. On this basis, argue that a transformation of the form yt = log xt − log xt−1 might be applied to the data where xt is the oil or gas price series. (c) Use lagged multiple scatterplots and the auto and cross correlation functions of the transformed oil and gas price series to investigate the properties of these series. Is it possible to guess whether gas prices are raised more quickly in response to increasing oil prices than they are decreased when oil prices are decreased ? Do you think that it might be possible to predict log percentage changes in gas prices from log percentage changes in oil prices? Plot the two series on the same scale. 7. Monthly Handgun Sales and Firearms Related Deaths in California Legal handgun purchase information for 227 months spanning the time period February 1, 1980 through December 31, 1998 was obtained from the Department of Justices automated Firearms systems database. California resident firearms death data was obtained form the California Department of Health Services. The data are plotted in the figure, with both rates given in numbers per 100,000 residents. Suppose that the main question of interest for this data pertains to the possible relations between handgun sales and death rates over this time period. Include the possibility of lagging relations in your analysis. In particular, answer the questions below:

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

53

2.0

Gun sales and gun death rate

1.0

1.5

Hangun sales/100 per 100,000

Gun death rate per 100,000

0

50

100 months

150

200

Figure 2.22: Handgun sales (per 10,000,000) in California and monthly gun death rate (per 100,00) in California (February 2, 1980 -December 31, 1998.

(a) Use scatterplots to argue that there is a potential nonlinear relation between death rates and handgun sales and indicate whether you think that there might be a lag. (b) Bolster your argument for a lagging relationship by examining the cross correlation function. What do the autocorrelation functions indicate for this data? (c) Examine the first difference for the two processes and indicate what the ACF’s and CCF’s show for the differenced data. (d) Smooth the two series with a 12 point moving average and plot the two series on the same graph. Subtract the moving average from the original unsmoothed series. What do the residual series show in the ACF and CCF for this case? 2.6

Computer Code

The following R commands are used for making the graphs in this chapter.

CHAPTER 2. CHARACTERISTICS OF TIME SERIES

# 3-28-2006 graphics.off()

54

# clean the previous graph on the scre

############################################################ # This is Southern Oscillation Index data and Recruits data ############################################################

y −0.5, then xt is invertible and has the infinite AR representation. x t = wt +

∞ $

k=1



 ψk wt−k with ψk = (0−d)(1−d) · · · (k−1−d)/k! = 

k−d k

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

134

3. For |d| < 0.5, the ACF of xt is ρx(h) =

d(1 + d) · · · (h − 1 + d) , (1 − d)(2 − d) · · · (h − d)

h ≥ 1.

In particular, ρx(1) = d/(1 − d) and as h → ∞, ρx(h) ≈

(−d)! 2d−1 h . (d − 1)!

4. For |d| < 0.5, the PACF of xt is φh,h = d/(h − d) for h ≥ 1. 5. For |d| < 0.5, the spectral density function fx(·) of xt, which is the Fourier transform of the ACF γx(h) of xt, that is 1 fx(ν) = 2π

∞ $

h=−∞

for ν ∈ [−π, π], where i =

√

γx(h) exp(−i h ν)

−1, satisfies

fx(ν) ∼ ν −2d as ν → 0,

(3.38)

where ν ∈ [0, π] denotes the frequency. See Chapter 6 of Hamilton (1994) for details about the spectral analysis. Of particular interest here is the behavior of ACF of xt when d < 0.5. The property says that ρx(h) ∼ h2d−1, which decays at a polynomial, instead of exponential rate. For this reason, such an xt process is called a long-memory time series. A special characteristic of the spectral density function in (3.38) is that the spectrum diverges to infinity as ν → 0. However, the spectral density function of a stationary ARMA process is bounded for all ν ∈ [−π, π]. Earlier we used the binomial theorem for non-integer powers (1 − L)d =

∞ $

 

 (−1)k 

k=0

d k L . k

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

135

0.4

0.4

If the fractionally differenced series (1−L)d xt follows an ARMA(p, q) model, then xt is called an fractionally differenced autoregressive moving average (ARFIMA(p, d, q)) process, which is a generalized ARIMA model by allowing for non-integer d. In practice, if the sample ACF of a time series is not large in magnitude, but decays slowly, then the series may have long memory. For more discussions, we refer to the book by Beran (1994). For the pure fractionally differenced model in (3.37), one can estimate d using either a maximum likelihood method in the time domain or the Whittle likelihood or a regression method with logged periodogram at the lower frequencies in the frequency domain. Finally, long-memory models have attracted some attention in the finance literature in part because of the work on fractional Brownian motion in the continuous time models.

100

200

300

400

0

100

200

300

400

Log Spectral Density of EW

−13

−11

Log Spectral Density of VW

−0.1 0.0

0.1

0.2

0.3

ACF for equal−weighted index

−9 −8 −7 −6

0

−12 −11 −10 −9 −8 −7 −6

−0.1 0.0

0.1

0.2

0.3

ACF for value−weighted index

0.00

0.02

0.04

0.06

0.00

0.02

0.04

0.06

Figure 3.18: Sample autocorrelation function of the absolute series of daily simple returns for the CRSP value-weighted (left top panel) and equal-weighted (right top panel) indexes. The log spectral density of the absolute series of daily simple returns for the CRSP value-weighted (left bottom panel) and equal-weighted (right bottom panel) indexes.

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

136

Example 2.15: As an illustration, Figure 3.18 show the sample ACFs of the absolute series of daily simple returns for the CRSP value-weighted (left top panel) and equal-weighted (right top panel) indexes from July 3, 1962 to December 31, 1997. The ACFs are relatively small in magnitude, but decay very slowly; they appear to be significant at the 5% level even after 300 lags. For more information about the behavior of sample ACF of absolute return series, see Ding, Granger, and Engle (1993). To estimate the long memory parameter estimate d, we can use the package fracdiff in R and results are d" = 0.1867 for the absolute returns of the value-weighted index and d" = 0.2732 for the absolute returns of the equal-weighted index. To support our conclusion above, we plot the log spectral density of the absolute series of daily simple returns for the CRSP value-weighted (left bottom panel) and equal-weighted (right bottom panel). They show clearly that both log spectral densities decay like a log function and they support the spectral densities behavior like (3.38). 3.12

Periodicity and Business Cycles

Let us first recall what we have observed from from Figure 2.1 in Chapter 2. From Figure 2.1, we can conclude that both series in tend to exhibit repetitive behavior, with regularly repeating (stochastic) cycles or periodicity that are easily visible. This periodic behavior is of interest because underlying processes of interest may be regular and the rate or frequency of time series characterizing the behavior of the underlying series would help to identify them. One can also remark that the cycles of the SOI are repeating at a faster rate than those of the recruitment series. The recruits series also shows several kinds of oscillations, a faster frequency that seems to repeat about every 12 months and a slower frequency that seems to repeat

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

137

about every 50 months. The study of the kinds of cycles and their strengths are also very important, particularly in macroeconomics to determine the business cycles. For more discussions, we refer to the books by Franses (1996, 1998) and Ghysels and Osborn (2001). As we mention in Chapter 2, one way to identify the cycles is to compute the power spectrum which shows the variance as a function of the frequency of oscillation. For other modeling methods such as periodic autoregressive model (PAR), PAR(p) modeling techniques, we refer to the books by Franses (1996, 1998) and Ghysels and Osborn (2001) for details. Next, we introduce one method to describe the cyclical behavior using the ACF. As indicated above, there exits cyclical behavior for the recruits. From Example 2.4, an AR(2) fits this series quite well. Therefore, we consider the ACF ρx(h) of a stationary AR(2) series, which satisfies the second order difference equation (1 − φ1 L − φ2 L2) ρx(h) = φ(L) ρx (h) = 0 for h ≥ 2, ρx(0) = 1, and ρx(1) = φ1/(1 − φ2). This difference equation determines the properties of the ACF of a stationary AR(2) time series. It also determines the behavior of the forecasts of xt. Corresponding to the prior difference equation, there is a second order polynomial equation 1 − φ1 x − φ2 x2 = 0. Solutions of this equation are G

φ1 ± φ21 + 4 φ2 x= . −2 φ2

In the time series literature, the inverse of two solutions are referred as the characteristic roots of the AR(2) model. Denote the

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

138

two solutions by ω1 and ω2. If both ωi are real valued, then the second order difference equation of the model can be factored as (1 − ω1 L)(1 − ω2 L) and the AR(2) model can be regarded as an AR(1) model operates on top of another AR(1) model. The ACF of xt is then a mixture of two exponential decays. Yet if φ21 + 4 φ2 < 0, then ω1 and ω2 are complex numbers (called a complex conjugate pair), and the plot of ACF of xt would show a picture of damping sine and cosine waves; see the top two panels of Figure 2.14 for the ACF of the SOI and recruits. In business and economic applications, complex characteristic roots are important. They give rise to the behavior of business cycles. It is then common for economic time series models to have complex valued characteristic roots. For an AR(2) model with a pair of complex characteristic roots, the average length of the stochastic cycles is T0 =

2π √ , cos−1[φ1/(2 −φ2)]

where the cosine inverse is stated in radians. If one wishes√the complex solutions as a ± b i, then we have φ1 = 2 a, φ2 = − a2 + b2, and 2π √ T0 = , cos−1(a/ a2 + b2) √ where a2 + b2 is the absolute value of a ± b i. To illustrate the above idea, Figure 3.19 shows the ACF of four stationary AR(2) models. The right top panel is the ACF of the AR(2) model (1 − 0.6 L + 0.4 L2 ) xt = wt. Because φ21 + 4 φ2 = 1.0 + 4 × (−0.7) = −1.8 < 0, this particular AR(2) model contains two complex characteristic roots, and hence its ACF exhibits damping sine and cosine waves. The other three AR(2) models have realvalued characteristic roots. Their ACFs decay exponentially.

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

139

0.5 0.0 −0.5 5

10

15

20

−1.0

−1.0

−0.5

0.0

0.5

1.0

(b)

1.0

(a)

5

10

20

15

20

0.5 0.0 −0.5 5

10

15

20

−1.0

−1.0

−0.5

0.0

0.5

1.0

(d)

1.0

(c)

15

5

10

Figure 3.19: The autocorrelation function of an AR(2) model: (a) φ1 = 1.2 and φ2 = −0.35, (b) φ1 = 1.0 and φ2 = −0.7, (c) φ1 = 0.2 and φ2 = 0.35, (d) φ1 = −0.2 and φ2 = 0.35.

0.0

−0.02 −0.01

0.2

0.00

0.4

0.01

0.6

0.02

0.8

0.03

1.0

0.04

Example 2.16: As an illustration, consider the quarterly growth rate of U.S. real gross national product (GNP), seasonally adjusted, from the second quarter of 1947 to the first quarter of 1991, which is shown in the left panel of Figure 3.20. The right panel of Figure

1950

1960

1970

1980

1990

0

5

10

15

20

25

30

Figure 3.20: The growth rate of US quarterly real GNP from 1947.II to 1991.I (seasonally adjusted and in percentage): the left panel is the time series plot and the right panel is the ACF.

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

140

3.20 displays the ACF of this series and it shows that a picture of damping sine and cosine waves. We can conclude that cycles exist. This series can be used as an example of nonlinear economic time series; see Tsay (2005, Chapter 4) for the detailed analyses using the Markov switching model. Here we simply employ an AR(3) model for the data. Denoting the growth rate by xt, we can use the model building procedure to estimate the model. The fitted model is xt = 0.0047+0.35 xt−1+0.18 xt−2−0.14 xt−3+wt,

#2 = 0.0098. with σ w

Rewriting the model as xt − 0.35 xt−1 − 0.18 xt−2 + 0.14 xt−3 = 0.0047 + wt, we obtain a corresponding third-order difference equation (1 − 0.35 L − 0.18 L2 + 0.14 L3) = 0, which can be factored as (1 + 0.52 L)(1 − 0.87 L + 0.27 L2) = 0. The first factor (1 + 0.52 L) shows an exponentially decaying feature of the GNP growth rate. Focusing on the second order factor 1 − 0.87 L − (−0.27) L2 = 0, we have φ21 + 4 φ2 = 0.872 + 4(−0.27) = −0.3231 < 0. Therefore, the second factor of the AR(3) model confirms the existence of stochastic business cycles in the quarterly growth rate of U.S. real GNP. This is reasonable as the U.S. economy went through expansion and contraction periods. The average length of the stochastic cycles is approximately T0 =

2π √ = 10.83 quarters, cos−1[φ1/(2 −φ2)]

which is about 3 years. If one uses a nonlinear model to separate U.S. economy into “expansion” and “contraction” periods, the data show that the average duration of contraction periods is about three quarters and that of expansion periods is about 3 years. The average duration of 10.83 quarters is a compromise between the two separate durations. The periodic feature obtained here is common among

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

141

growth rates of national economies. For example, similar features can be found for other countries. For a stationary AR(p) series, the ACF satisfies the difference equation (1 − φ1 L − φ2 L2 − · · · − φp Lp) ρx(h) = 0, for h > 0. The plot of ACF of a stationary AR(p) model would then show a mixture of damping sine and cosine patterns and exponential decays depending on the nature of its characteristic roots. Finally, we continue our analysis of the recruits series as entertained in Example 2.4, from which an AR(2) model is fitted for this series as xt − 1.3512 xt−1 + 0.4612 xt−2 = 61.8439 + wt. Clearly, φ21 + 4 φ2 = 1.35122 − 4 × 0.4612 = −0.0191 < 0, which implies the existence of stochastic business cycles in the recruits series. The average length of the stochastic cycles based on the above fitted AR(2) model is approximately T0 =

2π √ = 61.71 months, cos−1[φ1/(2 −φ2)]

which is about 5 years. Note that this average length of cycles is not close to what we have observed (about 50 months). Please figure out the reason why there is a big difference. 3.13

Impulse Response Function

The task facing the modern time-series econometrician is to develop reasonably simple and intuitive models capable of forecasting, interpreting and hypothesis testing regarding economic and financial data. In this respect, the time series econometrician is often concerned with

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

142

the estimation of difference equations containing stochastic components. 3.13.1

First Order Difference Equations

Suppose we are given the dynamic equation yt = φ1 yt−1 + wt, where yt is the value of variable at period t, wt is the value of variable at period t, 1 ≤ t ≤ n. Indeed, this is an AR(1) model. The equation relates a variable yt to its previous (lagged) values with only the first lag appears on the right hand side (RHS) of the equation. For now, the input variable, {w1, w2, w3, . . .}, will simply be regarded as a sequence of deterministic numbers which can be generated from a distribution. Later on, we will assume that they are stochastic. By solving a difference equation by recursive substitution, assuming that we know the starting value of y−1, called the initial condition, then, we have

y0 = φ1 y−1 + w0, y1 = φ1 y0 + w1 = φ(φ1 y−1 + w0) + w1 = φ21 y−1 + φ1 w0 + w1, y2 = φ1 y1 + w2 = φ1(φ21 y−1 + φ1 w0 + w1) + w2 = φ31 y−1 + φ21 w0 + φ1 w1 + ... t t−1 yt = φ1 yt−1 + wt = φt+1 w1 + · · · + φ1 wt−1 + wt 1 y−1 + φ1 w0 + φ =

φt+1 1 y−1

+

t $

k=0

φk1 wt−k .

The procedure to express yt in term of the past values of wt and the starting value y−1 is known as recursive substitution. The last term on RHS of (3.39) is called the linear process (with finite terms) generated by {wt} if {wt} is random. Next we consider the computation of dynamic multiplier. To do so, we consider one simple experiment by assuming that y−1 and

(3

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

143

{w0, w1, · · · , wn} are given, fixed at this moment, and the value of φ1 is known. Then we can compute the time series yt by yt = φ1 yt−1 + wt for 1 ≤ t ≤ n. What happens to the series yt if at the period t = 50, we change the value of deterministic component w50 H and set the new w to be w 50 = w50 + 1? Of course, the change in w50 leads to changes in yt: H y!50 = φ1 y49 + w 50 = φ1 y49 + w50 + 1 = y50 + 1, y!51 = φ1 y50 + w51 = φ1(y50 + 1) + w51 = y51 + φ1, y!52 = φ1 y!51 + w52 = φ1(y51 + φ1) + w52 = y52 + φ21,

4

and so on. If we look at the difference between y!t and yt then we obtain that: y!50 − y50 = 1, y!51 − y51 = φ1, y!52 − y52 = φ21, and so on. This experiment is illustrated in Figures 3.21 and 3.22. Therefore, No Impulse With Impulse

−2

−2

0

0

2

2

No Impulse With Impulse

phi= − 0.8

40

60

80

100

−4

20

0

40

60

80

100

80

100

0.0 −0.5

Impulse Response Function

0

20

40

60

80

100

−1.0

−0.5

0.0

0.5

phi= − 0.8

0.5

phi=0.8

−1.0

20

1.0

0

1.0

−4

phi=0.8

Impulse Response Function

0

20

40

60

Figure 3.21: The time-series yt is generated with wt ∼ N (0, 1), y0 = 5. At period t = 50, there is an additional impulse to the error term, i.e. w!50 = w50 + 1. The impulse response function is computed as the difference between the series yt without impulse and the series y!t with the impulse.

one can say that one unit increase in w50 leads to y50 increased by 1,

144

10 15 20 25 30

20

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

No Impulse With Impulse

−10

0

10

No Impulse With Impulse

60

80

100

0

20

40

60

80

100

80

100

1

1

2

20

2

0

−20

5

phi= − 1.01 phi=1.01 40

0

phi= − 1.01

0

phi=1.01

−2

−2

−1

Impulse Response Function

−1

Impulse Response Function

0

20

40

60

80

100

0

20

40

60

Figure 3.22: The time-series yt is generated with wt ∼ N (0, 1), y0 = 3. At period t = 50, there is an additional impulse to the error term, i.e. w!50 = w50 + 1. The impulse response function is computed as the difference between the series yt without impulse and the series y!t with the impulse.

y51 increased by φ1, y52 increased by φ21, and so so. The question is how yt+j changes if change wt by one? This is exactly the question that dynamic multipliers answer. Assume that we start with yt−1 instead of y−1, i.e. we observe the value of yt−1 at period t − 1. Can we say something about yt+j ? Well, let us answer this question. yt = φ1 yt−1 + wt, yt+1 = φ1 yt + wt+1 = φ1(φ1 yt−1 + wt) + wt+1 = φ21 yt−1 + φ1 wt + wt+1, ... yt−1 + φj1 wt + φj−1 wt+1 + · · · + φ1 wt+j−1 + wt+j yt+j = φj+1 1 The dynamic multiplier is defined by ∂yt+j = φj1 for an AR(1) model. ∂wt

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

145

Note that the multiplier does not depend on t. The dynamic multiplier ∂yt+j /∂wt is called sometimes the impact multiplier. Impulse Response Function

We can plot dynamic multipliers as a function of lag j, i.e plot ∂y { ∂wt+jt }Jj=1. Because dynamic multipliers calculate the response of yt+j to a single impulse in wt, it is also referred to as the impulse response function (IRF). This function has many important applications in time series analysis because it shows how the entire path of a variable is effected by a stochastic shock. Obviously, the dynamic of impulse response function depends on the value of φ1 for an AR(1) model. Let us look at what the relationship is between the IRF and system. (a) 0 < φ1 < 1 implies that the impulse response converges to zero and the system is stable. (b) −1 < φ1 < 0 gives that the impulse response oscillates but converges to zero and the system is stable. (c) φ1 > 1 concludes that the impulse response is explosive and the system is unstable. (d) φ1 < −1 implies that the impulse response is explosive (oscillationally) and the system is unstable. Impulse response functions for all possible cases are presented in Figure 3.23. Permanent Change in wt

In calculating dynamic multipliers in Figure 3.23, we were asking what would happen if wt were to increase by one unit with wt+1, wt+2, · · · , , wt+

o

30

o o

20

phi1=1.2

o

o

o o o o o o o o o o 5 10

o

o

o

1.0

146

o

o

o

o

0.99 o phi1= o −o o

o

o

0.5

o

o

o

phi1= − 0.5 o

0.0

o

−0.5

o o o o

− 0.9 o phi1= o o o o o o o o o o o o o o o o o o o o o o o o o o o o

o 5

10

o

o

o

o

o

o

o

o

o

15

20

o o phi1= − 1.2

o

o

o

o

o

o

o

o

o

o

o

o

o

o

o o o

−30

0

10

o o

o o

−1.0

o o o o o o o o o o o o phi1=0.99 o o o o o o o o o o o o o o o o o o o o phi1=0.5 o ophi1=0.9 o o o o o o o o o o o o o o o o o o o o o o o o o o 5 10 15 20

−10 0 10 20 30 40

0.0

0.2

0.4

0.6

0.8

1.0

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

15

20

o 5

10

15

20

Figure 3.23: Example of impulse response functions for first order difference equations.

unaffected. We were finding the effect of a purely transitory change in wt. Permanent change in wt means that wt, wt+1, · · · , , wt+j would all increase by one unit. The effect on yt+j of a permanent change in wt beginning in period t is then given by ∂yt+j ∂yt+j 1 − φj+1 ∂yt+j 1 j−1 j + +· · ·+ = φ1+φ1 +· · ·+φ1 = ∂wt ∂wt+1 ∂wt+j 1 − φ1

for the AR(1) m

The different between transitory and permanent change in wt is illustrated in Figure 3.24. 3.13.2

Higher Order Difference Equations

Consider the model: A linear pth order difference equation has the following form: yt = φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + wt, which is an AR(p) model if wt is the white noise. It is easy to drive the properties of the pth order difference equation. To illustrate the properties of pth order difference equation, we consider the second

147

2

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

4

No Impulse With Impulse

−2

0

−3 −2 −1

0

2

1

No Impulse With Impulse

phi=0.8 40

60

80

100

0

20

40

60

80

100

80

100

0.8

5

−4

20

1.0

0

phi=0.8

0.6

4

phi=0.8

Impulse Response Function

0.4

2 1

phi=0.8

0

0.0

0.2

3

Impulse Response Function

0

20

40

60

80

100

0

20

40

60

Figure 3.24: The time series yt is generated with wt ∼ N (0, 1), y0 = 3. For the transitory impulse, there is an additional impulse to the error term at period t = 50, i.e. w!50 = w50 + 1. For the permanent impulse, there is an additional impulse for period t = 50, · · ·, 100, i.e. w!t = wt + 1, t = 50, 51, · · ·, 100. The impulse response function (IRF) is computed as the difference between the series yt without impulse and the series y!t with the impulse.

order difference equation: yt = φ1 yt−1 + φ2 yt−2 + wt so that p = 2. We write equation as a first order vector difference equation. ξ t = F ξ t−1 + vt,

where





y ξt =  t  , yt−1





φ φ2  , F= 1 1 0





w and vt =  t  . 0

If the starting value ξ −1 is known we use the recursive substitution to obtain: ξ t = Ft+1 ξ −1 + Ft v0 + Ft−1 v1 + · · · + F vt−1 + vt. Or if the starting value ξ t−1 is known we use the recursive substitution to obtain: ξ t+j = Fj+1 ξ t−1 + Fj vt + Fj−1 vt+1 + · · · + F vt+j−1 + vt+j

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

148

for any j ≥ 0. To compute the dynamic multipliers, we recall the rules of of vector and matrix differentiation. If x(β) is an m × 1 vector that depends on the n × 1 vector β, then,   ∂x1 (β ) ∂x1 (β ) ···  ∂β1 ∂βn    ∂x   . . .  . . . . = . . .   ∂β m×n  ∂xm(β ) ∂xm (β )  ··· ∂β1

∂βn

It is easy to verify that the dynamic multipliers are given by ∂ξ t+j ∂Fj vt = = Fj . ∂vt vt

Since the first element in ξ t+j is yt+j and the first element in vt is wt then the element(1, 1) in the matrix Fj is ∂yt+j /∂wt, i.e. the dynamic multiplier. For larger values of j, an easy way to obtain a numerical values for dynamic multiplier ∂yt+j /∂wt is to simulate the system. This is done as follows. Set y−1 = y−2 = · · · = y−p = 0 and w0 = 1, and set the values of w for all other dates to 0. Then use AR(p) to calculate the value for yt for ≤ t ≤ n. To illustrate dynamic multipliers for the pth order difference equation, we consider the second order difference equation as follows:   φ φ2  . The impulse yt = φ1 yt−1 + φ2 yt−2 + wt, so that F =  1 1 0 response functions for this example with four different settings are presented in Figure 3.25: (a) φ1 = 0.6 and φ2 = 0.2, (b) φ1 = 0.8 and φ2 = 0.4, (c) φ1 = −0.9 and φ2 = −0.5, and

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

149 (b) o

10 12

0.8

1.0

(a) o

Impulse Response Function

o Impulse Response Function o

0.6

8

o o

o

0.4

o

o

o

0

5

o

o o o o o o o o o o o

10

15

2

o

0.0

0.2

o

phi1=0.6 and phi2=0.2

o o o o o o o o o

20

0

5

0 10 20 30

o

0.0

o

o o o o o o o o o o o o

−0.5 −1.0

o

5

20

o

Impulse Response Function o

o o o o o o o

o

o o

o o

o

o

o

10

15

phi1= − 0.5 and phi2= − 1.5

−20

phi1= − 0.9 and phi2= − 0.5

o

15

o

20

−40

1.0 0.5

o o

o

(d)

Impulse Response Function

o

o

o

10

(c) o

o

o

o

0

o

4

o

0

o

phi1=0.8 and phi2=0.4

6

o

o

o

o

0

5

10

15

20

Figure 3.25: Example of impulse response functions for second order difference equation.

(d) φ1 = −0.5 and φ2 = −1.5. Similar to the first order difference equation, the impulse response function for the pth order difference equation can be explosive or converge to zero. What determines the dynamics of impulse response function? The eigenvalues of matrix F determine whether the impulse response oscillates, converges or is explosive. The eigenvalues of a p × p matrix A are those numbers λ for which |A − λ Ip| = 0. The eigenvalues of the general matrix F defined above are the values of λ that satisfy: λp − φ1 λp−1 − φ2 λp−2 − · · · − φp−1 λ − φp = φ(λ−p) = 0. Note that the eigenvalues are also called the characteristic roots of AR(p) model. If the eigenvalues are real but at least one eigenvalue is greater than unity in absolute value, the system is explosive.

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

150

Why do eigenvalues determine the dynamics of dynamic multipliers? Recall that if the eigenvalues of p × p matrix A are distinct, there exists a nonsingular p × p matrix T such that A = T Λ T−1, where Λ is a p × p matrix with the eigenvalues of A on the principal diagonal and zeros elsewhere. Then, Fj = T Λj T−1 and the value of eigenvalues in Λ determines whether the elements of Fj explode or not. Recall that the dynamic multiplier is equal to ∂ξ t+j /∂v-t = Fj . Therefore, the size of eigenvalues determines whether the system is stable or not. Now we compute the the eigenvalues for each case in the above example: (a) λ1 = 0.838 and λ2 = −0.238 so that |λk | < 1 and the system is stable. (b) λ1 = 1.148 and λ2 = −0.348 so that |λ1| > 1 and the system is unstable. &

(c) λ = −0.45 ± 0.545 i so that |λ| = (−0.45)2 + 0.5452 = 0.706 < 1 and the system is stable. Since eigenvalues are complex, the impulse response function oscillates. &

(d) λ = −0.25 ± 1.198 i so that |λ| = (−0.25)2 + 1.1982 = 1.223 > 1 and the system is unstable. Since eigenvalues are complex, the impulse response function oscillates. 3.14

Problems

1. Consider the regression model yt = β1 yt−1 + wt, where wt is white noise with zero-mean and variance σw2 . Assume that we observe {yt}nt=2. Show that the least squares estimator of β1 is % % 2 β#1 = nt=2 yt yt−1/ t=2 yt−1 . If we pretend that yt−1 would be

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

151

%

2 . Relate your answer to fixed, show that Var(β#1) = σe2/ nt=2 yt−1 a method for fitting a first-order AR model to the data yt.

2. Consider the autoregressive model AR(1), i.e. xt −φ1 xt−1 = wt. (a) Find the necessary condition for {xt} to be invertible. (b) Show that xt can be expressed as a linear process. (c) Show that E[wt xt] = σw2 and E[wt xt−1] = 0, so that future errors are uncorrelated with past data. 3. The auto-covariance and autocorrelation functions for AR processes are often derived from the Yule-Walker equations, obtained by multiplying both sides of the defining equation, successively by xt, xt−1, · · ·. Use the Yule-Walker equations to drive ρx(h) for the first-order AR. 4. For an ARMA series we define the optimal forecast based on xt, xt−1, · · · as the conditional expectation xtt+h = E[xt+h | xt, xt−1, · · ·] for h ≥ 1. (a) Show, for the general ARMA model that E[wt+h | xt, xt−1, · · ·] = 0 if h > 0, wt+h if h ≤ 0. (b) For the AR(1) and AR(2) models, derive the optimal forecast xtt+h and the prediction error variance of the one-step forecast. 5. Suppose we have the simple linear trend model yt = β1 t + xt, 1 ≤ t ≤ n, where xt = φ1 xt−1 + wt. Give the exact form of the equations that you would use for estimating β1, φ1 and σw2 using the Cochran-Orcutt procedure. 6. Suppose that the simple return of a monthly bond index follows the MA(1) model xt = wt + 0.2 wt−1,

σw = 0.025.

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

152

Assume that w100 = 0.01. Compute the 1-step (xtt+1) and 2step (xtt+2) ahead forecasts of the return at the forecast origin t = 100. What are the standard deviations of the associated forecast errors? Also compute the lag-1 (ρ(1)) and lag-2 (ρ(2)) autocorrelations of the return series. 7. Suppose that the daily log return of a security follows the model xt = 0.01 + 0.2 xt−2 + wt, where {wt} is a Gaussian white noise series with mean zero and variance 0.02. What are the mean and variance of the return series xt? Compute the lag-1 (ρ(1)) and lag-2 (ρ(2)) autocorrelations of xt. Assume that x100 = −0.01, and x99 = 0.02. Compute the 1-step (xtt+1) and 2-step (xtt+2) ahead forecasts of the return series at the forecast origin t = 100. What are the associated standard deviations of the forecast errors? 8. Consider the file “la-regr.dat”, in the syllabus, which contains cardiovascular mortality, temperature values and particulate levels over 6-day periods from Los Angeles County (1970-1979). The file also contains two dummy variables for regression purposes, a column of ones for the constant term and a time index. The order is as follows: Column 1: 508 cardiovascular mortality values (6-day averages), Column 2: 508 ones, Column 3: the integers 1, 2, · · ·, 508, Column 3: Temperature in degrees F and Column 4: Particulate levels. A reference is Shumway et. al. (1988). The point here is to examine possible relations between the temperature and mortality in the presence of a time trend in cardiovascular mortality. (a) Use scatter diagrams to argue that particulate level may be linearly related to mortality and that temperature has either a

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

153

linear or quadratic relation. Check for lagged relations using the cross correlation function. (b) Adjust temperature for its mean value and fit the model Mt = β0 + β1(Tt − T ) + β2(Tt − T )2 + β3 Pt + et, where Mt, Tt and Pt denote the mortality, temperature and particulate pollution series. You can use as inputs Columns 2 and 3 for the trend terms and run the regression analysis without the constant option. (c) Plot the residuals and compute the autocorrelation (ACF) and partial autocorrelation (PACF) functions. Do the residuals appear to be white? Suggest an ARIMA model for the residuals and fit the residuals. The simple ARIMA(2, 0, 0) model is a good compromise. (d) Apply the ARIMA model obtained in part (c) to all of the input variables and to cardiovascular mortality using a transformation. Retain the forecast values for the transformed mortality, say mt = Mt − φ1 Mt−1 − φ2 Mt−2. 9. Generate 10 realizations of a (n = 200 points each) series from an ARIMA(1, 0, 1) Model with φ1 = 0.90, θ1 = 0.20 and σw2 = 0.25. Fit the ARIMA model to each of the series and compare the estimators to the true values by computing the average of the estimators and their standard deviations. 10. Consider the bivariate time series record containing monthly U.S. Production as measured monthly by the Federal Reserve Board Production Index and unemployment as given in the file “frb.asd”. The file contains n = 372 monthly values for each series. Before you begin, be sure to plot the series. Fit a seasonal

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

154

ARIMA model of your choice to the Federal Reserve Production Index. Develop a 12 month forecast using the model. 11. The file labelled “clim-hyd.asd” has 454 months of measured values for the climatic variables Air Temperature, Dew Point, Cloud Cover, Wind Speed, Preciptation, and Inflow at Shasta Lake. We would like to look at possible relations between the weather factors and between the weather factors and the inflow to Shasta Lake. (a) Fit the ARIMA(0, 0, 0) × (0, 1, 1)12 model to transformed √ precipitation Pt = pt and transformed flow it = log(it). Save the residuals for transformed precipitation for use in part (b). (b) Apply the ARIMA model fitted in part (a) for transformed precipitation to the flow series. Compute the cross correlation between the flow residuals using the precipitation ARIMA model and the precipitation residuals using the precipitation model and interpret. 12. Consider the daily simple return of CRSP equal-weighted index, including distributions, from January 1980 to December 1999 in the file “d-ew8099.txt” (day, ew). The indicator variables for Mondays, Tuesdays, Wednesdays, and Thursdays are in the first four columns of the file “wkdays8099.dat”. Use a regression model, possibly with time series errors, to study the effects of trading days on the index return. What is the fitted model? Are the weekday effects significant in the returns at the 5% level? Use the HAC estimator of the covariance matrix to obtain the t-ratios of regression estimates. Does it change the conclusion of weekday effect? Are there serial correlations in the residuals? Use the Ljung-Box test to perform the test. Draw your conclusion. If

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

155

yes, build a regression model with time series error to study weekday effects. 13. This problem is concerned with the dynamic relationship between the spot and futures prices of the S&P500 index. The data file “sp5may.dat” has three columns: log(futures price), log(spot price), and cost-of-carry (×100). The data were obtained from the Chicago Mercantile Exchange for the S&P 500 stock index in May 1993 and its June futures contract. The time interval is 1 minute (intraday). Several authors used the data to study index futures arbitrage. Here we focus on the first two columns. Let ft and st be the log prices of futures and spot, respectively. Build a regression model with time series errors between {ft} and {st}, with ft being the dependent variable. You need to provide all details (reasons and analysis results) at each step. 14. The quarterly gross domestic product implicit price deflator is often used as a measure of inflation. The file “q-gdpdef.dat” contains the data for U.S. from the first quarter of 1947 to the first quarter of 2004. The data are seasonally adjusted and equal to 100 for year 2000. The data are obtained from the Federal Reserve Bank of St Louis. Build a (seasonal) ARIMA model for the series and check the validity of the fitted model. Use the model to forecast the gross domestic product implicit price deflator for the rest of 2004. 15. Consider the monthly simple returns of the Decile 1, Decile 5, and Decile 10 of NYSE/AMEX/NASDAQ based on market capitalization. The data span is from January 1960 to December 2003, and the data are obtained from CRSP with the file name:

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

156

“m-decile1510.txt”. For each series, test the null hypothesis that the first 12 lags of autocorrelations are zero at the 5% level. Draw your conclusion. Build an AR and MA model for the series Decile 5. Use the AR and MA model built to produce 1-step and 3-step ahead forecasts of the series. Compare the fitted AR and MA models. 16. The data file “q-unemrate.txt” contains the U.S. quarterly unemployment rate, seasonally adjusted, from 1948 to the second quarter of 1991. Consider the change series ∆ xt = xt − xt−1, where xt is the quarterly unemployment rate. Build an AR model for the ∆ xt series. Does the fitted model suggest the existence of business cycles? 17. In this exercise, please construct the impulse response function for a a third order difference equation: yt = φ1 yt−1 + φ2 yt−2 + φ3 yt−3 + wt for 1 ≤ t ≤ n, where it is assumed that {wt}nt=1 is a sequence of deterministic numbers, say generated from N (0, 1). (a) Set φ1 = 1.1, φ2 = −0.8, φ3 = 0.1, y0 = y−1 = y−2 = 0, and n = 150. Generate yt using a third order difference equation for 1 ≤ t ≤ n. (b) Check eigenvalues of this model to determine whether IRF are converging or explosive. (c) Construct the impulse response function for the generated yt. Set the number of periods in the impulse response function to J = 25. Comment your results. (d) Set φ1 = 1.71 and repeat steps (a) - (c). Comment your results.

CHAPTER 3. UNIVARIATE TIME SERIES MODELS

3.15

157

Computer Code

The following R commands are used for making the graphs in this chapter. # 3-28-2006 graphics.off()

############################################################ ibm T B ) model (4.5) becomes: B

xt = [µ + θ] + [β t + γ(t − T )] + α xt−1 +

k $

i=1

ci ∆ xt−i + et.

Model (4.5) is estimated by OLS with the break points ranging over the sample and the t-statistic for testing α = 1 is computed. The minimum t-statistic is reported. Critical values for 1%, 5% and 10% critical values are −5.34, −4.8 and −4.58, respectively. The appropriate number of lags in differences is estimated for each value of τ . Please read the paper by Sadorsky (1999) for more details about this method and empirical applications.

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

4.6.3

205

Cautions

The appropriate way to adjust for a break in the population parameters depends on the sources of the break. If a distinct break occurs at a specific date, this break will be detected with high probability by the QLR statistic, and the break date can be estimated. The regression model can be estimated using a dummy variable. If there is a distinct break, then inference on the regression coefficients can proceed as usual using t-statistics for hypothesis testing. Forecasts can be produced using the estimated regression model that applies to the end of the sample. The problem is more difficult if the break is not distinct and the parameters slowly evolve over time. In this case a state-space modelling is required. 4.7

Problems

1. You will build a time-series model for real oil price listed in the tenth column in file “MacroData.xls”. (a) Construct graphs of time series data: time plots and scatter plots of the levels of real oil prices (OP) and the log-difference of oil prices. The log-difference of oil price is defined as follows ∆ log(OPt) = log(OPt) − log(OPt−1). Comment your results. (b) Try to identify lag structure for levels of real oil prices and the log-difference of oil prices by using ACF and PACF. Comment your results. (c) We will not estimate ARMA models yet. So, estimate AR(p) model, 1 ≤ p ≤ 8. Compute Akaike Information Criteria (AIC) or AICC and Schwarz Information Criteria (SIC). Present your results. Choose the AR lag length based on the AIC or AICC.

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

206

Which lag length would you choose based on the AIC or AICC or SIC? Comment. (d) Estimate the AR(p) model with the optimal lag length. Present your results nicely. (e) Conduct the diagnostic checking of the estimated AR model. You need do the followings and comment your results: (i) Construct graphs of residuals (time series plots, scatterplots, squared residuals). (ii) Check for serial correlation using sample ACF. (iii) Check for serial correlation using Ljung-Box test statistic. (iv) Conduct Jarque-Bera test of normality and make a Q-Q plot. (v) Estimate the following AR(1) model for the estimated squared residuals and test the null hypothesis that the slope is insignificant. How would interpret your results? What does it say about the constancy of variance? (vi) Based on diagnostic checking in the above, can you use the model or you should go back to identification of lag structure in (b)? (f) Is there any structure change for oil price? Note: The Jarque-Bera (1980, 1987) test evaluates the hypothesis that X has a normal distribution with unspecified mean and variance, against the alternative that X does not have a normal distribution. The test is based on the sample skewness and kurtosis of X. For a true normal distribution, the sample skewness should be near 0 and the sample kurtosis should be near 3. A test has the following general form: 

2



T  (K − 3)  2 Sk +  → χ , 2 6 4 where Sk and K are the measures of skewness and kurtosis respectively. To use the build-in function for the Jarque-Bera test JB =

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

207

in the package tseries in R, the command for the Jarque-Bera test is library(tseries) jb=jarque.bera.test(x) print(jb)

# call the package "tseries" # x is the series for the test # print the testing result

2. Based on the economic theory, the description of inflation suggests two fundamental causes, excess monetary growth (faster than real output) and the dissipation of external shocks. The precise mechanisms at work, appropriate lag structures are not perfectly defined. In this exercise, you will estimate the following simple model: ∆ log(Pt) = β1+β2[∆ log(M 1t−1)−∆ log(Qt−1)]+β3 ∆ log(Pt−1)+ut, where Pt is the quarterly price level (CPI) listed in the eighth column in “MacroData.xls”, Qt is the quarterly real output (listed in the third column), and M 1t is the quarterly money stock (listed in the thirteenth column). (a) Nicely present the results of OLS estimation. Comment your results. (b) Explain what may be an advantage of the above model compared to a simple autoregressive model of prices. To answer this question, you might need to do some statistical analysis. (c) Is there any structure change for CPI? (d) Any suggestions to build a better model? 3. In this exercise, you will build a time series model for the real GDP process listed in the second column in “MacroData.xls”.

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

208

(a) Build informatively and formally an AR(p) model with the optimal lag length based on some criterion and conduct the diagnostic checking of the estimated AR model. (b) Based on the data 1959.1-1999.3 construct forecasts for the quarters 1999.4 -2002.3. Plot the constructed forecasts and the realized values. Comment your results. 4. You need to replicate some of the steps in the analysis of oil price shocks and stock market activity conducted by Sadorsky (1999). You should write your report in such a way that an outside reader may understand what the report is about and what you are doing. Write a referee report for the paper of Sadorsky (1999). One possible structure of the referee report is: (a) Summary of the paper (This assures that you really read the paper carefully): (i) Is the economic/financial question of relevance? (ii) What have you learned from reading this paper? (iii) What contribution does this paper make to the literature? (b) Can you think of interesting extensions for the paper? (c) Expository quality of the paper: (i) Is the paper well structured? If not, suggest and alternative structure. (ii) Is the paper easy to read? 5. Analyze the stochastic properties of the following interest rates: (i) federal funds rate, (ii) 90-day T-bill rate, (iii) 1-year T-bond interest rate, (iv) 5-year T-bond interest rate; (v) 10-year T-bond interest rate. The interest rates may be found in the Excel file “IntRates.xls”. (a) Use the ADF or PP approach to test the null hypothesis that five interest rate are difference stationary against the alternative

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

209

that they are stationary. Explain carefully how you conduct the test. (b) Use the ADF or PP approach to test the null hypothesis that five interest rate are difference stationary against the alternative that they are stationary around a deterministic trend. Explain carefully how you conduct the test. (c) Use the QLR testing procedure to test whether there was at least one structural break in interest rates series. 4.8

Computer Code

The following R commands are used for making the graphs in this chapter. # 5-20-2006 graphics.off()

############################################################ y=read.csv("c:\\teaching\\time series\\data\\MacroData.csv", cpi=y[,8] qt=y[,3] m0=y[,12] m1=y[,13] m2=y[,14] m3=y[,15] op=y[,10] v0=cpi*qt/m0 v1=cpi*qt/m1 v2=cpi*qt/m2

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

v3=cpi*qt/m3 vt=cbind(v0,v1,v2,v3) win.graph() par(mfrow=c(2,2),mex=0.4,bg="light blue") ts.plot(cpi,type="l",lty=1,ylab="",xlab="") title(main="CPI",col.main="red") ts.plot(qt,type="l",lty=1,ylab="",xlab="") title(main="Industry Output",col.main="red") ts.plot(qt,type="l",lty=1,ylab="",xlab="") title(main="Oil Price",col.main="red") win.graph() par(mfrow=c(2,2),mex=0.4,bg="light grey") ts.plot(m0,type="l",lty=1,ylab="",xlab="") title(main="Money Aggregate",col.main="red") ts.plot(m1,type="l",lty=1,ylab="",xlab="") title(main="Money Aggregate",col.main="red") ts.plot(m2,type="l",lty=1,ylab="",xlab="") title(main="Money Aggregate",col.main="red") ts.plot(m3,type="l",lty=1,ylab="",xlab="") title(main="Money Aggregate",col.main="red") win.graph() par(mfrow=c(2,2),mex=0.4,bg="yellow") ts.plot(v0,type="l",lty=1,ylab="",xlab="") title(main="Velocity",col.main="red") ts.plot(v1,type="l",lty=1,ylab="",xlab="") title(main="Velocity",col.main="red") ts.plot(v2,type="l",lty=1,ylab="",xlab="")

210

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

211

title(main="Velocity",col.main="red") ts.plot(v3,type="l",lty=1,ylab="",xlab="") title(main="Velocity",col.main="red") library(tseries) library(urca) library(quadprog) library(zoo)

# call library(tseries) # call library(urca)

adf_test=adf.test(cpi) # Augmented Dickey-Fuller test print(adf_test) adf_test=pp.test(cpi) # do Phillips-Perron test print(adf_test) #adf_test2=ur.df(y=cpi,lag=5,type=c("drift")) #print(adf_test2) adf_test=adf.test(op) print(adf_test) adf_test=pp.test(op) print(adf_test)

# Augmented Dickey-Fuller test # do Phillips-Perron test

for(i in 1:4){ adf_test=pp.test(vt[,i]) print(adf_test) adf_test=adf.test(vt[,i]) print(adf_test) }

############################################################

CHAPTER 4. NON-STATIONARY PROCESSES AND STRUCTURAL BREAKS

212

y=read.csv("c:\\teaching\\time series\\data\\MacroData.csv", op=y[,10] library(strucchange) win.graph() par(mfrow=c(2,2),mex=0.4,bg="green") op=ts(op) fs.op 0, minimizing the AMSE gives the "

hopt



1/3

θ f (ξp)    = µ22(K)[f -(ξp)]2

n−1/3,

and with this asymptotically optimal bandwidth, the optimal AMSE is given by 3 n AMSEopt(ξ"p) = σF2 (ξp)/f 2(ξp) − 4



2/3

θ2     µ2(K)f - (ξp)f (ξp )

n−1/3,

which indicates a reduction to the AMSE of the second order. Chen and Tang (2005) conducted an intensive study on simulations to demonstrate the advantages of nonparametric estimation ξ"p over the sample quantile ξpn under the VaR setting. We refer to the paper by Chen and Tang (2005) for simulation results and empirical examples. Exercise: Please use the above procedures to estimate nonparametrically the ES and discuss its properties as well as conduct simulation studies and empirical applications. 7.5

Computer Code

# July 20, 2006 graphics.off() # clean the previous garphs on the secreen

CHAPTER 7. NONPARAMETRIC DENSITY, DISTRIBUTION & QUANTILE ESTIMATION262

######################################################### # Define the Epanechnikov kernel function kernel mQ. The basic idea is first to use Q subseries of lengths n − qm (q = 1, , · · · , Q) to estimate the unknown coefficient functions and then compute the one-step forecasting errors of the next section of the time series of length m based on the estimated models. More precisely, we choose h that minimizes the average mean squared (AMS) error AMS(h) = where for q = 1, · · · , Q,

Q $

q=1



AMSq (h),

(8.15)

2

  p   1 n−qm+m $ $ " AMSq (h) = Y − a (U )X , i j,q i i,j    m i=n−qm+1  j=1

and {a" j,q (·)} are computed from the sample {(Ui, Xi, Yi), 1 ≤ i ≤ n − qm} with bandwidth equal h[n/(n − qm)]1/5. Note that we rescale bandwidth h for different sample sizes according to its optimal rate, i.e. h ∝ n−1/5. In practical implementations, we may use m = [0.1n] and Q = 4. The selected bandwidth does not depend critically on the choice of m and Q, as long as mQ is reasonably large so that the evaluation of prediction errors is stable. A weighted version of AMS(h) can be used, if one wishes to down-weight the prediction errors at an earlier time. We believe that this bandwidth should be good for modeling and forecasting for time series.

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

8.6.4

296

Smoothing Variable Selection

Of importance is to choose an appropriate smoothing variable U in applying functional-coefficient regression models if U is a lagged variable. Knowledge on physical background of the data may be very helpful, as Cai, Fan and Yao (2000) discussed in modeling the lynx data. Without any prior information, it is pertinent to choose U in terms of some data-driven methods such as the Akaike information criterion (AIC) and its variants, cross-validation, and other criteria. Ideally, we would choose U as a linear function of given explanatory variables according to some optimal criterion, which can be fully explored in the work by Fan, Yao and Cai (2003). Nevertheless, we propose here a simple and practical approach: let U be one of the given explanatory variables such that AMS defined in (8.15) obtains its minimum value. Obviously, this idea can be also extended to select p (number of lags) as well. 8.6.5

Goodness-of-Fit Test

To test whether model (8.10) holds with a specified parametric form which is popular in economic and financial applications, such as the threshold autoregressive (TAR) models  

aj (u) = 

aj1, if u ≤ η aj2, if u > η,

or generalized exponential autoregressive (EXPAR) models aj (u) = αj + (βj + γj u) exp(−θj u2), or smooth transition autoregressive (STAR) models aj (u) = [1 − exp(−θj u)]−1 (logistic), or

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

297

aj (u) = 1 − exp(−θj u2) (exponential), or aj (u) = [1 − exp(−θj |u|)]−1 (absolute), [for more discussions on those models, please see the survey papers by van Dijk, Ter¨asvirta and Franses (2002)], we propose a goodness-offit test based on the comparison of the residual sum of squares (RSS) from both parametric and nonparametric fittings. This method is closely related to the sieve likelihood method proposed by Fan, Zhang and Zhang (2001). Those authors demonstrated the optimality of this kind of procedures for independent samples. Consider the null hypothesis H0 : aj (u) = αj (u, θ),

1 ≤ j ≤ p,

(8.16)

where αj (·, θ) is a given family of functions indexed by unknown # parameter vector θ. Let θ be an estimator of θ. The RSS under the null hypothesis is RSS0 = n

n 5 −1 $ i=1

#

#

62

Yi − α1(Ui, θ)Xi1 − · · · − αp(Ui, θ)Xip .

Analogously, the RSS corresponding to model (8.10) is RSS1 = n

n −1 $

i=1

{Yi − a" 1(Ui)Xi1 − · · · − a" p(Ui)Xip}2 .

The test statistic is defined as

Tn = (RSS0 − RSS1)/RSS1 = RSS0/RSS1 − 1, and we reject the null hypothesis (8.16) for large value of Tn. We use the following nonparametric bootstrap approach to evaluate the p value of the test:

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

298

1. Generate the bootstrap residuals {ε∗i }ni=1 from the empirical dis" n , where tribution of the centered residuals {ε"i − ¯ε} i=1 εi = Yi − a1(Ui) Xi1 − · · · − ap(Ui) Xip, "

and define

"

"

¯ε" =

n 1 $ ε"i, n i=1

# # ∗ Yi∗ = α1(Ui, θ)X i1 + · · · + αp (Ui , θ)Xip + εi .

2. Calculate the bootstrap test statistic Tn∗ based on the sample {Ui, Xi, Yi∗}ni=1. 3. Reject the null hypothesis H0 when Tn is greater than the upperα point of the conditional distribution of Tn∗ given {Ui, Xi, Yi}ni=1. The p-value of the test is simply the relative frequency of the event {Tn∗ ≥ Tn} in the replications of the bootstrap sampling. For the sake of simplicity, we use the same bandwidth in calculating Tn∗ as that in Tn. Note that we bootstrap the centralized residuals from the nonparametric fit instead of the parametric fit, because the nonparametric estimate of residuals is always consistent, no matter whether the null or the alternative hypothesis is correct. The method should provide a consistent estimator of the null distribution even when the null hypothesis does not hold. Kreiss, Neumann, and Yao (1998) considered nonparametric bootstrap tests in a general nonparametric regression setting. They proved that, asymptotically, the conditional distribution of the bootstrap test statistic is indeed the distribution of the test statistic under the null hypothesis. It may be proven that # the similar result holds here as long as θ converges to θ at the rate n−1/2. It is a great challenge to derive the asymptotic property of the testing statistics Tn under time series context and general assumptions.

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

299

That is to show that bn [Tn − λn] → N (0, σ 2) for some bn and λn, which is a great project for future research. Note that Fan, Zhang and Zhang (2001) derived the above result for the iid sample. 8.6.6

Asymptotic Results

We first present a result on mean squared convergence that serves as a building block for our main result and is also of independent interest. We now introduce some notation. Let   S S n,0 n,1  Sn = Sn(u0) =  Sn,1 Sn,2 and





T (u ) Tn = Tn(u0) =  n,0 0  Tn,1(u0)

with Sn,j

1 = Sn,j (u0) = n

and 1 Tn,j (u0) = n





j T  Ui − u0  Kh(Ui − u0) Xi Xi h i=1 n $ 



Ui − u0 j  Xi Kh(Ui − u0) Yi. h i=1 n $

(8.17)

Then, the solution to (8.12) can be expressed as # β = H−1 S−1 n Tn ,

(8.18)

where H = diag (1, . . . , 1, h, . . . , h) with p-diagonal elements 1’s and p diagonal elements h’s. To facilitate the notation, we denote +

T

,

Ω = (ωl,m )p×p = E X X | U = u0 .

(8.19)

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

300

Also, let f (u, x) denote the joint density of (U, X) and fu(u) be the marginal density of U . We use the following convention: if U = Xj0 for some 1 ≤ j0 ≤ p, then f (u, x) becomes f (x) the joint density of X. Theorem 1. Let condition A.1 in hold, and let f (u, x) be continuous at the point u0. Let hn → 0 and n hn → ∞, as n → ∞. Then it holds that E(Sn,j (u0)) → fu(u0) Ω(u0) µj , and n hn Var(Sn,j (u0)l,m ) → fu(u0) ν2j ωl,m

for each 0 ≤ j ≤ 3 and 1 ≤ l, m ≤ p.

As a consequence of Theorem 1, we have Sn

P −→

fu(u0) S,

Sn,3

and

P −→

µ3 fu(u0) Ω

in the sense that each element converges in probability, where   Ω µ Ω 1 . S= µ1 Ω µ2 Ω Put σ 2(u, x) = Var(Y | U = u, X = x) (8.20)

and

∗

7

T

2

8

Ω (u0) = E X X σ (U, X) | U = u0 .

J

K

J

K

Let c0 = µ2/ µ2 − µ21 and c1 = −µ1/ µ2 − µ21 .

(8.21)

Theorem 2. Let σ 2(u, x) and f (u, x) be continuous at the point u0. Then under conditions A.1 and A.2,   2 2 + , √ h µ − µ µ 1 3 -D 2 "  2 n hn a(u0) − a(u0) − a (u0) −→ N 0, Θ (u0) , 2 µ2 − µ21 (8.22)

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

301

provided that fu(u0) )= 0, where

c20 ν0 + 2 c0 c1 ν1 + c21 ν2 −1 Θ (u0) = Ω (u0) Ω∗(u0) Ω−1(u0). fu(u0) (8.23) 2

Theorem 2 indicates that the asymptotic bias of a" j (u0) is h2 µ22 − µ1 µ3 -a (u0) 2 µ2 − µ21 j

and the asymptotic variance is (n hn)−1 θj2(u0), where θj2(u0)

c20 ν0 + 2 c0 c1 ν1 + c21 ν2 T −1 = ej,p Ω (u0) Ω∗(u0) Ω−1(u0) ej,p . fu(u0)

When µ1 = 0, the bias and variance expressions can be simplified as h2 µ2 a--j (u0)/2 and ν0 θj2(u0) = eTj,p Ω−1(u0) Ω∗(u0) Ω−1(u0) ej,p . fu(u0) The optimal bandwidth for estimating aj (·) can be defined to be the one that minimizes the squared bias plus variance. The optimal bandwidth is given by 

−1

∗

−1

1/5

µ22 ν0 − 2 µ1 µ2 ν1 + µ21 ν2 eTj,p Ω (u0) Ω (u0) Ω (u0) ej,p   hj,opt = 5 62 2  2 -fu(u0) (µ2 − µ1 µ3) aj (u0) (8.24)    

8.6.7

Conditions and Proofs

We first impose some conditions on the regression model but they might not be the weakest possible.

n−1/

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

302

Condition A.1

a. The kernel function K(·) is a bounded density with a bounded support [−1, 1]. b. |f (u, v | x0, x1; l)| ≤ M < ∞, for all l ≥ 1, where f (u, v, | x0, x1; l) is the conditional density of (U0, Ul )) given (X0, Xl ), and f (u | x) ≤ M < ∞, where f (u | x) is the conditional density of U given X = x. c. The process {Ui, Xi, Yi} is α-mixing with for some δ > 2 and c > 1 − 2/δ.

%

k c[α(k)]1−2/δ < ∞

d. E|X|2 δ < ∞, where δ is given in condition A.1c. Condition A.2

a. Assume that E

5

Y02

6

2

+ Yl | U0 = u, X0 = x0; Ul = v, Xl = x1 ≤ M < ∞, (8.25) for all l ≥ 1, x0, x1 ∈ 4p, u, and v in a neighborhood of u0. b. Assume that hn → and n hn → ∞. Further, assume that there exists a+ sequence, of positive integers sn such that sn → ∞, sn = o (n hn)1/2 , and (n/hn)1/2 α(sn) → 0, as n → ∞. c. There exists δ ∗ > δ, where δ is given in Condition A.1c, such that 5 6 δ∗ E |Y | |U = u, X = x ≤ M4 < ∞ (8.26) for all x ∈ 4p and u in a neighborhood of u0, and +

α(n) = O n where θ∗ ≥ δ δ ∗/{2(δ ∗ − δ)}.

−θ∗

,

,

(8.27)

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION ∗

d. E|X|2 δ < ∞, and n1/2−δ/4 hδ/δ

∗ −1/2−δ/4

303

= O(1).

Remark A.1. We provide a sufficient condition for the mixing coefficient α(n) to satisfy conditions A.1c and A.2b. Suppose that hn = A n−ρ (0 ,< ρ < 1, A > 0), sn = (n hn/ log n)1/2 and + α(n) = O n−d for some d > 0. Then condition A.1c is satisfied for d > 2(1 − 1/δ)/(1 − 2/δ) and condition A.2b is satisfied if d > (1 + ρ)/(1 − ρ). Hence both conditions are satisfied if    + , 1 + ρ  2(1 − 1/δ)  −d α(n) = O n , d > max  , . 1 − ρ  1 − 2/δ  Note that this is a trade-off between the order δ of the moment of Y and the rate of decay of the mixing coefficient; the larger the order δ, the weaker the decay rate of α(n). " To study the joint asymptotic normality of a(u 0 ), we need to center the vector Tn(u0) by replacing Yi with Yi − m(Ui, Xi) in the expression (8.17) of Tn,j (u0). Let

T∗n,j (u0)

1 = n

and





j U − u i 0  K (U − u ) [Y − m(U , X )], Xi  h i 0 i i i h i=1 n $





∗ T ∗ Tn =  ∗n,0  . Tn,1 Because the coefficient functions aj (u) are conducted in the neighborhood of |Ui − u0| < h, by Taylor’s expansion, 



h2  Ui − u0 2 T -T T m(Ui, Xi) = Xi a(u0)+(Ui−u0) Xi a (u0)+ Xi a (u0)+op(h2 2 h where a-(u0) and a--(u0) are the vectors consisting of the first and second derivatives of the functions aj (·). Then, Tn,0 −

T∗n,0

h2 = Sn,0 a(u0) + h Sn,1 a (u0) + Sn,2 a--(u0) + op(h2) 2 -

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

304

and Tn,1 −

T∗n,1

h2 = Sn,1 a(u0) + h Sn,2 a (u0) + Sn,3 a--(u0) + op(h2), 2 -

so that Tn −

T∗n

h2 = Sn H β + 2





S  n,2  a-- (u ) + o (h2 ), 0 p Sn,3

(8.28)

where β = (a(u0)T , a-(u0)T )T . Thus it follows from (8.18), (8.28), and Theorem 1 that   + , h2 −1  µ2 Ω  -# −1 ∗ −1 a (u0) + op(h2), H β − β = fu (u0) S Tn + S µ3 Ω 2 (8.29) # from which the bias term of β(u0) is evident. Clearly,

−1 7 8 h2 µ2 − µ µ Ω 1 3 -2 ∗ ∗ " a(u0)−a(u0) = µ T − µ T + a (u0)+o 2 1 n,0 n,1 fu(u0) (µ2 − µ21) 2 µ2 − µ21 (8.30) " Thus (8.30) indicates that the asymptotic bias of a(u 0 ) is

h2 µ22 − µ1 µ3 -a (u0). 2 µ2 − µ21

Let

1 Qn = n

where





n $

i=1

Zi,

(8.31)



Ui − u0  Zi = Xi c0 + c1  Kh(Ui − u0) [Yi − m(Ui, Xi)] (8.32) h K K J J with c0 = µ2/ µ2 − µ21 and c1 = −µ1/ µ2 − µ21 . It follows from (8.30) and (8.31) that √





Ω−1 √ h2 µ22 − µ1 µ3 -"  n hn a(u0) − a(u0) − n hn Qn+op(1). a (u0) = 2 µ2 − µ21 fu(u0) (8.33)

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

305

We need the following lemma, whose proof is more involved than that for Theorem 1. Therefore, we prove only this lemma. Throughout this Appendix, we let C denote a generic constant, which may take different values at different places. Lemma A.1. Under conditions A.1 and A.2 and the assumption that hn → 0 and n hn → ∞, as n → ∞, if σ 2(u, x) and f (u, x) are continuous at the point u0, then we have D

E

(a) hn Var(Z1) → fu(u0) Ω∗(u0) c20 ν0 + 2 c0 c1 ν1 + c21 ν2 ; (b) hn

%n−1|

l=1

|Cov(Z1, Zl+1)| = o(1); and

E

D

(c) n hn Var(Qn) → fu(u0) Ω∗(u0) c20 ν0 + 2 c0 c1 ν1 + c21 ν2 . Proof: First, by conditioning on (U1, X1) and using Theorem 1 of Sun (1984), we have 

 

Var(Z1) = E X1 XT1 σ 2(U1, X1) c0 + c1

 

U1 − u h

2 0  



Kh2(U1 − u0)

5 6 8 17 ∗ 2 2 = fu(u0) Ω (u0) c0 ν0 + 2 c0 c1 ν1 + c1 ν2 + o(1) . (8.34) h

The result (c) follows in an obvious manner from (a) and (b) along with   1 l 2 n−1 $ 1 −  Cov(Z , Z Var(Qn) = Var(Z1) + (8.35) 1 l+1 ). n n l=1 n It thus remains to prove part (b). To this end, let dn → ∞ be a sequence of positive integers such that dn hn → 0. Define J1 =

dn$ −1 l=1

|Cov(Z1, Zl+1)|

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

and J2 =

n−1 $

l=dn

306

|Cov(Z1, Zl+1)|. J

K

J

K

It remains to show that J1 = o h−1 and J2 = o h−1 . We remark that because K(·) has a bounded support [−1, 1], aj (u) is bounded in the neighborhood of u ∈ [u0 − h, u0 + h]. % Let B = max1≤j≤p sup|u−u0| 0 be bandwidths in the step of estimating the regression surface. Here, to handle various degrees of smoothness, Cai and Fan (2000) propose using h11 and h12 differently although the implementation may not be easy in practice. The reader is referred to the paper by Cai and # Fan (2000) for details. Given observations {Xt, Yt, Zt}nt=1, let β j be the minimizer of the following locally weighted least squares n 5 $

t=1

Zt − β0 −

β T1

(Xt − x) −

β T2

62

(Yt − y)

Kh11 (Xt−x) Lh12 (Yt−y),

where Kh(·) = K(·/h)/hp and Lh(·) = L(·/h)/hq . Then, the local P linear estimator of the regression surface m(x, y) is m(x, y) = β#0.

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

318

P By computing the sample average of m(·, ·) based on (8.62), the projection estimators of g1(·) and g2(·) are defined as, respectively,

and

g"1(x) =

n 1 $ P # m(x, Yt) − µ, n t=1

n 1 $ P # g2(y) = m(X t , y) − µ, n t=1 % n # = n−1 where µ t=1 Zt . Under some regularity conditions, by using the same arguments as those employed in the proof of Theorem 3 in Cai and Masry (2000), it can be shown (although not easy and tedious) that the asymptotic bias and asymptotic variance of g"1(x) are, respectively, h211 tr{µ2(K) g1--(x)}/2 and v1(x) = ν0(K) A(x), where L A(x) = p22(y) σ 2(x, y) p−1(x, y) dy "

and σ 2(x, y) = Var ( Zt | Xt = x, Yt = y) .

Here, p(x, y) stands for the joint density of Xt and Yt, p1(x) denotes the marginal density of Xt, p2(y) is the marginal density of Yt, M M ν0(K) = K 2(u)du, and µ2(K) = uuT K(u) du. The foregoing method has some advantages, such as it is easy to understand, it can make computation fast, and it allows an asymptotic analysis. However, it can be quite inefficient in an asymptotic sense. To demonstrate this idea, let us consider the ideal situation that g2(·) and µ are known. In such a case, one can estimate g1(·) by H directly regressing the partial error Z t = Zt − µ − g2 (Yt ) on Xt and such an ideal estimator is optimal in an asymptotic minimax sense (see, e.g., Fan and Gijbels, 1996). The asymptotic bias for the ideal

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

319

estimator is h211 tr{ µ2(K) g1--(x)}/2 and the asymptotic variance is v0(x) = ν0(K) B(x) with B(x) =

p−1 1 (x)

5

2

6

E σ (Xt, Yt) | Xt = x (8.64) (see, e.g., Masry and Fan, 1997). It is clear that v1(x) = v0(x) if Xt and Yt are independent. If Xt and Yt are correlated and when σ 2(x, y) is a constant, it follows from the Cauchy-Schwarz inequality that

σ 2 L 1/2 p2(y) σ 2 L p22(y) B(x) = p (y |x) 1/2 dy ≤ d y = A(x), p1(x) p (y |x) p1(x) p(y|x) which implies that the ideal estimator has always smaller asymptotic variance than the projection method although both have the same bias. This suggests that the projection method could lead to an inefficient estimation of g1(·) and g2(·) when Xt and Yt are serially correlated, which is particularly relevant for autoregressive models. To alleviate this shortcoming, I propose the two-stage approach described next. 8.7.4

Two-Stage Procedure

The two-stage method due to Linton (1997, 2000) is introduced. The basic idea is to get an initial estimate for g"2(·) using a small bandwidth h12. The initial estimate can be obtained by the projection method and h12 can be chosen so small that the bias of estimating g"2(·) can be asymptotically negligible. Then, using the partial #−g " (Y ), we apply the local linear regression residuals Zt∗ = Zt − µ 2 t technique to the pseudo regression model Zt∗ = g1(Xt) + ε∗t

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

320

to estimate g1(·). This leads naturally to the weighted least-squares problem n 5 $

t=1

Zt∗

− β1 −

β T2

62

(Xt − x)

Jh2 (Xt − x),

(8.65)

where J(·) is the kernel function in 4p and h2 = h2(n) > 0 is the bandwidth at the second-stage. The advantage of this is twofold: the bandwidth h2 can now be selected purposely for estimating g1(·) only and any bandwidth selection technique for nonparametric regression can be applied here. Maximizing (8.65) with respect to β1 and β 2 gives the two-stage estimate of g1(x), denoted by g!1(x) = β#1, where # β#1 and β 2 are the minimizer of (8.65).

It is shown in Theorem 1, in which follows, that under some regularity conditions, the asymptotic bias and variance of the two-stage estimate g!1(x) are the same as those for the ideal estimator, provided that the initial bandwidth h12 satisfies h12 = o (h2).

Sampling Properties

To establish the asymptotic normality of the two-stage estimator, it is assumed that the initial estimator satisfies a linear approximation; namely, n 1 $ " g2(Yt) − g2(Yt) ≈ Lh (Yi − Yt)Γ(Xi, Yt) δi n i=1 12 1 (8.66) + h212 tr{µ2(L) g2--(Yt)}, 2 where δt = Zt −m(Xt, Yt) and Γ(x, y) = p1(x)/p(x, y). Note that under some regularity conditions, by following the same arguments as in Masry (1996), one might show (although the proof is not easy, quite lengthy, and tedious) that (8.66) holds. Note that this assumption is also imposed in Linton (2000) for iid samples to simplify the

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

321

proof of the asymptotic results of the two-stage estimator. Now, the asymptotic normality for the two-stage estimator is stated here and its proof is relegated to the Appendix. THEOREM 1. Under (8.66) and Assumptions A1 – A9 stated in the Appendix, if bandwidths h12 and h2 are chosen such that h12 → 0, n hq12 → ∞, h2 → 0, and n hp2 → ∞ as n → ∞, then,

&

n hp2

7

g1(x) − g1(x) − bias(x) + !

+

op h212

+

h22

,8

D −→

N {0, v0(x)} ,

where the asymptotic bias is h212 h22 -tr{µ2(J) g1 (x)} − tr {µ2(L) E (g2--(Yt) | Xt = x)} bias(x) = 2 2 and the asymptotic variance is v0(x) = ν0(J) B(x). We remark that by Theorem 1, the asymptotic variance of the two-stage estimator is independent of the initial bandwidths. Thus, the initial bandwidths should be chosen as small as possible. This is another benefit of using the two-stage procedure: the bandwidth selection problem becomes relatively easy. In particular, when h12 = o (h2), the bias from the initial estimation can be asymptotically negligible. For the ideal situation that g2(·) is known, Masry and Fan (1997) show that under some regularity conditions, the optimal estimate of g1(x), denoted by g"1∗(x), by using (8.65) in which the H partial residual Zt∗ is replaced by the partial error Z t = Yt − µ − g2(Yt), is asymptotically normally distributed, &



n hp2 g"1∗(x) − g1(x) −

h22 2



tr{µ2(J) g1--(x)} + op(h22)

D −→

N {0, v0(x)} .

This, in conjunction with Theorem 1, shows that the two-stage estimator and the ideal estimator share the same asymptotic bias and variance if h12 = o (h2).

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

322

Finally, it is worth to pointing out that under some regularity conditions, the nonlinear additive ARX processes are stationary and α-mixing with geometric decay mixing coefficient, see Masry and Tjøstheim (1997), so that Assumptions A6, A7, and A8 in the Appendix imposed on the mixing coefficient are automatically satisfied. Therefore, assuming that the other technical assumptions of this paper are satisfied, the result in Theorem 1 can be applied to the nonlinear additive ARX models. 8.7.5

Monte Carlo Simulations and Applications

See Cai (2002) for the detailed Monte Carlo simulation results and applications. 8.8

Computer Code

# 07-31-2006 graphics.off() # clean the previous graphs on the screen ############################################################ z1=matrix(scan(file="c:\\teaching\\time series\\data\\w-3mtb byrow=T,ncol=4) # dada: weekly 3-month Treasury bill from 1970 to 1997 x=z1[,4]/100 n=length(x) y=diff(x) # Delta x_t=x_t-x_{t-1} x=x[1:(n-1)] n=n-1 x_star=(x-mean(x))/sqrt(var(x)) z=seq(min(x),max(x),length=50) #win.graph()

CHAPTER 8. NONPARAMETRIC REGRESSION ESTIMATION

323

postscript(file="c:\\teaching\\time series\\figs\\fig-8.1.ep horizontal=F,width=6,height=6) par(mfrow=c(2,2),mex=0.4,bg="light blue") scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)",evaluatio title(main="(a) y(t) vs x(t-1)",col.main="red") scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)",eval title(main="(b) |y(t)| vs x(t-1)",col.main="red") scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)",evaluat title(main="(c) y(t)^2 vs x(t-1)",col.main="red") dev.off() ############################################################ ######################### # Nonparametric Fitting # ######################### ######################################################### # Define the Epanechnikov kernel function kernel