This is page i
Printer: Opaque this
Jianqing Fan
Qiwei Yao
Nonlinear Time Series:
Nonparametric and Parametric Meth-
ods
This is page ii
Printer: Opaque this
TO THOSE
WHO EDUCATE US;
WHOM WE LOVE; AND
WITH WHOM WE COLLABORATE
Back-cover
iii
This book is on the contemporary statistical methods and theory of
nonlinear time series analysis. The principal focus is on nonparametric
and semiparametric techniques developed in the last decade. It covers the
techniques for modelling in state-space, in frequency-domain as well as in
time-domain. To reflect the integration of parametric and nonparametric
methods in analyzing time series data, the book also presents an up-to-date
exposure on some parametric nonlinear models, including ARCH/GARCH
models and threshold models. A compact view on linear ARMA models
is also provided. Data arising in real applications are used throughout to
show how nonparametric approaches may help to reveal local structure in
high-dimensional data. Important technical tools are also introduced.
The book is designed to appeal to graduate students, application-oriented
time series analysts, new and experienced researchers. It will have the value
both within the statistical community and across a broad spectrum of other
fields such as econometrics, empirical finance, population biology and ecol-
ogy. The prerequisites are merely sound basic courses in probability and
statistics.
Jianqing Fan, coauthor of the highly regarded book Local Polynomial
Modeling, is Professor of Statistics at the University of North Carolina at
Chapel Hill and the Chinese University of Hong Kong. His published work
on nonparametric modeling, nonlinear time series, financial econometrics,
analysis of longitudinal data, model selection, wavelets and other aspects
of methodological and theoretical statistics has been recognized with the
Presidents’ Award from the Committee of Presidents of Statistical Soci-
eties, the Hettleman Prize for Artistic and Scholarly Achievement from the
University of North Carolina, and by his election as a fellow of the Ameri-
can Statistical Association and the Institute of Mathematical Statistics.
Qiwei Yao is Professor of Statistics at the London School of Economics
and Political Science. He is an elected member of the International Sta-
tistical Institute, has served on editorial board for Journal of the Royal
Statistical Society (Series B) and Australian and New Zealand Journal of
Statistics.
Preface
This is page v
Printer: Opaque this
Among many exciting developments in statistics over the last two decades,
Nonlinear Time Series and Data-analytic Nonparametric Methods have
greatly advanced along seemingly unrelated paths. In spite of the fact that
the application of nonparametric techniques in time series can be backdated
to as early as 1940’s, there still exists healthy and justified skepticism about
the capability of nonparametric methods in analyzing time series data. As
enthusiastic explorers of the modern nonparametric tool-kit, we feel obliged
to assemble together the newly developed relevant techniques in one place.
The aim of this book is to advocate those modern nonparametric techniques
which have proved useful for analyzing real time series data, and to provoke
further research in both methodology and theory for nonparametric time
series analysis.
Exponential increases in computing power (Moore’s law) and falling costs
have had a profound impact on the development of statistics including time
series. Modern computers and the information age have brought us oppor-
tunities with challenges. Technological inventions have lead to the explosion
in data collection (e.g. daily grocery sales, stock market trading, microar-
ray data). The internet makes big data warehouses readily accessible. It
has become commonplace to collect and analyze time series data of un-
precedented size and complexity. Whilst classic parametric models, which
postulate global structures for underlying systems, are still very useful,
large sizes of data sets invite the search for more refined structures, which
leads to better understandings and approximations of the real world. Be-
yond postulated parametric models, there are infinite other possibilities.
Nonparametric techniques provide useful exploratory tools to this venture,
vi
PREFACE
including the suggestion of new and the validating of existing parametric
models.
In this book, we present an up-to-date picture of techniques for analyz-
ing time series data. Although we have tried to maintain a good balance
among methodology, theory, and numerical illustration, our primary goal
is to present a comprehensive and self-contained account for each of the
key methodologies. For practical relevant time series models, we aim for
exposure with definition, probability properties (if possible), statistical in-
ference methods, and numerical examples with real data sets. We also in-
dicate where to find our (only our!) favorite computing codes to implement
these statistical methods. When soliciting real-data examples, we attempt
to maintain a good balance among different disciplines, though our per-
sonal interests in quantitative finance, risk management, and biology can
be easily seen. It is our hope that readers can apply these techniques to
their own data sets.
We trust that the book will be of interest to those coming to the area
for the first time, and to readers more familiar with the field. Application-
oriented time series analysts will also find this book useful as it focuses on
methodology and includes several case studies with real data sets. We be-
lieve that nonparametric methods must go hand in hand with parametric
methods in applications. In particular, parametric models provide explana-
tory power and concise descriptions of the underlying dynamics, which,
when used sensibly, is an advantage over nonparametric models. For this
reason, we have also provided a compact view on the parametric methods
for both linear and selected nonlinear time series models. This will also
give new comers sufficient information on the essence of the more classical
approaches. We hope that this book will reflect the power of the integration
of nonparametric and parametric approaches in analyzing time series data.
The book has been prepared for a broad readership — the prerequisites are
merely sound basic courses in probability and statistics. Whilst advanced
mathematics has provided valuable insights into nonlinear time series, the
methodological power of both nonparametric and parametric approaches
can be understood without sophisticated technical details. Due to the in-
nate nature of the subject, it is inevitable that we occasionally appeal for
more advanced mathematics; such sections are marked with a ‘*’. Most
technical arguments are collected in ‘Complements’ section at the end of
each chapter, but key ideas are left within text.
The introduction in Chapter 1 sets the scene for the book. Chapter 2
deals with basic probabilistic properties of time series processes. The high-
lights include strict stationarity via ergodic Markov chains (§2.1) and mix-
ing properties (§2.6). We also provide a generic central limit theorm for
kernel-based nonparametric regression estimation for α-mixing processes.
A compact view of linear ARMA models is given in Chapter 3, including
Gaussian MLE (§3.3), model selection criteria (§3.4), and linear forecasting
with ARIMA models (§3.7). Chapter 4 introduces three types of paramet-
PREFACE
vii
ric nonlinear models. An introduction on threshold models that emphasizes
developments after Tong (1990) is provided. ARCH and GARCH models
are presented in detail, as they are less exposed in statistical literature.
The chapter concludes with a brief account of bilinear models. Chapter 5
introduces the nonparametric kernel density estimation. This is arguably
the simplest problem for understanding nonparametric techniques. The re-
lation between “localization” for nonparametric problems and “whitening”
for time series data is elucidated in §5.3. Applications of nonparametric
techniques for estimating time trends and univariate autoregressive func-
tions can be found in Chapter 6. The ideas in Chapter 5 and §6.3 provide a
foundation for the nonparametric techniques introduced in the rest of the
book. Chapter 7 introduces spectral density estimation and nonparametric
procedures for testing whether a series is white noise. Various high-order au-
toregressive models are highlighted in Chapter 8. In particular, techniques
for estimating nonparametric functions in FAR models are introduced in
§8.3. The additive autoregressive model is exposed in §8.5, and methods for
estimating conditional variance or volatility functions are detailed in §8.7.
Chapter 9 outlines approaches to testing a parametric family of models
against a family of structured nonparametric models. The wide applicabil-
ity of the generalized likelihood ratio test is emphasized. Chapter 10 deals
with nonlinear prediction. It highlights the features that distinguish non-
linear prediction from linear prediction. It also introduces nonparametric
estimation for conditional predictive distribution functions and conditional
minimum volume predictive intervals.
1
2
ppppppppppppppppp-
@
@
@R
I
@
@
@
6
-
-
4
3
pppppppppppppppppppppppppppppppppppppppppppppppppppU
ppppppppppppppppppppppppppppppppppppppppppp
pppppppppppppppppppp
7
5
6
8
ppppppppppppppppppppppppppppppppppppppppppp
10
-
-
6
@
@
?
9
@
@@R
-
The interdependence of the chapters is depicted above, where solid di-
rected lines indicate prerequisites and dotted lines indicate weak associ-
ations. For lengthy chapters, the dependence among sections is not very
strong. For example, the sections in Chapter 4 are fairly independent, and
so are those in Chapter 8 (except that §8.4 depends on §8.3, and §8.7 de-
pends on the rest). They can be read independently. Chapter 5 and §6.3
provide a useful background for nonparametric techniques. With under-
viii
PREFACE
standing of these, readers can jump directly to sections in Chapters 8 and
9. For readers who wish to obtain an overall impression of the book, we
suggest reading Chapters 1, §2.1, §2.2, 3, §4.1, §4.2, 5, §6.3, §8.3, §8.5, §8.7,
§9.1, §9.2, §9.4 and §9.5. These core materials could even serve as the text
of a graduate course on nonlinear time series.
Although the scope of the book is wide, we have not achieved complete-
ness. The nonparametric methods are mostly centered around kernel/local
polynomial based smoothing. Nonparametric hypothesis testing with struc-
tured nonparametric alternatives is mainly confined to the generalized like-
lihood ratio test. In fact, many techniques that are introduced in this
book have not been formally explored mathematically. State-space mod-
els are only mentioned briefly within the discussion on bilinear models and
stochastic volatility models. Multivariate time series analysis is untouched.
Another noticeable gap is the lack of exposure of the variety of paramet-
ric nonlinear time series models listed in Chapter 3 of Tong (1990). This
is undoubtedly a shortcoming. In spite of the important initial progress,
we feel that the methods and theory of statistical inference for some of
those models are not as well-established as, for example, ARCH/GARCH
models or threshold models. Their potential applications should be further
explored.
Extensive effort was expended in the composition of the reference list,
which, together with the biographical notes, should guide readers to a
wealth of available materials. Although our reference list is long, it merely
reflects our immediate interests. Many important papers which do not fit
our presentation have been omitted. Other omissions and discrepancies are
inevitable. We apologize for their occurrence.
Although we both share the responsibility for the whole book, Jianqing
Fan was the lead author for Chapters 1 and 5–9, and Qiwei Yao for Chapters
2-4 and 10.
Many people have been of great help to our work on this book. In par-
ticular we would like to thank Hong-Zhi An, Peter Bickel, Peter Brockwell,
Yuzhi Cai, Zongwu Cai, Kung-Sik Chan, Cees Diks, Rainer Dahlhaus, Li-
udas Giraitis, Peter Hall, Wai-Keung Li, Jianzhong Lin, Liang Peng, Stathis
Paparodistis, Wolfgang Polonik, John Rice, Peter Robinson, Richard Smith,
Howell Tong, Yingcun Xia, Wenyang Zhang and anonymous reviewers. We
are also grateful to all of our collaborators for their friendly and stimulating
collaboration. Many of their ideas have been reflected in this book.
Jianqing Fan’s research was partially supported by the National Science
Foundation and National Institute of Health of the USA and the Research
Grant Council of the Hong Kong Special Autonomic Region. Qiwei Yao’s
work was partially supported by the Engineering and Physical Sciences
Research Council and the Biotechnology and Biological Sciences Research
Council of UK. This book was written while Jianqing Fan was employed
by the University of California at Los Angeles, the University of North
Carolina at Chapel Hill, and the Chinese University of Hong Kong; and
PREFACE
ix
Qiwei Yao was employed by the University of Kent at Canterbury and
the London School of Economics and Political Science. We would like to
acknowledge gratefully for the generous support and inspirations of our
colleagues.
May 2002
Jianqing Fan
Qiwei Yao