Biometrika (1967), 54, 3 and 4, p.

403 403
With 1 text-figure
Printed in Great Britain

Time series with periodic structure

B Y RICHARD H. JONES AND WILLIAM M. BRELSFORD
Johns Hopkins University

SUMMARY
Time series which are encountered in meteorology exhibit non-stationary behaviour.
If a variable is observed at the same time each day, it appears to be stationary over a period
of a few weeks, but there is a seasonal variation in structure (Monin, 1963). For example,

Downloaded from http://biomet.oxfordjournals.org/ at University of Toronto Library on November 22, 2014

the mean and spectral density are different in the summer and in the winter. The usual
approach to this problem is to divide the year into parts and analyse the data separately
for the same two or three months from several years. This paper is concerned with the
problem of predicting time series with periodic structure, including the estimation of the
necessary parameters and variances.

1. INTRODUCTION
In the strict sense, a stochastic process, X(t), has periodic structure if the joint probability
distribution of X(t1),X(t2), ...,X(tn) is the same as the joint probability distribution of
X(t1 + kT),X(ti + kT),...,X(tn + kT) for any n time points tit any integer k, and some
positive number T called the period. A strictly stationary process is a special case since the
joint probability distributions are the same for any value of T. Gladyshev (1961, 1963)
defines a periodically correlated random sequence as a process which has periodic structure
in the usual wide sense. Assuming that E{X2(f)} is finite for all t, let m(t) = E{X(t)}. Then
m(t) = m(t + kT) for all t and any integer k. The covariance function
R(s,t) = E[{X(s)-m(s)}{X(t)-m(t)}]
satisfies R(s, t) = R(s + kT, t + kT) for all s and t and any integer k. If t is a continuous
parameter it will be assumed that X(t) is continuous in quadratic mean. Gladyshev (1963)
also shows that a stochastic process with periodic structure is harmonizable (Loeve, 1963,
p. 474) since it has the representation
X(t)-m(t)= LwdZiJ).
Here Z{f) is a random function with zero mean, and the limits of integration are taken as
( \, \) if t = 0, + 1, 2,..., and ( oo, oo) if t is a continuous parameter. The covariance
function of a harmonizable process has the representation

R(s, t) = JJexp {27ri(A -1/2)} dS{fltft),

where dS(fvf2) = E{dZ(fx) dZ(fa)}.
For a process with periodic structure Z(f) is not necessarily a process of uncorrelated
increments. The two-dimensional function #(/i,/j>) is constant unless T{fxft) is an integer.
If S(fltf2) is considered a two-dimensional spectral distribution function, the spectral mass
is concentrated on parallel lines.
404 R. H. JONES AND W. M. BRELSFOBD

2. ATTTOREGRESSIVE REPRESENTATION
A practical method of predicting time series is to fit a finite autoregression by least squares.
If a discrete-time stationary process has a spectral density bounded away from zero, which
will always be true for observed series because of observational error or rounding after a
finite number of decimals, the spectrum can be approximated by a finite autoregression. If
the order of the autoregression is not decided before the data are analysed, this is, in a sense,
a non-parametric procedure for linear prediction. Using the step-wise procedure suggested
by Whittle (1963), autoregressions of increasing order can be estimated using a minimum of
calculation.
For a process with periodic structure, the autoregressive coefficients can vary throughout
the period; however, the coefficients must vary in a periodic manner. The use of time-varying

Downloaded from http://biomet.oxfordjournals.org/ at University of Toronto Library on November 22, 2014

coefficients for predictors of processes with seasonal variation in structure has been used by
Harrison (1965) for exponentially weighted moving averages and discussed by Whittle
(1965) as a special case of more general non-stationary processes.
If a process with periodic structure, X(t), has zero mean, the finite autoregression can be
p) = e(t), (2-1)
where
E{e(t)} = 0, E{e(s)e(t)} = 0 (=M), E{ez(t)} = E{e\t + JcT)}
for all s, t and all integers A;. As for stationary processes, the autoregressive coefficients,
am(t), are not completely arbitrary. The necessary restrictions can be obtained by trans-
forming the process to a multivariate stationary process by breaking it into periods and
treating each time point within a period as one component of a multivariate process. This
technique has been used by Gladyshev (1961) to demonstrate the harmonizability of
periodically correlated random sequences. The conditions on the autoregressive coefficient
matrices of a multivariate stationary process (Whittle, 1963) give the necessary conditions
for am(t).
Since the autoregressive coefficients are periodic with period T they can be expanded in
a Fourier Series,
ajt-m)= 2 [amncoa{27Tn(t-m)IT} + bmnBin{27Tn(t-m)IT}]. (2-2)
n-0

If the structure of the process varies slowly with time, am{t m) can be approximated by
a Fourier Series consisting of only a few terms,

am{t-m)~ i[amncos{2nn(t-m)IT} + bmnBin{2nn(t-m)IT}lq<ttT]. (2-3)

n-0

Equation (2-1) becomes

X(t) + 2 S X(t - m) [amn cos {2nn(t - m)/T} + bmn sin {2nn{t - m)/T}] = e(t) (2-4)
m-1n-0

or X(t) = - am0X(t-m)- f; [amnX(t-m) cos {2nn{t-m)IT}

m1 m1 n = l
+ bmnX(t-m)mn{2nn(t-m)IT}]+e(t). (2-5)
This equation shows that linear predictions of time series with periodic structure are
obtained by regressing the present, not only on its own past, but also on the past of time
 Time series with periodic structure 405
series obtained by multiplying the data by sines and cosines of the fundamental and harmonic
frequencies of the period.
The extension to multivariate time series with periodic structure requires only that X(t) be
interpreted as a column vector and amn and bmn as square matrices.

3. A MTTLTIVARTATE EXAMPLE
Estimation of the autoregressive parameters has been carried out on bivariate temperature
data consisting of daily maximum and minimum temperatures at New York Central Park.
In order to remove the periodic mean so we could concentrate on the second-order properties,
the data were divided into 4-year segments consisting of 1460 time points starting 1 March

Downloaded from http://biomet.oxfordjournals.org/ at University of Toronto Library on November 22, 2014

on a leap year. Two segments were paired and subtracted giving a bivariate series of 1460
"time points and zero mean. This method is not recommended if there is a shortage of
observations since the periodic mean could be removed by trigonometric regression or by
the method used by Jones (1964).
This problem could be set up in the usual linear regression framework
y = Xfi + e, (3-1)
where, for a bivariate process, y is an nx 2 matrix, X wnxp, fiiapx2 and e is nx 2. The
difficulty is the calculation of X'X which requires duplication of lag product multiplications,
and as in stationary processes, stability of the solution is not guaranteed.
The method used reduced the problem to the prediction of a subset of the components of
a multivariate stationary process. By using a positive definite estimate of the covariance
function matrices, a stable solution is guaranteed (Whittle, 1963).
From the bivariate time series

the following i-dimensional multivariate time series is formed:

U(t) = [ x ^ O . z ^ W O s i n (2ntjT),xi(t)sin(2ntjT),x1(t) cos[2ntjT),...]' (3-3)
a general element having the form x^t) times a sinusoidal term of wavelength T\r. The
prediction is obtained by regressing X(t) on U(t 1), U(t 2),..., U(t p),
= B(l)U(t-l)+B(2)U(t-2) + ...+B(p)U(t-p) + e(t). (3-4)
The variation of the usual linear regression solution was obtained by first calculating the
matrices N
T(m)= U(t)U'(t-m) (m = 0,l,...,p-l
N-m
H(m)= 2 U(t)X'(t + m) (m=l,2,...,p)
t-i

and forming the modified normal equations

no)
3(2) r(0) (2)
(3-6)
rjo)J
Biom. 54
406 R. H. J O N E S AND W. M. BRELSFOBD

The matrix F in (3-6) was inverted by a stage-wise partitioning procedure, which required
that only the r(0), F(l), ..., T{p 1) matrices and the inverse be stored. The inverse was
generated using the well-known partitioning formula for a symmetric matrix. At a typical
stage,
F(0)
r'(i) F(0) F(r-2) (3-7)

F'(r-2) rjo) J
is available from the previous step.
The estimated regression coefficients were obtained,
(3-8)

Downloaded from http://biomet.oxfordjournals.org/ at University of Toronto Library on November 22, 2014

and the residuals calculated
e(t) = fi U(t-l)-&(2) U(t-2)-...-'(p) U(t-p). (3-9)-
As is usual in autoregression, the mean square error is an estintate of the one-step prediction
variance. However, if the process has periodic structure, the error variance can vary
periodically. This type of process was studied by Herbst (1963) and a multivariate version
of bis techniques was applied to the residuals. An estimate of the variation was obtained by
assuming that the error variance could be approximated by a Fourier Series consisting of
only a few terms. The coefficients were estimated for n = 1,..., 7

F - e(t)e'(t),
N
2 e(t)e'(t) cos (2TTTUIT), (3-10)
N
W = 2 e{t)e'(t)8va(27TntlT).
N-pk
To obtain a weight function in time with small side lobes these coefficients were weighted
by {1 + cos (i7m)}
V V 1
y v
0 0'
Fn = F n {l + cos(7m)}, (3-11)
Wn = ^ ^ { l + COSliTTO)},
and the weights calculated
7

n-1
The regression was then repeated using the estimated error variance for weighted least
squares.
4. RESULTS
From past analyses of the data, it was decided that three lags add significantly to a
prediction. For the first trial in this study, U(t) consisted of ten components involving two
harmonics. The use of three lags gave thirty predictors. An approximate t-statistic was
calculated for each estimated regression coefficient by dividing it by its estimated standard
deviation. After examining the results, it was decided that for lags two and three the second
 Time series with periodic structure 407
harmonic did not contribute significantly to the reduction of the residual sum of squares;
the data were reanalysed using only one harmonic frequency for these lags. This gave
twenty-two predictors.
By assuming periodically varying autoregressive coefficients, there was a significant
reduction in the residual sum of squares for the miniTnuTn temperature, compared to fitting
constant autoregressive coefficients. The weighted least-squares analysis changed the
estimated regression coefficients slightly and decreased the error of estimation. This, of
course, would give the minimum variance linear unbiased estimate of the regression coeffi-
cients if the weights had been based on the true rather than the estimated error variance.
Figure 1 shows the estimated error variance for the maximum and minimum temperatures
as a function of time.

Downloaded from http://biomet.oxfordjournals.org/ at University of Toronto Library on November 22, 2014

70 , Weight function

60

50
Maximum

40

30

20

10

Spring Summer Autumn Winter

Fig. 1. Estimated time varying one-step prediction error variance for mivrimiim and minimum
temperature. Residuals from 4 years of daily observations were averaged and smoothed
using the weight function which is shown.

From the regression coefficients, B{\), B(2) and .6(3), it is possible to calculate the auto-
regressive coefficients, (^(t), a2(t) and ag(t), in the equation

which are 2 x 2 matrices and periodic functions of time. This, in turn, defines a time-varying
spectrum from the usual formula for stationary processes
s(f,t) = [A(f,t)]-iV(t)[A*(f,t)]-\
where * denotes the conjugate transposed matrix and

A[f,t)= S ajt-m)e-f,

ao(t) being defined as the 2 x 2 identity matrix. A discussion of instantaneous spectra for
non-stationary processes is given by Priestley (1965).
26-1
408 R. H. JONES AND W. M. BBELSFORD

5. CONCLUSION
Linear prediction of time series with periodic structure can be achieved by regressing
the present on the past of the time series and on the past of time series constructed by
multiplying the data by sines and cosines of harmonic frequencies of the period. This reduces
the problem to the prediction of a subset of the components of a multivariate stationary
process using the past of all components as predictors. In the example presented here it was
assumed that the structure varied slowly with respect to the spacing of observations so that
the periodic structure could be approximated using Fourier series of only a few terms.
However, in some cases this assumption is not necessary. Often in meteorology, observations
are taken twice a day and there is a daily variation in structure. Considering only this daily

Downloaded from http://biomet.oxfordjournals.org/ at University of Toronto Library on November 22, 2014

variation, all the linear information available for prediction can be extracted by regressing
the present on the past of the data and on the past of the data multiplied b y l , 1,1, 1,....
An intermediate situation is annual variation in structure and monthly observations.

Support by the U.S. Air Force Office of Scientific Research is gratefully acknowledged.

REFERENCES
GLADYSHEV, E. G. (1961). Periodically correlated random sequences. Soviet Math. 2, 385-8.
GLADYSHEV, E. G. (1963). Periodically and almost periodically correlated random processes with
continuous time parameter. Theory Prob. Appl. 8, 173-7.
HARBISON, P. J. (1965). Short-term sales forecasting. Appl. Statist. 14, 102-39.
HERBST, L. J. (1963). Periodogram analysis and variancefluctuations.J. R. Statist. Soc. B 25, 422-50.
JONES, R. H. (1964). Spectral analysis and linear prediction of meteorological time series. J. Appl.
Meteor. 3, 45-52.
LOEVE, M. (1963). Probability Theory (third edition). Princeton: D. Van Nostrand.
MONXN, A. S. (1963). Stationary and Periodic time series in the general circulation of the atmosphere.
Proceedings of the Symposium on Time Series Analysis, pp. 14451, edited by M. Rosenblatt. New
York: Wiley.
PRIESTLEY, M. B. (1965). Evolutionary spectra and non-stationary processes. J. R. Statist. Soc. B 27,
204-37.
WHITTLE, P. (1963). On the fitting of multivariate autoregressions, and the approximate canonical
factorization of a spectral density matrix. Biometrika 50, 129-34.
WHITTLE, P. (1965). Recursive relations for predictors of non-stationary processes. J. R. Statist. Soc.
B 27, 523-32.

[Received August 1966. Revised March 1967]