TS Module 7 Stationary mixed processes
(The attached PDF file has better formatting.)
Time Series Practice Problems ARMA Means and Variances
ARMA processes have autoregressive parameters Nj, moving average parameters 2j, a
constant * or 20, and a standard error F . We derive 2
! Means and variances of the Yt terms and an autocorrelation function.
! Forecasts with their standard errors and confidence intervals.
The practice problems here cover means and variances of the time series. Other sets of
practice problems show how to work more complex items.
The textbook shows all the relations tested on the final exam and used in the student
projects. As you review the modules, work through the derivations of each result. The
relations follow directly from first principles.
The final exam problems emphasize the intuition. An exam problem might give the mean
or variance of the time series and back into a coefficient. If you understand how
autoregressive and moving average parameters affect the means, forecasts, variances,
and standard errors, the problems are straight-forward.
*Question 7.1: Mean of ARMA Process
An ARMA(p,q) process is Yt = 3 + ,t + moving average parameters 2j = 0.5 for j = 1 to 4 j
and autoregressive parameters Nk = 0.366 for k = 1 to 2. Note that the constant term is k
3. What is the mean of this time series?
A. 2
B. 4
C. 6
D. 8
E. 10
Answer 7.1: C
The constant term is called 20 in the Cryer and Chan textbook. Other authors use * or ".
The constant term causes a displacement, not a change in the time series pattern.
Intuition: If all observed terms are the mean and all residuals are zero, the forecast of the
next term is the mean. Substitute : for the Y terms and 0 for the , terms, giving a linear
equation in one unknown :.
For this exercise, : = 3 + : × (0.3661 +0.3662) + 0 × (terms with 2)
The mean : of this process is : = 3 / (1 – 3Nj) = 3 / (1 – 0.366 – 0.366 ) = 5.999 2
The textbook has two ways of writing an ARMA process:
! Using terms of (Yt – :), with no constant term.
! Using terms of Yt, with a constant term.
They are equivalent.
*Question 7.2: Mean of ARMA Process
An ARMA(p,q) process is Yt = 3 + ,t + moving average parameters 2j = 0.5 for j = 1 to 4 j
and autoregressive parameters Nk = 0.366 for k = 1 to 2. All values for t < 10 were the k
mean, and all residuals were zero.
If the actual value y10 = 3, what is the expected value of y11?
A. 3.0
B. 3.3
C. 3.6
D. 6.0
E. 6.4
Answer 7.2: E
! The residuals for t < 10 are zero (by the assumptions).
! The expected value for t = 10 is the mean; work out the residual for t = 10.
The moving average coefficient = the negative of the moving average parameter.
The mean : of this ARMA process is : = 3 / (1 – 3Nj) A
: = 3 / (1 – 0.366 – 0.3662) = 5.999
! The residual for y10, or ,10, is 3 – 6 = –3.
! The expected value of y11 = 3 + 0.366 × 3 + 0.366 × 6 – 0.5 × (–3) = 6.402 2
*Question 7.3: Mean of ARMA Process
An ARMA(p,q) process has moving average parameters 2j = 0.5 for j = 1 to 4 and j
autoregressive parameters Nk = 0.366 for k = 1 to 2. The parameter * = 5. What is the k
mean of this time series?
A. 7
B. 8
C. 9
D. 10
E. 11
Answer 7.3: D
The formula relating : and * for an autoregressive process is : = * / (1 – 3Nj) A
: = 5 / (1 – 0.366 – 0.3662) = 9.999
To verify, we assume all past values are the mean of 10 and all past residuals are zero.
The next value is expected to be 5 + 0.366 × 10 + 0.3662 × 10 = 10.000 .
*Question 7.4: Mean of ARMA Process
An ARMA(p,q) process has moving average parameters 2j = 0.5 for j = 1 to 4 and j
autoregressive parameters Nk = 0.366 for k = 1 to 2. The parameter * = 5. Assume that k
all values for t < 10 were the mean, and all residuals were zero.
If y10 = 9, what is the expected value of y11? We worked out the mean in the previous
problem, which gives the values for y9 and y8. The residuals for t # 9 are zero (by the
assumptions). The expected value for t = 10 is the mean, from which we work out the
residual for t = 10. The convention in the textbook is that the moving average coefficient
is the negative of the moving average parameter.
A. 8
B. 9
C. 10
D. 11
E. 12
Answer 7.4: C
The residual for y10, or ,10, is 9 – 10 = –1.
The expected value of y11 = 5 + 0.366 × 9 + 0.366 × 10 – 0.5 × (–1) = 10.005 2
*Question 7.5: ARMA Mean
We use an ARMA(3,1) model to forecast a time series:
The mean of this time series is 9. What is the constant term * of the time series?
A. –2.70
B. –2.25
C. –1.20
D. +2.25
E. +2.70
Answer 7.5: E
Cryer and Chan use the variable 20 instead of *. The discussion forum postings use 20,
*, or ". The final exam problems may derive the constant term, the mean, or one
autoregressive parameter. The solution is straight-forward.
The mean is the constant term * divided by (1 – 3Nj). The autoregressive coefficients
affect the mean; the moving average coefficients do not affect the mean.
j j : = * / (1 – 3N) A * = : × (1 – 3N) = 9 × (1 – 0.3 – 0.2 – 0.2) = 2.700
*Question 7.6: AR(1) Process
We use a time series process to model a sequence of values:
What is the mean of this process?
A. 3
B. 6
C. –6
D. 12
E. The time series is not stationary and has no mean
Answer 7.6: E
If the absolute value of the N1 coefficient is more than or equal to one, the time series has
no mean (is not stationary). For higher order processes, the conditions for stationarity are
more complex.
Jacob: Why does the time series have no mean? Suppose N1 is a negative number less
than or equal to –1. Why can’t we use the formula : = * / (1 – N1)?
Rachel: Suppose N1 is –2 and y1 is zero. The formula : = * / (1 – N1) = 6 / (1 – –2) = 2.
The expected values of the next entries are +6, –6, +18, –30, +66, and so forth. The
expected values assume , = 0.
The values oscillate about 2, but they move away from 2; they don’t approach 2.
Jacob: What if N1 = –1, as in the practice problem here?
Rachel: The formula gives : = * / (1 – N1) = 6 / (1 – –1) = 3.
The expected values of the next entries are +6, 0, +6, 0, +6, 0, and so forth.
The values oscillate about 3, but they don’t converge or diverge. The series has no mean.
Take heed: The mean : is the expected value as t ÿ 4. Even if an error term , moves the
time series away from this mean for a few periods, later values come back to the mean.
In this time series, if a random fluctuation moves the process away from 3, later terms do
not revert to 3.
*Question 7.7: AR(1) Process
We use an time series process to model a sequence of values:
What is (0, the variance of yt?
A. 4
B. 6
C. 8
D. 12
E. The time series is not stationary and has no finite variance.
Answer 7.7: E
The N1 coefficient is greater in absolute value than one. The uncertainty in the forecasts
increases without bound as the number of periods ahead increases. The variance of yt is
like the variance of the forecast for an infinite number of periods ahead.
Jacob: Is the variance of Yt the same as the variance of the error term?
Rachel: Distinguish the variance of the error term and the variance of the Y terms.
Suppose we know the past terms of the time series and we must estimate the next value.
! The value is stochastic, so we do not know it with certainty.
! We estimate the distribution of this value.
! The distribution is normal with a variance F2.
Suppose we do not know any past terms of the time series and we estimate the next value.
! The mean of the distribution depends on the past values.
" For an AR(1) process, it depends on one past value.
! For other ARMA processes, it may depend on several past values.
" Each unknown value adds uncertainty and increases the variance of the estimate.
" We combine the various sources of uncertainty.
If the variances of the sources are independent, we add them. Some sources of
uncertainty are correlated, such as autoregressive and moving average parameters for the
same period.
In an ARMA(1,1) process, the uncertainty is perfectly correlated. We add the
autoregressive and moving average coefficients (i.e., subtract the parameters).
Think of the variances in a continuum.
! The variance of the one period ahead forecast is the variance of the error term.
! The variance of the l period ahead forecast as l ÿ 4 is the variance of Yt.
! In between are the variance of other forecasts.
*Question 7.8: AR(1) Process
We use an AR(1) process (autoregressive of order 1) to model a time series:
, with F2 = 21.
What is (0, the variance of yt?
A. 2.1
B. 7
C. 21
D. 25
E. 49
Answer 7.8: D
21 / (1 – 0.42) = 25.000
This is not the variance of the forecast, which assumes we know the past values. This is
the variance of the time series value itself.
Each Yt is the sum of two random variables: ,t and 0.4yt-1. In an autoregressive process,
yt depends on the lagged values, but the residuals are independent of the lagged values.
You can solve this problem by the formula in the textbook or by intuition. The intuition is
that the variance is the same for all elements of the time series. The variance of yt equals
the variance of yt-1. Form an equation for the relation of the variances from the time series
equation Var(yt) = 0.4 × Var(yt-1) + 0 + F 2 2
Jacob: In the regression analysis course, the variance of Y is the variance of the error term.
Why is that not true here?
Rachel: In the regression analysis course, Y is a function of X, which is not a random
variable. In an autoregressive time series, Yt is a function of Yt-1, which is a random
variable.
*Question 7.9: Moving Average Model of Period N
A time series follows a moving average model of period N:
What is the mean of this time series?
A. ½
B. 1
C. 1 + 1/N
D. 2
E. The time series is not stationary and does not have a mean
Answer 7.9: B
For a moving average process, : = * = 20 = 1 (in this problem).
The mean does not depend on the moving average parameters.
*Question 7.10: Moving Average Model of Period N
A time series follows a moving average model of period N:
What is the variance of this time series?
A. 1
B. (N + 1)/N = 1 + 1/N
C. (N + 1)2/N2
D. 2
E. The time series is not stationary and does not have a variance
Answer 7.10: B
For a moving average process, the residuals are independent and have the same variance.
! The variance of the sum of the residuals is the sum of the variances.
! The variance of "Y, where " is a scalar and Y is a random variable, is "2 × variance(Y).
A moving average process is stationary if the sum of the moving average parameters is
finite.
The variance is F2 × (1 + N × 1/N2) = 1 × (1 + 1/N) = 1 + 1/N