TS Module 9 Non-stationary ARIMA time series
(The attached PDF file has better formatting.)
Time series ARIMA processes practice problems
** Exercise 9.1: ARIMA(0,1,1) process
What is an ARIMA(0,1,1) process?
Write an ARIMA(0,1,1) process as a series of error terms.
What is the variance of an ARIMA(0,1,1) process?
What is the mean of an ARIMA(0,1,1) process?
Part A:
An ARIMA(0,1,1) process is a once-integrated MA(1), called an IMA(1,1) process in the text.
The first difference of an ARIMA(0,1,1) process is an MA(1) process.
If Y_{t} is an ARIMA(0,1,1) process = an IMA(1,1) process, then
Part B:
Express Y_{t} in terms of Y_{t-1} and Y_{t-1} in terms of Y_{t–2}:
Y_{t} = Y_{t-1} +
å_{t} – è å_{t-1}.
Y_{t-1} = Y_{t-2} +
å_{t-1} – è å_{t-2}.
Y_{t} = Y_{t-2} +
å_{t} + å_{t-1} – è å_{1-1} – è å_{1-2} = Y_{t-2} + å_{t} + (1 – è) å_{1-1} – è å_{1-2}.
We continue in this fashion to get
Y_{t} =
å_{t} + (1 – è) å_{1-1} + (1 – è) å_{1-2} + (1 – è) å_{1-3} + …
Part C:
The error terms
å_{j} are independent with a variance of ó^{2}_{t} for each one. The variance of Y_{t} is
ó
^{2}_{t} + (1 – è)^{2} × ó^{2}_{t} + (1 – è)^{2} × ó^{2}_{t} + (1 – è)^{2} × ó^{2}_{t} + … = infinity.
Part D:
The mean of each error term
å_{j} is zero, so one is tempted to say that the mean of Y_{t} is zero. But the sum of an infinite number of random variables, each of which has a mean of zero, is not necessarily zero.
If the observed value of Y_{t} is 1, the expected value of Y_{t+1} is 1; if the observed value of Y_{t} is 2, the expected value of Y_{t+1} is 2. Given any observed value in the current period, the expected values in future periods differ. The values do not regress toward any point; that is, they have no mean.
Jacob:
What is the drift of an ARIMA process?
Rachel:
The drift of an ARIMA process with d = 1 is the mean of the underlying ARMA process. If the MA(1) process has a mean of
ì, the expected value of the ARIMA(0,1,1) process increases by ì each period.
ARIMA processes with fixed starting points.
Cryer and Chan begin with non-stationary processes that have a fixed starting point. If the time series has no starting point, its mean and variance are not defined. For an infinite ARIMA process, Y_{t} has no mean or variance. Each entry in the non-stationary time series is the sum of an infinite number of random variables that do not die out. To explain the pattern of these processes, we assume they begin at some time t.
This is not really a restrictive condition. Most commonly, we forecast future values of a time series based on historical observations. Before the first observation, we assume all values of the time series are zero. We model the evolution of the mean and variance of the process.
**Exercise 9.2: ARIMA(0,1,1) process
Suppose
with Y
_{t} = 0 for t < 1.
What is the variance of Y_{1}?
What is the variance of Y_{2}?
What is the variance of Y_{T}?
Part A:
Y_{1} = Y_{0} +
å_{1} – è å_{0} = 0 + å_{1} – è å_{0}
The variance of
å is ó^{2}_{å}, so the variance of Y_{1} is (1 + è^{2}) × ó^{2}_{t}.
Part B:
Y_{2} = Y_{1} +
å_{2} – è å_{1}
_{}Y_{1} = Y_{0} +
å_{1} – è å_{0} = 0 + å_{1} – è å_{0}, so Y_{2} = å_{2} + (1 – è) å_{1} – è å_{0}
The variance of
å is ó^{2}_{å}, so the variance of Y_{2} is (1 + (1 – è)^{2} + è^{2}) × ó^{2}_{t}.
Part C:
We continue this process T times, giving the variance of Y_{2} = (1 + (T–1) × (1 –
è)^{2} + è^{2}) × ó^{2}_{t}.
Jacob:
For an ARMA process, we evaluate the variance of Y_{t}, where t may be any element of the time series. Each element has the same mean, so it is a random variable, which has a variance.
For this ARIMA(0,1,1) process, Y_{1}, Y_{2}, and Y_{T} are specific observations; they are values, not random variables. How can they have variances?
Rachel:
We have formulated the exercise as Cryer and Chan do in their textbook. We are standing at time t=0, and we are projecting the values at times t=1, t=2, and t=T. Now these are random variables; each has a variance.
Jacob:
If we don’t specify that Y_{t} = 0 for t < 1, what is the variance of Y_{t}?
Rachel:
An ARIMA(0,1,1) process is not stationary and has no variance.
Cryer and Chan, P94: equation 5.2.7, for IMA(1,1) process, is
=
Cryer and Chan assume that
Y_{t} = 0 for periods t < –m. They assume we are now at Period 0, looking forward to estimate the variances at Periods 1, 2, 3, …. The first observed value is Period 1, but the process has been going on since Period –(m-1). We don’t observe the value at Period 0, but we know the time series process.
The Cryer and Chan scenario is complex, making it difficult to solve problems by first principles. Final exam problems use the scenario here: Y_{t} = 0 for t < 1.
** Exercise 9.3: ARIMA(0,1,1) process
Suppose
with
è = 0.4 and ó^{2}_{å} = 4 for t > 0 and Y_{t} = 0 for t < 1.
What is the variance of Y_{1}?
What is the variance of Y_{2}?
What is the variance of Y_{3}?
Part A:
Y_{1} = Y_{0} +
å_{1} – è å_{0} = 0 + å_{1} – è å_{0}
The variance of
å is 4, so the variance of Y_{1} is (1 + 0.4^{2}) × 4 = 4.64
Part B:
Y_{2} = Y_{1} +
å_{2} – è å_{1}
_{}Y_{1} = Y_{0} +
å_{1} – è å_{0} = 0 + å_{1} – è å_{0}, so Y_{2} = å_{2} + (1 – è) å_{1} – è å_{0}
The variance of
å is 4, so the variance of Y_{2} is (1 + (1 – 0.4)^{2} + 0.4^{2}) × 4 = 6.08
Part C:
The variance of Y_{T} is (1 + (T – 1) × (1 –
è)^{2} + è^{2}) × ó^{2}_{t}.
The variance of Y_{3} is (1 + 2 × (1 – 0.4)^{22} + 0.4^{2}) × 4 = 7.5200
** Exercise 9.4: ARIMA Process
A time series is Y_{t} =
á_{1} Y_{t-1} + á_{2} Y_{t-2} + (1 – á_{1} – á_{2}) Y_{t-3} + e_{t} + â_{1} e_{t-1} + â_{2} e_{t-2}
_{}Write the time series in terms of W_{t} ( Y_{t}).
What ARIMA process is this time series?
What are the
ö and è coefficients of this ARIMA process?
Part A:
Rewrite the ARIMA process as
Y_{t} – Y_{t-1} = (
á_{1} – 1) Y_{t-1} + á_{2} Y_{t-2} + (1 – á_{1} – á_{2}) Y_{t-3} + e_{t} + â_{1} e_{t-1} + â_{2} e_{t-2}
_{}Y_{t} – Y_{t-1} = (
á_{1} – 1) Y_{t-1} – (á_{1} – 1) Y_{t-2} + (á_{1} – 1) Y_{t-2} + á_{2} Y_{t-2} + (1 – á_{1} – á_{2}) Y_{t-3} + e_{t} + â_{1} e_{t-1} + â_{2} e_{t-2}
_{}Y_{t} – Y_{t-1} = (
á_{1} – 1) Y_{t-1} – (á_{1} – 1) Y_{t-2} + (á_{1} + á_{2} – 1) Y_{t-2} – (á_{1} + á_{2} – 1) Y_{t-3} + e_{t} + â_{1} e_{t-1} + â_{2} e_{t-2}
_{}W_{t} = (
á_{1} – 1) W_{t-1} + (á_{1} + á_{2} – 1) W_{t-2} + e_{t} + â_{1} e_{t-1} + â_{2} e_{t-2}
_{}Jacob:
What is the procedure for this transformation?
Rachel:
The sum of the coefficients for the Y terms are equal on both sides of the equation.
The left side has a coefficient of 1. The right side has coefficients of
á_{1} + á_{2} + (1 – á_{1} – á_{2}) = 1.
Part B:
The time series is an ARIMA(2,1,2) process.
d = 1: we took one difference ( Y_{t}).
p = 2: we use W_{t-1} and W_{t-2}.
q = 2: we use e_{t-1} and e_{t-2}.
Part C:
The ARMA coefficients are
ö
_{1} = (á_{1} – 1)
ö
_{2} = (á_{1} + á_{2} – 1)
è
_{1} = –â_{1}
è
_{2} = –â_{2}
Illustration:
A time series is Y_{t} = 1.4Y_{t-1} + 0.1Y_{t-2} – 0.5Y_{t-3} + e_{t} + 0.3e_{t-1} + 0.2e_{t-2}
_{}Write the time series in terms of W_{t} ( Y_{t}).
What are the coefficients of this ARIMA process?
Part A:
Rewrite the ARIMA process as
Y_{t} – Y_{t-1} = 0.4Y_{t-1} + 0.1Y_{t-2} – 0.5Y_{t-3} + e_{t} + 0.3e_{t-1} + 0.2e_{t-2}
_{}= 0.4Y_{t-1} – 0.4Y_{t-2} + 0.4Y_{t-2} + 0.1Y_{t-2} – 0.5Y_{t-3} + e_{t} + 0.3e_{t-1} + 0.2e_{t-2}
_{}= 0.4Y_{t-1} – 0.4Y_{t-2} + 0.5Y_{t-2} – 0.5Y_{t-3} + e_{t} + 0.3e_{t-1} + 0.2e_{t-2}
_{} W_{t} = 0.4 W_{t-1} + 0.5W_{t-2} + e_{t} + 0.3e_{t-1} + 0.2e_{t-2}
_{}Part B:
The ARIMA coefficients are
ö
_{1} = 0.4
ö
_{2} = 0.5
è
_{1} = –0.3
è
_{2} = –0.2
** Exercise 9.5: ARIMA Process
A time series is Y_{t} =
è_{0} + á_{1} × Y_{t-1} + á_{2} × Y_{t-2} + á_{3} × Y_{t-3} + â_{1} × e_{t} + â_{2} × e_{t-1} + â_{3} × e_{t-2}
_{}Write the time series in terms of W_{t} ( Y_{t}).
What is the ARIMA process followed by this time series?
Part A:
Rewrite the ARIMA process as
Y_{t} – Y_{t-1} =
è_{0} + (á_{1} – 1) × Y_{t-1} + á_{2} × Y_{t-2} + á_{3} × Y_{t-3} + â_{1} × e_{t} + â_{2} × e_{t-1} + â_{3} × e_{t-2}
_{}Y_{t}–Y_{t-1} =
è_{0} + [(á_{1} – 1) × Y_{t-1} – (á_{1} – 1) × Y_{t-2}] + [(á_{1} – 1) × Y_{t-2} + á_{2} × Y_{t-2} + á_{3} × Y_{t-3}] + â_{1} × e_{t} + â_{2} × e_{t-1} + â_{3} ×e_{t-2}
_{}The coefficient of Y_{t-1} – Y_{t-2} is (
á_{1} – 1).
For this to be an ARIMA(2,1,2) process, we must have
(
á_{1} – 1) + á_{2} = –á_{3}
á
_{1} + á_{2} + á_{3} = 1
Jacob:
What about the
â coefficients?
Rachel:
Any
â coefficients are fine.
If
â_{1} = 1, then ö_{1} = –â_{2} and ö_{2} = –â_{3}.
If
â_{1} 1, then ö_{1} = –â_{2} / â_{1} and ö_{2} = –â_{3} / â_{1}.
** Exercise 9.6: Non-Stationary Series
A time series Y_{t} = 2 Y_{t-1} +
å_{t} has ó^{2}_{å} = 3.
Y_{t} = 0 for t < 1.
What is the variance of Y_{t} for t = 1?
What is the variance of Y_{t} for t = 2?
What is the variance of Y_{t} for t = 3?
Is this time series stationary?
Part A:
Y_{1} = 2 Y_{0} +
å_{1} = å_{1}, so the variance of Y_{1} = the variance of å_{t} = ó^{2}_{å} = 3.
Part B:
Y_{2} = 2 Y_{1} +
å_{2} = 4 Y_{0} + 2 å_{1} + å_{3} = å_{1}, so the variance of Y_{2} = 2^{2} × ó^{2}_{å} + ó^{2}_{å} = 15.
Part C:
The variance of Y_{t} is
ó^{2}_{å} + 2^{2} ó^{2}_{å} + (2^{2})^{2} ó^{2}_{å} + … + (2^{2})^{t-1}
=
ó^{2}_{å} × (2^{2 × t} – 1) / (2^{2} – 1)
= × (4^{t} – 1) ×
ó^{2}_{å}, which is equivalent to equation 5.1.4 on page 89.
The variance of Y_{3} is × (64 – 1) × 3 = 63.
Part D:
A stationary time series has the same mean and variance for all values of t. The variance of this time series depends on t.
Jacob:
What if the exercise did not say that Y_{t} = 0 for t < 1. What the time series be stationary?
Rachel:
A stationary time series need havd no beginning. It is in a stochastic equilibrium: the mean and variance are the same in all periods. If this time series has no beginning, its variance is infinite and its has no mean.
Jacob:
Why does it have no mean? The mean of
å is zero, so isn’t the mean of Y_{t} also zero?
Rachel:
If Y_{j} = k, the expected value of Y_{j+1} is 2 × k, and the expected value of Y_{j+1} is 2 × 2 × k. if k is zero, these are all zero. But Y_{j} has infinite variance, so k could be anything.
** Exercise 9.7: Combining error terms
Suppose Y_{t} = M_{t} + e_{t} and M_{t} = M_{t-1} +
å_{t}
Write Y_{t} as a function of M_{t-1} and error terms.
What type of time series is M_{t}?
What type of time series is Y_{t}?
What is Y_{t} (the first difference of Y_{t})?
What is the variance of Y_{t}?
What is the variance of Y_{t}?
What is the covariance of Y_{t} and Y_{t-1}?
What is
ñ_{1}, the autocorrelation of Y_{t} and Y_{t-1}?
Part A:
Y_{t} = M_{t} + e_{t} = M_{t-1} + e_{t} +
å_{t}
Part B:
M_{t} is a random walk.
Part C:
Y_{t} = M_{t-1} + e_{t} +
å_{t} = Y_{t-1} + å_{t} + e_{t} – e_{t-1}. This is a random walk with a more complex error term.
Part D:
Y_{t} = Y_{t} – Y_{t-1} = M_{t-1} + e_{t} +
å_{t} – ( M_{t-1} + e_{t-1}) = å_{t} + e_{t} – e_{t-1}
_{}Part E:
If the random walk has no beginning, the variance is infinite, so it does not exist. If the random walk has a beginning, the variance depends on the period.
Part F:
The variance of Y_{t} = var(
å_{t} + e_{t} – e_{t-1} ). The three random variables are independent, so the variance = 2ó^{2}_{e} + ó^{2}_{å}.
Part G:
The covariance of Y_{t} and Y_{t-1} is covariance (
å_{t} + e_{t} – e_{t-1} , å_{t-1} + e_{t-1} – e_{t-2} ) = –ó^{2}_{e}.
Part H:
The autocorrelation of Y_{t} and Y_{t-1} (
ñ_{1}) is –ó^{2}_{e} / 2ó^{2}_{e} + ó^{2}_{å} = –1 / (2 + ó^{2}_{å} / ó^{2}_{e} ). This is equation 5.1.10 on page 90.
**
Exercise 9.8: IMA(1,1) process
Each of the following time series is an IMA(1,1) process. What is the value of
è for each time series?
Y_{t} = Y_{t-1} + e_{t} – 0.4e_{t-1}
_{}Y_{t} = Y_{t-1} – e_{t} – 0.4e_{t-1}
_{}Y_{t} = Y_{t-1} + 0.4e_{t} – 0.4e_{t-1}
_{}Y_{t} = Y_{t-1} – 0.4e_{t} – 0.4e_{t-1}
_{}Part A:
The first difference of an IMA(1,1) is an MA(1) process.
The first difference of this time series is e_{t} – 0.4e_{t-1}, which is an MA(1) process with
è = 0.4.
Part B:
The first difference of this time series is – e_{t} – 0.4e_{t-1}. Use a change of the error term
å_{t } = –å_{t}, which gives a first difference of + e_{t } + 0.4e_{t -1}, which is an MA(1) process with è = –0.4.
Part C:
The first difference of this time series is 0.4 e_{t} – 0.4e_{t-1}. Use a change of the error term
å_{t } = 2.5å_{t}, which gives a first difference of + e_{t } – e_{t -1}, which is an MA(1) process with è = 1.
Part D:
The first difference of this time series is –0.4 e_{t} – 0.4e_{t-1}. Use a change of the error term
å_{t } = –2.5å_{t}, which gives a first difference of + e_{t } + e_{t -1}, which is an MA(1) process with è = –1.
(Cryer and Chan Page 93)
** Exercise 9.9: ARI(1,1) process
The time series Y_{t} =
è_{0} + á Y_{t-1} + â Y_{t-2} + e_{t} is an ARI(1,1) process.
Write the time series in terms of W_{t} ( Y_{t}).
What is the relation of
á and â?
What is the value of
ö for this ARI(1,1) process?
Part A:
W_{t} = Y_{t} = Y_{t} – Y_{t-1} =
è_{0} + (á – 1) × Y_{t-1} + â Y_{t-2} + e_{t}
Part B:
If
á – 1 = –â, we can write the time series as Y_{t} – Y_{t-1} = è_{0} + (–â) × (Y_{t-1} – Y_{t-2}) + e_{t}
Part C:
ö = –â = á – 1
** Exercise 9.10: Time series process
A time series is Y_{t} =
è_{0} + 1.75 Y_{t-1} – 0.75Y_{t-2} + e_{t} is an ARI(1,1) process.
Write the time series in terms of W_{t} ( Y_{t}).
What is the value of
ö_{1} for this ARI(1,1) process?
What is the value of
ö_{2} for this ARI(1,1) process?
Part A:
W_{t} = Y_{t} = Y_{t} – Y_{t-1} =
è_{0} + (á – 1) × Y_{t-1} + â Y_{t-2} + e_{t}
Part B:
If
á – 1 = –â, we can write the time series as Y_{t} – Y_{t-1} = è_{0} + (–â) × (Y_{t-1} – Y_{t-2}) + e_{t}
Part C:
ö = – â
** Exercise 9.11: IMA(1,1) process
An IMA(1,1) process is Y_{t} = Y_{t-1} +
å_{t} – 0.4 å_{t-1}, with Y_{t} = 0 for t < 1.
Write Y_{t} as
â_{t} × å_{t} + â_{t-1} × å_{t-1} + … + â_{1} × å_{1} + â_{0} × å_{0}
What is
â_{t}?
What is
â_{t-1}?
What is
â_{1}?
What is
â_{0}?
Part A:
Expand the time series period by period:
Y_{t} = Y_{t-1} +
å_{t} – 0.4 å_{t-1}
_{}Y_{t-1} = Y_{t-2} +
å_{t-1} – 0.4 å_{t-2}
_{}Y_{t-2} = Y_{t-3} +
å_{t-2} – 0.4 å_{t-3}
_{}The expanded time series is
Y_{t} =
å_{t} – 0.4 å_{t-1} + å_{t-1} – 0.4 å_{t-2} + å_{t-2} – 0.4 å_{t-3} + … + Y_{0} + å_{1} – 0.4 å_{0}
_{}Y_{0} = 0, so we have finished expanding. We group error terms with the same subscript to get the
â_{t} values.
Part B:
â_{t} is the coefficient of the å_{t} term = 1.
Part C:
â_{t-1} is the coefficient of the å_{t-1} term = (1 – 0.4).
Part D:
â_{1} is the coefficient of the å_{1} term = (1 – 0.4).
Part E:
â_{0} is the coefficient of the å_{0} term = –0.4.