Residual Analysis

Final Prediction Error

For scalar autoregressive models it can be shown that the estimated mean square prediction error of the process is

FPE=N+n

N^,n ^{^2} (5.9)

where ^²is the maximum likelihood estimate of the variance of the driving white noise sequence, see (Brockwell & Davis, 1987). The FPEcriterion is asymptotically equivalent to the AIC criterion and thus determines the same optimalnfor large N.

the residuals. Instead of checking the individual values of the autocorrela-tion funcautocorrela-tion, it is possible to pool the informaautocorrela-tion into a single statistic, and perform aPortmanteau Test, see e.g. (Ljung & Box, 1978). In addi-tion to the tests based on the sample autocorrelaaddi-tion funcaddi-tion there are a number of tests for checking the hypothesis of the residuals being an i.i.d.

sequence, such as e.g. Test Based on Turning Points, Dierence-Sign TestandRank Test, see (Brockwell & Davis, 1987).

5.2.2 Kolmogorov-Smirnov test

The Kolmogorov - Smirnov test is designed for testing hypotheses con-cerning equality between e.g. an assumed, F0, and an empirical distribu-tion,Fn. The test statistic is

Dn= sup_x ^jFn(x)^,F0(x)^j

which has a known distribution under F0. Fnis the empirical distribution ofx1;;xn. A test on level is therefore given by the critical areaC=

f(x1;;xn)^jDn> c^g, wherecis evaluated from Pr^fDn> c^jF=F0^g=, see (Kendall & Stuart, 1979). The test has a number of advantages, com-pared to other methods. It is exact and easy to apply, since the probability fornovalue exciting the condence limits is computed.

The information basis for the grey box modelling may be formulated as prior distributions of e.g. data or disturbances. The modelling results in a posteriori distributions for data as well as disturbances, which can be tested against the prior belief, cf. (Holst et al., 1992).

Tests for distribution may also be used as a means for checking the white-ness of the residuals as in the black box case. The cumulated periodogram

for the residual sequence has the same properties as a distribution func-tion, and is tested against the distribution of white noise, cf. (Brockwell

& Davis, 1987). There is no need for (troublesome) smoothing in the fre-quency domain, when using this test.

5.2.3 Cross Spectra

If the residuals are white noise, then the model is in good agreement with the true system. Another interesting question is whether the residuals are independent of inputs. If not, then there is more information contained in the output that originates from the input than explained by the current model. Independence may be tested using the sample cross covariance function, see e.g. (Box & Jenkins, 1976).

The cross spectrum, given as the Fourier transform of the cross covariance function between residuals and inputs is primarily used as a diagnostic tool, to indicate how further improvement is possible.

A useful quantity derived from the cross spectrum, ,_u(k), is the coherency spectrum

_2u(!) = ^j,u(!)^j2

,(!),_u(!) (5.10)

where ,and ,uare the auto covariance functions of^ft^gand^fut^g respec-tively. _2u(!)²[0;1] can be interpreted as a non dimensional measure of the correlation between two time series at a certain frequency.

An alternative way of inspecting the correlation between residuals and mul-tiple inputs is by the mulmul-tiple coherency spectrum. Consider the sequence of residuals^ft^gandqdierent inputs^fu1;t^g;^fu2;t^g;;^fuq;t^gwith their

means subtracted. It is then possible to separate the variation of^ft^gthe following way

t=^X

k h;1u1;t^,k++^X

k h;quq;t^,k+Zt (5.11) In other words, a linear model for the residuals based on the inputs is tted, and a new set of residuals ^fZt^gobtained. The spectrum of the noise process ^fZt^gis given by

,z(!) = ,(!)^,Xq

i=1Hi,ui(!) (5.12)

which can also be written as

,_z(!) = ,(!)[1^,_212q(!)] (5.13) where _212q is the squared multiple coherency spectrum of the output process and theqinput processes. This quantity measures the proportion of the residual spectrum which can be predicted linearly from the inputs at the dierent frequencies. It is a useful diagnostic tool for model improve-ment and assessimprove-ment of the partial prior knowledge.

5.2.4 Bispectra

The bispectrum is the frequency domain representation of the third order moment of the stochastic process ^fxt^g.

M3(u;v) = E[(xt^,)(xt+u^,)(xt+v^,)] (5.14) It is the natural extension to the usual second order representation of the spectrum, and is useful when dealing with non Gaussian/non linear

pro-cesses. The bispectrum of the stationary process^fxt^gis dened as the two dimensional Fourier transform of the third moment

,3(!1;!2) =^X

u;vM3(u;v)exp(^,i2(u!1+v!2)) (5.15) where ^,!1;!2 < . By inserting an estimate of (5.14) in (5.15) we obtain an estimate of the spectrum which is called the raw spectrum. This estimate is central but not consistent, and we have to smooth the estimate with a lag kernel, in order to get a consistent estimate.

For a linear and Gaussian process we have ,3(!1;!2) = 0in the whole frequency plane. There is an approximative test, cf. (Subba Rao & Gabr, 1984), for the hypothesis

H0 : ⁸!1;!2: ,3(!1;!2) =0 H1 : ⁹!1;!2: ,3(!1;!2)⁶=0

IfH0 is accepted, then the stochastic process^fxt^gis considered linear and Gaussian. If on the other hand H0 is rejected, the process can be either non Gaussian and/or nonlinear. A second step is proposed in Subba Rao

& Gabr (1984) to test the hypothesis:

H0 : xt is linear andM3 =0 H1 : all alternatives

where M3 is the third order moment given in (5.14). It should be noted that the tests above is just one example of a nonlinearity test. Other possibilities are e.g. the Lagrange Multiplier test discussed previously, see also (Priestley, 1988; Tong, 1990).

The bispectrum and the tests above are useful to check the assumptions on which the methods are build. Furthermore if the stochastic process is

nonlinear or non Gaussian, the bispectrum contains valuable information about the process, which can be used for model improvement. In general it is necessary to estimate and analyze higher order moments or spectra in order to be able to establish the need for nonlinear or non-Gaussian components in the model, cf. (Tong, 1990).