SEE5211 Chapter10 2017 - P
SEE5211 Chapter10 2017 - P
SEE5211 Chapter10 2017 - P
(SEE5211/SEE8212)
(SEE5211/SEE8212)
periodical behavior
Linear fit
• Stationary signals:
• Statistics don’t change with time
• Frequency contents don’t change with time
• Information doesn’t change with time
• Non-stationary signals:
• Statistics change with time
• Frequencies change with time
• Information quantity increases
Summer
For example, the likelihood of tomorrow being rainy is greater if today is rainy than if
today is dry. Geophysical time series are frequently auto correlated because of inertia or
carryover processes in the physical system.
Partial Autocorrelation Function (PACF)
• For a time series, the partial autocorrelation between X (t) and X(t-k) is
defined as the conditional correlation between X(t) and X(t-k)
conditional on X (t-(k-1)),…X(t-1), the set of observations that come
between the time points t and t-k
• The 1st order partial autocorrelation will be defined to equal the 1st order
autocorrelation
• The 2nd order (lag) partial autocorrelation is
Co var iance( xt , xt 2 xt 1 )
Variance( xt xt 1 )Variance( xt 2 xt 1 )
ACF and PACF (Partial Autocorrelation)
• ACF: Auto correlation refers to the correlation of a time series with its
own past and future values. Positive ACF might be considered a specific
form of persistence, a tendency for a system to remain in the same state
from one observation to the next.
• PACF:takes into consideration the correlation between a time series and
each of its intermediate lagged values.
Yt 0 1 yt 1 et
The second-order autoregression, AR(2):
Yt 0 1 yt 1 2 yt 2 et
xt t 1t 1
xt t 1t 1 2t 2
ACF(k=1,2)
If the analyst does not check for randomness, then the validity of many of the statistical
conclusions becomes suspect. The autocorrelation plot is an excellent way of checking for
such randomness.
Moderate positive autocorrelation
The plot starts with a moderately high autocorrelation at lag 1 (approximately 0.75) that
gradually decreases. The decreasing autocorrelation is generally linear, but with significant
noise. Such a pattern is the autocorrelation plot signature of "moderate autocorrelation", which
in turn provides moderate predictability if modeled properly.
Autocorrelation and Partial autocorrelation
The autocorrelation graph describes the correlation between all the pairs of points in the time series
for a given separation in time (lag). Autocorrelation and partial autocorrelation graphs can help you
determine whether the time series is stationary (meaning it has a fixed mean and standard deviation
over time) and what model might be appropriate to fit the time series.
Partial autocorrelation plots are a commonly used tool for model identification in
Box-Jenkins models.
• The partial autocorrelation at lag p is the autocorrelation between Xt and Xt−p that
is not accounted for by lags 1 through p−1.
• Specifically, partial autocorrelations are useful in identifying the order of an
autoregressive model. The partial autocorrelation of an AR(p) process is zero at lag
p+1 and greater. If the sample autocorrelation plot indicates that an AR model may
be appropriate, then the sample partial autocorrelation plot is examined to help
identify the order. We look for the point on the plot where the partial autocorrelations
essentially become zero.
• The approximate 95 % confidence interval for the partial autocorrelations are at
±2/√N
Partial Autocorrelation Plot
Yt 0 1 yt 1 2 yt 2 et AR(2)
Yt 0 1 yt 1 2 yt 2 3 yt 3 et AR(3)
This partial autocorrelation plot shows clear statistical significance for lags 1 and 2 (lag 0 is
always 1). The next few lags are at the borderline of statistical significance. If the
autocorrelation plot indicates that an AR model is appropriate, we could start our modeling
with an AR(2) model. We might compare this with an AR(3) model.
For ARMA models(Box-Jenkins model)
Z t ( xt xt 1 ) ( xt 1 xt 2 )
ARMA(p, q)
( xt ) 1 ( xt 1 ) ... p ( xt p ) t 1t 1... qt q
Box-Jenkins Analysis of Seasonal Data
Non-constant variance can be we remove trend in the series
removed by performing a by taking first differences.
natural log transformation.
To identify an appropriate
model, we plot the ACF of
the time series.
The autocorrelation plot has a 95% confidence band, which is constructed based on the assumption
that the process is a moving average process. The autocorrelation plot shows that the sample
autocorrelations are very strong and positive and decay very slowly. The autocorrelation plot
indicates that the process is non-stationary and suggests an ARMA model. The next step is to
difference the data.
The autocorrelation plot of the differenced The partial autocorrelation plot of the
data with a 95% confidence band shows that differenced data with 95% confidence bands
only the autocorrelation at lag 1 is shows that only the partial autocorrelations of
significant. The autocorrelation plot together the first and second lag are significant. This
with run sequence of the differenced data suggests an AR(2) model for the differenced
suggest that the differenced data are data.
stationary. Based on the autocorrelation plot,
an MA(1) model is suggested for the
differenced data.
Ljung-Box Q test
• Used to test whether or not observations over time are random and
independent
----Ho: the autocorrelations up to lag K are all 0
----Ha: the autocorrelations of one or more lags differ from 0.
k rj2
Qk n(n 2)
j 1 n j
2
Which is approximately x k
distribution
2
x k
If k=1, Q1 =9.08, p value =0.0026 < a =0.05
Reject H0, accept Ha, there is strong evidence that
lag-1 autocorrelation is non-zero.
Augmented Dickey-Fuller (ADF) Test
Test for a unit root with drift and deterministic time trend
• The difference between the three equations concerns the presence of the deterministic
elements α (a drift term) and βt (a linear time trend).
• The focus of testing is whether the coefficient γ equals to zero, what means that the original
process has a unit root;
• hence, the null hypothesis of γ = 0 (random walk process) is tested against the alternative
hypothesis γ < 0 of stationarity.
The ADF test ensures that the null hypothesis is accepted unless there is strong evidence
against it to reject in favor of the alternate stationarity hypothesis.
Time series-autocorrelation
• Open CO2.jmp
• Select Analyze > specialized modelling>time series
• Click OutputCO2 to Y, time series
• Red triangle– Spectral density
• Red Triangle-Autocorrelation, Partial autocorrelation
• Red triangle-decomposition-remove linear trend
The two surfaces are similar. The impact of X1 and X2 on the response Y can be visualized.
You can rotate the plot to view it from different angles. Marginal plots are another tool to
use to understand the impact of the factors on the response.
Temporal Variation of Visibility in Hong Kong
(a) The normalized local wavelet power spectrum of SCMR (1910–2001) using the
real-valued Mexican hat wavelet (derivate of a Gaussian; DOG m = 2). The thick
curve on either end indicates the edge effects.
(b) The normalized reconstructed time series of SCMR at period of 8yr, 16–32 yr,
32-64yr.
Wavelet Analysis
• Wavelets are functions that satisfy certain mathematical requirements and are used to
represent data or other functions
• Idea is not new--- Joseph Fourier--- 1800's
• Wavelet-- the scale we use to see data plays an important role
• Fourier transform FT non local -- very poor job on sharp spikes
• Wavelet analysis is becoming a common tool for analyzing localized variations of
power within a time series.
• By decomposing a time series into time–frequency space, one is able to determine
both the dominant modes of variability and how those modes vary in time.
• The wavelet transform has been used for numerous studies in geophysics
Sine wave
Wavelet db10
Fourier Analysis
Frequency analysis
Idea: Transforms time-based signals to frequency-based signals.
Drawback:
1.Location information is stored in phases and difficult to extract.
2. The Fourier transform is very sensitive to changes in the function.
Fourier Analysis
F (v) Frequency
e 2ivt cos(2vt ) i sin(2vt )
v [ HZ ]
2
Fourier Series
the sum of simple sinusoids of different frequencies.
2
1
ak
0
f ( x ) cos( kx ) dx
2
1
bk
0
f ( x ) sin( kx ) dx
Energy of a function f ( x)
2
1
energy
2
f ( x) dx
2 0
Any periodic function can be decomposed to a sum of sine and cosine waves, i.e.:
any periodic function f(x) can be represented by
From: http://www.bv-elbtal.de/html/was_ist_larm_.html
Vanishing Moments: if the average value of xkψ (x) is zero
(where ψ (x) is the wavelet function), for k = 0, 1, …, n then
the wavelet has n + 1 vanishing moments and polynomials of
degree n are suppressed by this wavelet.
There are infinite sets of Wavelets Transforms.
Different wavelet families: Different families provide different relationships between
how compact the basis function are localized in space and how smooth they are.
This decomposition can be done with a Fourier transform (or
Fourier series for periodic waveforms)
• Definition:
1 * t
CWT , s , s
xt dt
s
x x
s
Translation
(The location of Scale
the window)
Mother Wavelet
Scale
Low scale => Compressed wavelet => Rapidly changing details => High frequency.
High scale => Stretched wavelet => Slowly changing, coarse features => Low frequency.
Position
Steps to a Continuous Wavelet Transform
1.Take a wavelet and compare it to a section at the start of the original signal.
2. Calculate C, i.e., how closely correlated the wavelet is with this section of the signal.
3. Shift the wavelet to the right and repeat steps 1 and 2 until you’ve covered the whole
signal.
1 * t
CWTx , s x , s x t dt
s s
X T ,s t dt
1 t
,s t
s s
• CWT is the inner product of the signal and the basis function
,s t
Wavelet basis functions
-1
j0 2 2
Morlet (0 frequency) : e 4
e
2 m i m m! m1
Paul m order : DOG 1 i
2m !
DOG m devivative :
- 1m1 d m 2
e 2
1 d m
m
2
2nd derivative of a Gaussian
is the Marr or Mexican hat wavelet
Wavelet basis functions
Frequency domain
Time domain
Example: Run wavetest.m
%------------------------------------------------------ Computation
-2
-4
1880 1900 1920 1940 1960 1980 2000
Time (year)
2
Period (years)
16
32
64
1880 1900 1920 1940 1960 1980 2000 0 0.5 1 1.5 2 2.5 3
Time (year) Power (degC2)
0.6
0.4
0.2
0
1880 1900 1920 1940 1960 1980 2000
Time (year)
Project Aims (due time: week12)
• to develop and demonstrate students’ creativity and ability to carry out industrially-
related or research-type project work;
• to demonstrate application of mathematics, science, engineering, economics and
policy knowledge in practical situations to arrive at innovative solution; and
• to develop problem-solving skills, demonstrate teamwork, build self-confidence and
ability to make good oral presentations and report writing.
Group Project --- 20% (group presentation 10% and term paper 10%
(Individual Participation 2% ) students will first be divided into 10-12
small groups (4-6 students form a group) .
Each small group will conduct a forum on a topic of your choice. Your
group will select one type of datasets (such as Air pollutant concentration,
weather data, Power data, or others). Group members will work together
to prepare a 15-minute presentation and a term paper (1500 words + 4
figures) about data analysis , each project should first introduce the
environmental datasets or historical events and discuss the types of
datasets, especially focus on collecting, analyzing, and drawing
conclusions from data.
Group report (10%)
Group Presentation (10 mins) (10%)