Downloaded 3,736 times
![Autocorrelation Function (ACF)
• Autocorrelation is a correlation coefficient. However, instead
of correlation between two different variables, the correlation
is between two values of the same variable at times Xi and
Xi+k.
• Correlation with lag-1, lag2, lag3 etc.,
• The ACF represents the degree of persistence over respective
lags of a variable.
ρk = γk / γ0 = covariance at lag k/ variance
ρk = E[(yt – μ)(yt-k – μ)]2
E[(yt – μ)2]
ACF (0) = 1, ACF (k) = ACF (-k)
DataAnalysisCourse
VenkatReddy
27](https://image.slidesharecdn.com/6-130914140240-phpapp01/85/ARIMA-27-320.jpg)
![PACF Graph
DataAnalysisCourse
VenkatReddy
30
-0.50
0.000.501.00
0 10 20 30 40
Lag
95% Confidence bands [se = 1/sqrt(n)]](https://image.slidesharecdn.com/6-130914140240-phpapp01/85/ARIMA-30-320.jpg)
![Autocorrelation Function (ACF)
• Autocorrelation is a correlation coefficient. However, instead
of correlation between two different variables, the correlation
is between two values of the same variable at times Xi and
Xi+k.
• Correlation with lag-1, lag2, lag3 etc.,
• The ACF represents the degree of persistence over respective
lags of a variable.
ρk = γk / γ0 = covariance at lag k/ variance
ρk = E[(yt – μ)(yt-k – μ)]2
E[(yt – μ)2]
ACF (0) = 1, ACF (k) = ACF (-k)
DataAnalysisCourse
VenkatReddy
27](https://image.slidesharecdn.com/6-130914140240-phpapp01/75/ARIMA-27-2048.jpg)
![PACF Graph
DataAnalysisCourse
VenkatReddy
30
-0.50
0.000.501.00
0 10 20 30 40
Lag
95% Confidence bands [se = 1/sqrt(n)]](https://image.slidesharecdn.com/6-130914140240-phpapp01/75/ARIMA-30-2048.jpg)
Uploaded byVenkata Reddy Konasani
ARIMA
The document discusses time series analysis and forecasting using ARIMA models, focusing on the methodologies for model identification, estimation, and forecasting. It emphasizes the importance of stationarity in time series data and the steps involved in the Box-Jenkins approach, including differencing, model identification through ACF and PACF analysis, and diagnostic checking for model adequacy. Key components such as autoregressive and moving average processes are explained, along with practical guidelines for implementing ARIMA modeling.
More Related Content
ARIMA
- 1. Data Analysis Course TimeSeries Analysis & Forecasting Venkat Reddy
- 2. Contents • ARIMA • Stationarity •AR process • MA process • Main steps in ARIMA • Forecasting using ARIMA model • Goodness of fit DataAnalysisCourse VenkatReddy 2
- 3. Drawbacks of theuse of traditional models • There is no systematic approach for the identification and selection of an appropriate model, and therefore, the identification process is mainly trial-and-error • There is difficulty in verifying the validity of the model • Most traditional methods were developed from intuitive and practical considerations rather than from a statistical foundation DataAnalysisCourse VenkatReddy 3
- 4. ARIMA Models • AutoregressiveIntegrated Moving-average • A “stochastic” modeling approach that can be used to calculate the probability of a future value lying between two specified limits DataAnalysisCourse VenkatReddy 4
- 5. AR & MAModels • Autoregressive AR process: • Series current values depend on its own previous values • AR(p) - Current values depend on its own p-previous values • P is the order of AR process • Moving average MA process: • The current deviation from mean depends on previous deviations • MA(q) - The current deviation from mean depends on q- previous deviations • q is the order of MA process • Autoregressive Moving average ARMA process DataAnalysisCourse VenkatReddy 5
- 6. AR Process DataAnalysisCourse VenkatReddy 6AR(1) yt= a1* yt-1 AR(2) yt = a1* yt-1 +a2* yt-2 AR(3) yt = a1* yt-1 + a2* yt-2 +a3* yt-2
- 7. MA Process DataAnalysisCourse VenkatReddy 7 MA(1) εt= b1*εt-1 MA(2) εt = b1*εt-1 + b2*εt-2 MA(3) εt = b1*εt-1 + b2*εt-2+ b3*εt-3
- 8. ARIMA Models • Autoregressive(AR) process: • Series current values depend on its own previous values • Moving average (MA) process: • The current deviation from mean depends on previous deviations • Autoregressive Moving average (ARMA) process • Autoregressive Integrated Moving average (ARIMA)process. • ARIMA is also known as Box-Jenkins approach. It is popular because of its generality; • It can handle any series, with or without seasonal elements, and it has well- documented computer programs DataAnalysisCourse VenkatReddy 8
- 9. ARIMA Model → ARfilter → Integration filter → MA filter → εt (long term) (stochastic trend) (short term) (white noise error) ARIMA (2,0,1) yt = a1yt-1 + a2yt-2 + b1εt-1 ARIMA (3,0,1) yt = a1yt-1 + a2yt-2 + a3yt-3 + b1εt-1 ARIMA (1,1,0) Δyt = a1 Δyt-1 + εt , where Δyt = yt - yt-1 ARIMA (2,1,0) Δyt = a1 Δyt-1 + a2Δ yt-2 + εt where Δyt = yt - yt-1 To build a time series model issuing ARIMA, we need to study the time series and identify p,d,q DataAnalysisCourse VenkatReddy 9
- 10. ARIMA equations • ARIMA(1,0,0) •yt = a1yt-1 + εt • ARIMA(2,0,0) • yt = a1yt-1 + a2yt-2 + εt • ARIMA (2,1,1) • Δyt = a1 Δyt-1 + a2Δ yt-2 + b1εt-1 where Δyt = yt - yt-1 DataAnalysisCourse VenkatReddy 10
- 11. Overall Time seriesAnalysis & Forecasting Process • Prepare the data for model building- Make it stationary • Identify the model type • Estimate the parameters • Forecast the future values DataAnalysisCourse VenkatReddy 11
- 12. ARIMA (p,d,q) modeling Tobuild a time series model issuing ARIMA, we need to study the time series and identify p,d,q • Ensuring Stationarity • Determine the appropriate values of d • Identification: • Determine the appropriate values of p & q using the ACF, PACF, and unit root tests • p is the AR order, d is the integration order, q is the MA order • Estimation : • Estimate an ARIMA model using values of p, d, & q you think are appropriate. • Diagnostic checking: • Check residuals of estimated ARIMA model(s) to see if they are white noise; pick best model with well behaved residuals. • Forecasting: • Produce out of sample forecasts or set aside last few data points for in-sample forecasting. DataAnalysisCourse VenkatReddy 12
- 13. The Box-Jenkins Approach DataAnalysisCourse VenkatReddy 13 1.Differencingthe series to achieve stationary 2.Identify the model 3.Estimate the parameters of the model Diagnostic checking. Is the model adequate? No Yes4. Use Model for forecasting
- 14.
- 15. Some non stationaryseries DataAnalysisCourse VenkatReddy 15 1 2 3 4
- 16. Stationarity • In orderto model a time series with the Box-Jenkins approach, the series has to be stationary • In practical terms, the series is stationary if tends to wonder more or less uniformly about some fixed level • In statistical terms, a stationary process is assumed to be in a particular state of statistical equilibrium, i.e., p(xt) is the same for all t • In particular, if zt is a stationary process, then the first difference zt = zt - zt-1and higher differences d zt are stationary DataAnalysisCourse VenkatReddy 16
- 17. Testing Stationarity • Dickey-Fullertest • P value has to be less than 0.05 or 5% • If p value is greater than 0.05 or 5%, you accept the null hypothesis, you conclude that the time series has a unit root. • In that case, you should first difference the series before proceeding with analysis. • What DF test ? • Imagine a series where a fraction of the current value is depending on a fraction of previous value of the series. • DF builds a regression line between fraction of the current value Δyt and fraction of previous value δyt-1 • The usual t-statistic is not valid, thus D-F developed appropriate critical values. If P value of DF test is <5% then the series is stationary DataAnalysisCourse VenkatReddy 17
- 18. Demo: Testing Stationarity •Sales_1 data DataAnalysisCourse VenkatReddy 18 Stochastic trend: Inexplicable changes in direction
- 19. Demo: Testing Stationarity DataAnalysisCourse VenkatReddy 19 AugmentedDickey-Fuller Unit Root Tests Type Lags Rho Pr < Rho Tau Pr < Tau F Pr > F Zero Mean 0 0.3251 0.7547 0.74 0.8695 1 0.3768 0.7678 1.26 0.9435 2 0.3262 0.7539 1.05 0.9180 Single Mean 0 -6.9175 0.2432 -1.77 0.3858 2.05 0.5618 1 -3.5970 0.5662 -1.06 0.7163 1.52 0.6913 2 -3.7030 0.5522 -0.88 0.7783 1.02 0.8116 Trend 0 -11.8936 0.2428 -2.50 0.3250 3.16 0.5624 1 -7.1620 0.6017 -1.60 0.7658 1.34 0.9063 2 -9.0903 0.4290 -1.53 0.7920 1.35 0.9041
- 20. Achieving Stationarity • Differencing: Transformation of the series to a new time series where the values are the differences between consecutive values • Procedure may be applied consecutively more than once, giving rise to the "first differences", "second differences", etc. • Regular differencing (RD) (1st order) xt = xt – xt-1 (2nd order) 2xt = ( xt - xt-1 )=xt – 2xt-1 + xt-2 • It is unlikely that more than two regular differencing would ever be needed • Sometimes regular differencing by itself is not sufficient and prior transformation is also needed DataAnalysisCourse VenkatReddy 20
- 21.
- 22. Demo: Achieving Stationarity DataAnalysisCourse VenkatReddy 22 datalagsales_1; set sales_1; sales1=sales-lag1(sales); run; Augmented Dickey-Fuller Unit Root Tests Type Lags Rho Pr < Rho Tau Pr < Tau F Pr > F Zero Mean 0 -37.7155 <.0001 -7.46 <.0001 1 -32.4406 <.0001 -3.93 0.0003 2 -19.3900 0.0006 -2.38 0.0191 Single Mean 0 -38.9718 <.0001 -7.71 0.0002 29.70 0.0010 1 -37.3049 <.0001 -4.10 0.0036 8.43 0.0010 2 -25.6253 0.0002 -2.63 0.0992 3.50 0.2081 Trend 0 -39.0703 <.0001 -7.58 0.0001 28.72 0.0010 1 -37.9046 <.0001 -4.08 0.0180 8.35 0.0163 2 -25.7179 0.0023 -2.59 0.2875 3.37 0.5234
- 23.
- 24. Achieving Stationarity-Othermethods • Isthe trend stochastic or deterministic? • If stochastic (inexplicable changes in direction): use differencing • If deterministic(plausible physical explanation for a trend or seasonal cycle) : use regression • Check if there is variance that changes with time • YES : make variance constant with log or square root transformation • Remove the trend in mean with: • 1st/2nd order differencing • Smoothing and differencing (seasonality) • If there is seasonality in the data: • Moving average and differencing • Smoothing DataAnalysisCourse VenkatReddy 24
- 25.
- 26. Identification of ordersp and q • Identification starts with d • ARIMA(p,d,q) • What is Integration here? • First we need to make the time series stationary • We need to learn about ACF & PACF to identify p,q • Once we are working with a stationary time series, we can examine the ACF and PACF to help identify the proper number of lagged y (AR) terms and ε (MA) terms. DataAnalysisCourse VenkatReddy 26
- 27. Autocorrelation Function (ACF) •Autocorrelation is a correlation coefficient. However, instead of correlation between two different variables, the correlation is between two values of the same variable at times Xi and Xi+k. • Correlation with lag-1, lag2, lag3 etc., • The ACF represents the degree of persistence over respective lags of a variable. ρk = γk / γ0 = covariance at lag k/ variance ρk = E[(yt – μ)(yt-k – μ)]2 E[(yt – μ)2] ACF (0) = 1, ACF (k) = ACF (-k) DataAnalysisCourse VenkatReddy 27
- 28. ACF Graph DataAnalysisCourse VenkatReddy 28 -0.50 0.000.501.00 0 1020 30 40 Lag Bartlett's formula for MA(q) 95% confidence bands
- 29. Partial Autocorrelation Function(PACF) • The exclusive correlation coefficient • Partial regression coefficient - The lag k partial autocorrelation is the partial regression coefficient, θkk in the kth order auto regression • In general, the "partial" correlation between two variables is the amount of correlation between them which is not explained by their mutual correlations with a specified set of other variables. • For example, if we are regressing a variable Y on other variables X1, X2, and X3, the partial correlation between Y and X3 is the amount of correlation between Y and X3 that is not explained by their common correlations with X1 and X2. • yt = θk1yt-1 + θk2yt-2 + …+ θkkyt-k + εt • Partial correlation measures the degree of association between two random variables, with the effect of a set of controlling random variables removed. DataAnalysisCourse VenkatReddy 29
- 30. PACF Graph DataAnalysisCourse VenkatReddy 30 -0.50 0.000.501.00 0 1020 30 40 Lag 95% Confidence bands [se = 1/sqrt(n)]
- 31. Identificationof AR Processes&its order -p • For AR models, the ACF will dampen exponentially • The PACF will identify the order of the AR model: • The AR(1) model (yt = a1yt-1 + εt) would have one significant spike at lag 1 on the PACF. • The AR(3) model (yt = a1yt-1+a2yt-2+a3yt-3+εt) would have significant spikes on the PACF at lags 1, 2, & 3. DataAnalysisCourse VenkatReddy 31
- 32.
- 33. AR(1) model DataAnalysisCourse VenkatReddy 33 yt =0.77yt-1 + εt
- 34. AR(1) model DataAnalysisCourse VenkatReddy 34 yt =0.95yt-1 + εt
- 35. AR(2) model DataAnalysisCourse VenkatReddy 35 yt =0.44yt-1 + 0.4yt-2 + εt
- 36. AR(2) model DataAnalysisCourse VenkatReddy 36 yt =0.5yt-1 + 0.2yt-2 + εt
- 37. AR(3) model DataAnalysisCourse VenkatReddy 37 yt =0.3yt-1 + 0.3yt-2 + 0.1yt-3 +εt
- 38.
- 39. Identificationof MA Processes&its order - q • Recall that a MA(q) can be represented as an AR(∞), thus we expect the opposite patterns for MA processes. • The PACF will dampen exponentially. • The ACF will be used to identify the order of the MA process. • MA(1) (yt = εt + b1 εt-1) has one significant spike in the ACF at lag 1. • MA (3) (yt = εt + b1 εt-1 + b2 εt-2 + b3 εt-3) has three significant spikes in the ACF at lags 1, 2, & 3. DataAnalysisCourse VenkatReddy 39
- 40.
- 41.
- 42.
- 43.
- 44.
- 45. MA(3) DataAnalysisCourse VenkatReddy 45 yt = 0.8εt-1+ 0.9εt-2 + 0.6εt-3
- 46.
- 47.
- 48.
- 49. ARIMA(2,1) DataAnalysisCourse VenkatReddy 49 yt = 0.4yt-1+ 0.3yt-2 + 0.9εt-1
- 50. ARMA(1,2) DataAnalysisCourse VenkatReddy 50 yt = 0.8yt-1+ 0.4εt-1 + 0.55εt-2
- 51.
- 52. Demo1: Identification ofthe model • ACF is dampening, PCF graph cuts off. - Perfect example of an AR process DataAnalysisCourse VenkatReddy 52 proc arima data= chem_readings plots=all; identify var=reading scan esacf center ; run;
- 53. Demo: Identification ofthe model PACF cuts off after lag 2 1. d = 0, p =2, q= 0 DataAnalysisCourse VenkatReddy 53 SAS ARMA(p+d,q) Tentative Order Selection Tests SCAN ESACF p+d q p+d q 2 0 2 3 1 5 4 4 5 3 yt = a1yt-1 + a2yt-2 + εt
- 54. LAB: Identification ofmodel • Download web views data • Use sgplot to create a trend chart • What does ACF & PACF graphs say? • Identify the model using below table • Write the model equation DataAnalysisCourse VenkatReddy 54
- 55.
- 56. Parameter Estimate • Wealready know the model equation. AR(1,0,0) or AR(2,1,0) or ARIMA(2,1,1) • We need to estimate the coefficients using Least squares. Minimizing the sum of squares of deviations DataAnalysisCourse VenkatReddy 56
- 57. Demo1: Parameter Estimation •Chemical reading data DataAnalysisCourse VenkatReddy 57 proc arima data=chem_readings; identify var=reading scan esacf center; estimate p=2 q=0 noint method=ml; run; yt = 0. 424yt-1 + 0.2532yt-2 + εt Maximum Likelihood Estimation Parameter Estimate Standard Error t Value Approx Pr > |t| Lag AR1,1 0.42444 0.06928 6.13 <.0001 1 AR1,2 0.25315 0.06928 3.65 0.0003 2
- 58. Lab: Parameter Estimation •Estimate the parameters for webview data DataAnalysisCourse VenkatReddy 58
- 59.
- 60. Forecasting • Now themodel is ready • We simply need to use this model for forecasting DataAnalysisCourse VenkatReddy 60 proc arima data=chem_readings; identify var=reading scan esacf center; estimate p=2 q=0 noint method=ml; forecast lead=4 ; run; Forecasts for variable Reading Obs Forecast Std Error 95% Confidence Limits 198 17.2405 0.3178 16.6178 17.8633 199 17.2235 0.3452 16.5469 17.9000 200 17.1759 0.3716 16.4475 17.9043 201 17.1514 0.3830 16.4007 17.9020
- 61. LAB: Forecasting usingARIMA • Forecast the number of sunspots for next three hours DataAnalysisCourse VenkatReddy 61
- 62. Validation: How goodis my model? • Does our model really give an adequate description of the data • Two criteria to check the goodness of fit • Akaike information criterion (AIC) • Schwartz Bayesiancriterion (SBC)/Bayesian information criterion (BIC). • These two measures are useful in comparing two models. • The smaller the AIC & SBC the better the model DataAnalysisCourse VenkatReddy 62
- 63. Goodness of fit •Remember… Residual analysis and Mean deviation, Mean Absolute Deviation and Root Mean Square errors? • Four common techniques are the: • Mean absolute deviation, • Mean absolute percent error • Mean square error, • Root mean square error. DataAnalysisCourse VenkatReddy 63 n i ii n1 YˆY =MAD n i ii n1 2 YˆY =MSE MSERMSE n i i ii n 1 Y YˆY100 =MAPE
- 64. Lab: Overall Stepson sunspot example • Import the time series data • Prepare the data for model building- Make it stationary • Identify the model type • Estimate the parameters • Forecast the future values DataAnalysisCourse VenkatReddy 64
- 65.