Purpose
To go over the examples in Jonathan Cryer Chan book on time series , specifically Chapter 7 on estimation

Simulate ar(1) process

> setwd("C:/Cauldron/garage/R/soulcraft/Volatility/Learn/JonathanCryer_Chan/")
> x <- arima.sim(n = 200, list(ar = c(0.9)))

Use Conditional sum of squares method.

> fit <- arima(x, order = c(1, 0, 0), method = c("CSS"))
> print(fit)
Call:
arima(x = x, order = c(1, 0, 0), method = c("CSS"))


Coefficients:
         ar1  intercept
      0.8593    -0.5721
s.e.  0.0355     0.4637


sigma^2 estimated as 0.8486:  log likelihood = -267.37,  aic = NA

Use Maximum Likelihood estimate.

> fit <- arima(x, order = c(1, 0, 0), method = c("ML"))
> print(fit)
Call:
arima(x = x, order = c(1, 0, 0), method = c("ML"))


Coefficients:
         ar1  intercept
      0.8633    -0.4055
s.e.  0.0352     0.4637


sigma^2 estimated as 0.8527:  log likelihood = -268.54,  aic = 543.08

Use Conditional Sum of squares to find the initial values and then use MLE

> fit <- arima(x, order = c(1, 0, 0), method = c("CSS-ML"))
> print(fit)
Call:
arima(x = x, order = c(1, 0, 0), method = c("CSS-ML"))


Coefficients:
         ar1  intercept
      0.8633    -0.4060
s.e.  0.0352     0.4636


sigma^2 estimated as 0.8527:  log likelihood = -268.54,  aic = 543.08

Using OLS

> ar.ols(x)
Call:
ar.ols(x = x)


Coefficients:
     1
0.8593


Intercept: -0.01460 (0.0653)


Order selected 1  sigma^2 estimated as  0.8486

Using Yule Walker

> ar.yw(x)
Call:
ar.yw.default(x = x)


Coefficients:
      1        2        3
 0.8583   0.1227  -0.1417


Order selected 3  sigma^2 estimated as  0.8774

Using MLE

> ar.mle(x)
Call:
ar.mle(x = x)


Coefficients:
      1        2        3
 0.8672   0.1291  -0.1543


Order selected 3  sigma^2 estimated as  0.8324

Conditional MLE is far far easier to compute than Unconditional MLE Why ?

Unconditional MLE is essentially a non linear equation and hence one needs to use optimization tools to estimate the parameters

Conditional MLE on the other hand is a simple equation for which a simple differentiation would give the estimates.

Takeaway — Use conditional MLE in most of the cases…

method Fitting method: maximum likelihood or minimize conditional sum-of-squares. The default (unless there are missing values) is to use conditional-sum-of-squares to find starting values, then maximum likelihood. '' The above statements took me about an hour to understand. Even though I had read Hamilton just a month ago I forgot the crucial difference between Conditional Sum of Squares MLE and Unconditional Sum of Squares MLE

I had to go back to Hamilton to understand…

If you have an AR(p) process, you basically collect the first p terms, calculate the sigma, covariance matrix get the multivariate density for the first p terms, then use the multivariate density for the remaining terms and then maximize the likelihood function….. This is UNCONDITIONAL MLE

If you use the first p terms to compute the errors and then maximize MLE, then it is CONDITIONAL MLE and computationally easier…

If you have a very long time series, Conditional and Unconditional difference is hardly a matter of issue But for moderate series..you have to think abt it..