Autocorrelation, ACF, PACF & Ljung-Box Test

The Story Behind Autocorrelation

Autocorrelation is not a statistical curiosity — it is the reason time series models work. If a series had no autocorrelation at any lag, no model could forecast it better than a random guess. Every insight ARIMA, SARIMA, and ARMA extracts from a series ultimately traces back to the autocorrelation structure you will learn to read in this tutorial.

Autocorrelation — The Series Talking to Itself

In classical statistics, correlation measures how two different variables move together. Autocorrelation measures how a single series moves together with a lagged copy of itself.

Positive, Negative & Zero Autocorrelation

The ACF Plot — Reading the Full Memory Map

The ACF plot shows ρ(k) for k = 1, 2, 3, … lags on the x-axis, with correlation coefficient (−1 to +1) on the y-axis. The blue dashed lines are the 95% confidence bands: bars outside these bands are statistically significant at the 5% level. Bars inside are indistinguishable from zero.

The PACF — Cutting Through Indirect Relationships

Reading ACF & PACF Together — The Complete Guide

Lag Selection — How Many Lags Should You Include?

Including too few lags leaves autocorrelation in residuals. Including too many wastes parameters, inflates variance, and over-fits. Lag selection is one of the most practically important decisions in time series modelling.

The White Noise Test — When Autocorrelation Should Be Zero

The Ljung-Box Test — The Formal White Noise Diagnostic

Visually checking whether bars are inside confidence bands is useful but imprecise. The Ljung-Box test (Box and Ljung, 1978) provides a formal hypothesis test that jointly tests whether any of the first m autocorrelations is non-zero.

Complete Python Implementation

# ─── 0. Imports ───────────────────────────────────────────────────────────────
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')

from statsmodels.tsa.stattools         import acf, pacf, acovf
from statsmodels.graphics.tsaplots     import plot_acf, plot_pacf
from statsmodels.stats.diagnostic      import acorr_ljungbox
from statsmodels.tsa.arima_process     import ArmaProcess
from statsmodels.tsa.arima.model       import ARIMA

# ─── 1. Generate three series with known properties ────────────────────────────
np.random.seed(42)
n = 400

# White noise baseline
wn = np.random.normal(0, 1, n)

# AR(2): φ₁=0.7, φ₂=−0.3  →  PACF should cut off at lag 2
ar2_process = ArmaProcess(np.array([1, -0.7, 0.3]), np.array([1]))
ar2 = ar2_process.generate_sample(n)

# MA(2): θ₁=0.6, θ₂=0.3  →  ACF should cut off at lag 2
ma2_process = ArmaProcess(np.array([1]), np.array([1, 0.6, 0.3]))
ma2 = ma2_process.generate_sample(n)

series_dict = {'White Noise': wn, 'AR(2)': ar2, 'MA(2)': ma2}

# ─── 2. Compute and print autocorrelations at specific lags ───────────────────
for name, s in series_dict.items():
    acf_vals  = acf(s, nlags=10, fft=True)
    pacf_vals = pacf(s, nlags=10, method='ols')
    print(f"\n── {name} ──")
    print(f"{'Lag':<6} {'ACF':>10} {'PACF':>10}")
    print("-" * 30)
    for k in range(1, 6):
        print(f"{k:<6} {acf_vals[k]:>10.4f} {pacf_vals[k]:>10.4f}")

# ─── 3. Plot ACF and PACF ─────────────────────────────────────────────────────
fig, axes = plt.subplots(3, 2, figsize=(12, 9))

for row, (name, s) in enumerate(series_dict.items()):
    plot_acf(s,  lags=20, ax=axes[row, 0],
             title=f'ACF — {name}', alpha=0.05)
    plot_pacf(s, lags=20, ax=axes[row, 1],
              title=f'PACF — {name}', alpha=0.05, method='ols')
    for ax in axes[row]:
        ax.set_facecolor('#0d1117')
        ax.spines['bottom'].set_color('#2a3050')
        ax.spines['left'].set_color('#2a3050')
        ax.tick_params(colors='#8892a4')

plt.tight_layout(pad=1.8)
plt.savefig('acf_pacf_all.png', dpi=120, facecolor='#0d1117')
plt.show()

# ─── 4. Ljung-Box white noise tests ──────────────────────────────────────────
print(f"{'Series':<14} {'Lag':>6} {'LB Stat':>10} {'p-value':>10} {'Verdict'}")
print("-" * 60)

for name, s in series_dict.items():
    lb = acorr_ljungbox(s, lags=[10, 20], return_df=True)
    for lag, row in lb.iterrows():
        verdict = "White Noise ✓" if row['lb_pvalue'] > 0.05 else "NOT white noise ✗"
        print(f"{name:<14} {lag:>6} {row['lb_stat']:>10.3f} {row['lb_pvalue']:>10.4f}  {verdict}")

# ─── 5. Residual diagnostics after fitting ARMA(1,1) ─────────────────────────
# Fit a model with WRONG order to see bad residuals, then correct order

# WRONG: ARIMA(1,0,0) on MA(2) data
wrong_model = ARIMA(ma2, order=(1, 0, 0)).fit()
wrong_resid  = wrong_model.resid

# CORRECT: ARIMA(0,0,2) = MA(2)
right_model = ARIMA(ma2, order=(0, 0, 2)).fit()
right_resid  = right_model.resid

for label, resid in [('WRONG AR(1) on MA(2)', wrong_resid),
                      ('CORRECT MA(2)',        right_resid)]:
    lb = acorr_ljungbox(resid, lags=[10, 20], return_df=True)
    print(f"\n{label}")
    print(f"  Lag 10:  Q={lb['lb_stat'][10]:.3f}  p={lb['lb_pvalue'][10]:.4f}  "
          f"{'White Noise ✓' if lb['lb_pvalue'][10] > 0.05 else 'NOT white noise ✗'}")
    print(f"  Lag 20:  Q={lb['lb_stat'][20]:.3f}  p={lb['lb_pvalue'][20]:.4f}  "
          f"{'White Noise ✓' if lb['lb_pvalue'][20] > 0.05 else 'NOT white noise ✗'}")

End-to-End Example — From Raw Series to Final Diagnosis

# ─── Real-world workflow: Airline Passengers Dataset ─────────────────────────
url = 'https://raw.githubusercontent.com/jbrownlee/Datasets/master/airline-passengers.csv'
df = pd.read_csv(url, header=0, index_col=0, parse_dates=True)
series = df.squeeze()

# ─── Step 1: Stationarize: log + first diff ───────────────────────────────────
log_diff = np.log(series).diff().dropna()

# ─── Step 2: Compute ACF and PACF values ─────────────────────────────────────
n_lags = 24
acf_vals,  acf_ci  = acf(log_diff,  nlags=n_lags, alpha=0.05, fft=True)
pacf_vals, pacf_ci = pacf(log_diff, nlags=n_lags, alpha=0.05, method='ols')

# ─── Step 3: Identify significant lags ───────────────────────────────────────
conf_bound = 1.96 / np.sqrt(len(log_diff))   # approx 95% band width

sig_acf_lags  = [k for k in range(1, n_lags+1) if abs(acf_vals[k])  > conf_bound]
sig_pacf_lags = [k for k in range(1, n_lags+1) if abs(pacf_vals[k]) > conf_bound]

print(f"Significant ACF lags  : {sig_acf_lags}")
print(f"Significant PACF lags : {sig_pacf_lags}")
print(f"Confidence bound (±)  : {conf_bound:.4f}")

# ─── Step 4: Ljung-Box on the log-diff series ────────────────────────────────
lb_raw = acorr_ljungbox(log_diff, lags=[12, 24], return_df=True)
print("\nLjung-Box on log-diff series:")
print(lb_raw)

# ─── Step 5: Fit SARIMA(1,1,1)(1,1,1)[12] — guided by ACF/PACF ───────────────
from statsmodels.tsa.statespace.sarimax import SARIMAX

sarima = SARIMAX(np.log(series),
                  order=(1, 1, 1),
                  seasonal_order=(1, 1, 1, 12),
                  enforce_stationarity=False).fit(disp=False)

# ─── Step 6: Ljung-Box on SARIMA residuals ───────────────────────────────────
resid = sarima.resid[13:]   # skip initialisation period
lb_resid = acorr_ljungbox(resid, lags=[12, 24], return_df=True)
print("\nLjung-Box on SARIMA residuals:")
print(lb_resid)
print(f"\nModel AIC: {sarima.aic:.2f}")

Complete Reference — ACF, PACF & Ljung-Box Summary

Golden Rules — ACF, PACF & Ljung-Box in Practice