---
title: "5.4.3 Walk-forward with retraining"
description: "Expanding vs. rolling windows and retraining frequency for a ridge on a panel with regimes: what each gains, with its standard error, and why."
url: "https://pythonparatrading.com/en/python-trading-course/level-5/walk-forward-retraining-windows/"
lang: "en"
image: "https://pythonparatrading.com/og/en/python-trading-course/level-5/walk-forward-retraining-windows.png"
---
# 5.4.3 · Walk-forward with retraining: windows and frequency
> Walk-forward is the validation that mimics production: train on what was known, predict the following week and start again. Two decisions remain that lesson 2.5.2 did not have to make with a model: how much history to train on and how often to retrain. In a panel with effects that change sign depending on the regime, intuition says “short window and frequent retraining”; the data say otherwise.
- Course: [Level 5: Machine learning and deep learning (Keras/PyTorch)](https://pythonparatrading.com/en/python-trading-course/level-5/)
- Module: 5.4 · Validation for financial series
- Duration: 65 min
- Prerequisites: [5.4.2 · Purged k-fold with embargo and CPCV for ML models](https://pythonparatrading.com/en/python-trading-course/level-5/purged-k-fold-embargo-cpcv/), [2.5.2 · In-sample and out-of-sample: anchored and rolling walk-forward](https://pythonparatrading.com/en/python-trading-course/level-2/in-sample-out-of-sample-walk-forward/), [5.5.1 · Regularized regressions: Ridge, Lasso, Elastic Net and logistic](https://pythonparatrading.com/en/python-trading-course/level-5/ridge-lasso-elastic-net-logistic-regression/)
## By the end of this lesson
- Program a walk-forward on a panel that only trains on labels already completed at the decision date, deriving the rule from the end dates of each label.
- Retrain a ridge hundreds of times in seconds with sufficient statistics (X′X and X′y per week) and check it against scikit-learn.
- Compare expanding, rolling and exponentially weighted windows, and annual, quarterly, monthly and weekly retraining frequencies, with the IC, its standard error and paired differences.
- Read the result by regime and through the path of a coefficient against the true one, and explain the bias-variance trade-off that decides the window.
## Two decisions that validation does not make for you
Lesson 2.5.2 ran a walk-forward to choose a parameter of a strategy and 5.4.2, a purged validation to compare models.
A walk-forward with a machine learning model adds two design decisions:
- **The window**: with all the available history (**expanding**), with the last N years (**rolling**) or with all of
it but giving more weight to recent data (**exponential weights**, with a half-life).
- **The frequency**: retrain every year, every quarter, every month or every week.
```mermaid
flowchart TB
D[Decision date] --> E[Closed labels]
E --> V[Choose window]
V --> M[Retrain if due]
M --> P[Predict the week]
P --> A[Advance one week]
A --> D
```
The figure compares what the model sees before each test year with an expanding window and with a three-year rolling
window, both retrained every January:
*Fig. — Expanding versus three-year rolling window in a yearly-retrained walk-forward: the data the model sees before each test year (schematic).* Six rows. With an expanding window, training always starts in 2012 and grows: up to 2015, up to 2019, up to 2023. With a three-year rolling window, training is always the previous three years: 2012-2014, 2016-2018, 2020-2022. In each row the highlighted bar is the test year.
Intuition pushes toward short windows and frequent retraining: markets change. The level 5 panel is a good place to
put it to the test, because **the effects really do change** and you know by how much. According to the README, each
true coefficient is `base × regime × m`, with m a weekly AR(1) with persistence 0.9 (the effects weaken and sometimes
reverse) and momentum multiplied by −2.5 in the rebounds after a crisis (the momentum crash).
`panel5_coef_verdad.csv` contains the coefficients of each week and its regime.
## The panel as arrays
To retrain hundreds of times it is convenient to have the panel as arrays of weeks × stocks × features: 730 × 400 × 21.
The transformation is the README one (and that of lesson 5.2.4): the cross-sectional normal rank
$g(x) = \Phi^{-1}\big((\text{rank} - 0.5)/n\big)$ over the stocks that have data, and 0 where it is missing.
```python
import numpy as np
import pandas as pd
from scipy.stats import norm, rankdata
from sklearn.linear_model import Ridge
feat = pd.read_parquet("panel5_features.parquet")
etiq = pd.read_parquet("panel5_etiquetas.parquet")
feat = feat.sort_values(["fecha", "ticker"])
etiq = etiq.sort_values(["fecha", "ticker"])
FEATS = [c for c in feat.columns if c not in
("fecha", "ticker", "sector", "revision_analistas")]
fechas = pd.DatetimeIndex(feat["fecha"].unique())
nW, nA, nF = len(fechas), feat["ticker"].nunique(), len(FEATS)
F = feat[FEATS].to_numpy(dtype=float).reshape(nW, nA, nF)
with np.errstate(invalid="ignore", divide="ignore"):
rango = rankdata(F, axis=1, nan_policy="omit")
n = np.isfinite(F).sum(axis=1, keepdims=True)
G = np.nan_to_num(norm.ppf((rango - 0.5) / n))
R = etiq["ret_sig_semana"].to_numpy().reshape(nW, nA)
fin = etiq["fin_sig_semana"].to_numpy().reshape(nW, nA)[:, 0]
con_feat = np.isfinite(F[:, :, 0]).all(axis=1)
con_etiq = np.isfinite(R).all(axis=1)
valida = con_feat & con_etiq # weeks for training
Yc = np.zeros_like(R)
Yc[valida] = R[valida] - R[valida].mean(axis=1, keepdims=True)
coef = pd.read_csv("panel5_coef_verdad.csv", parse_dates=["fecha"])
coef = coef.set_index("fecha").reindex(fechas)
regimen = coef["regimen"].to_numpy()
print(f"{nW} weeks x {nA} stocks x {nF} features; "
f"valid weeks {valida.sum()}")
print(pd.Series(regimen[valida]).value_counts().to_string())
```
```output
730 weeks x 400 stocks x 21 features; valid weeks 729
calma 575
estres 89
rebote 65
```
## The time rule: train on closed labels
At decision date w you can only train on rows whose label has already ended. The rule comes from the data, not from
an assumption: for each date, the last usable week is the last one whose `fin_sig_semana` is **before** it.
```python
siguiente = (fin[:-1] == fechas[1:].to_numpy()).mean()
print(f"the label of week w ends at date w+1: "
f"{siguiente:.1%} of the weeks")
def ultima_conocida(w):
"""Index of the last week whose label ended before
date w (end < date: a one-week margin)."""
return int(np.searchsorted(fin, fechas[w], side="left")) - 1
w = fechas.get_loc(pd.Timestamp("2020-03-20"))
u = ultima_conocida(w)
print(f"decision on {fechas[w].date()}: train up to the week of "
f"{fechas[u].date()} (its label ended on "
f"{pd.Timestamp(fin[u]).date()})")
```
```output
the label of week w ends at date w+1: 100.0% of the weeks
decision on 2020-03-20: train up to the week of 2020-03-06 (its label ended on 2020-03-13)
```
The label of week w ends exactly at date w+1. So, at the close of date w, the label of w−1 has just closed and that of
w is open. The rule `fin < fecha` also leaves out the label of w−1 (which ends exactly that day): it is the margin the
README uses, a prudent one if orders are sent before the close. Including week w would be a leak: exercise 1 measures
how much.
## Retraining a thousand times in seconds
A ridge with intercept on 21 features has a closed-form solution from five quantities: the number of rows, the sums
of x and y, and the matrices $X'X$ and $X'y$. All of them **add up** across weeks. If they are stored per week, the
ridge of any window (or with per-week weights) is a sum and a 21 × 21 system:
$$
\hat\beta = \left(X_c' X_c + \alpha I\right)^{-1} X_c' y_c, \qquad
X_c'X_c = X'X - n\,\overline{x}\,\overline{x}', \quad X_c'y_c = X'y - n\,\overline{x}\,\overline{y}
$$
where the subscript c denotes centered data (scikit-learn centers the data to estimate the intercept, which is not
penalized).
```python
X1 = np.concatenate([np.ones((nW, nA, 1)), G], axis=2)
peso_valida = valida.astype(float)
S_xx = np.einsum("wni,wnj->wij", X1, X1) * peso_valida[:, None, None]
S_xy = np.einsum("wni,wn->wi", X1, Yc) * peso_valida[:, None]
def ridge(pesos, alfa=10.0):
"""Ridge with intercept on the weeks with `pesos` > 0."""
A = np.tensordot(pesos, S_xx, 1)
b = np.tensordot(pesos, S_xy, 1)
n_, mx, my = A[0, 0], A[0, 1:] / A[0, 0], b[0] / A[0, 0]
C = A[1:, 1:] - n_ * np.outer(mx, mx)
c = b[1:] - n_ * mx * my
return np.linalg.solve(C + alfa * np.eye(nF), c)
prueba = np.flatnonzero(valida & (fechas.year < 2015))
sk = Ridge(alpha=10.0).fit(G[prueba].reshape(-1, nF),
Yc[prueba].reshape(-1))
pesos = np.zeros(nW)
pesos[prueba] = 1.0
print(f"max. |difference| with scikit-learn: "
f"{np.abs(ridge(pesos) - sk.coef_).max():.1e}")
```
```output
max. |difference| with scikit-learn: 8.1e-18
```
The difference is in the last decimal place of the `float64`. From here on, each retraining is a weighted sum of 730
small matrices: the whole weekly walk-forward takes less than a second.
## The walk-forward and its metric
The function goes through the test weeks (2015-2025), retrains when due and predicts the score of each stock. The
metric is the **IC**: the rank correlation, each week, between the score and the following week's return, and its
mean. The standard error comes from the weekly series of ICs (the labels of one week do not overlap, so that series is
almost independent).
```python
anyo = fechas.year.to_numpy()
semanas_prueba = np.flatnonzero(con_feat & (anyo >= 2015))
def walk_forward(ventana=None, cada=None, anual=False,
semivida=None):
"""Scores (weeks x stocks) and coefficients per week.
ventana: weeks of history (None = expanding); cada: retrain
every n weeks; anual: every January; semivida: exponential weights."""
pred = np.full((nW, nA), np.nan)
betas = np.full((nW, nF), np.nan)
beta = None
for i, w in enumerate(semanas_prueba):
toca = (beta is None
or (anual and anyo[w] != anyo[semanas_prueba[i - 1]])
or (cada is not None and i % cada == 0))
if toca:
u = ultima_conocida(w)
p = peso_valida.copy()
p[u + 1:] = 0.0
if ventana is not None:
p[:max(0, u + 1 - ventana)] = 0.0
if semivida is not None:
p *= 0.5 ** ((u - np.arange(nW)) / semivida)
beta = ridge(p)
pred[w] = G[w] @ beta
betas[w] = beta
return pred, betas
evaluables = semanas_prueba[con_etiq[semanas_prueba]]
def ic_semanal(pred):
out = []
for w in evaluables:
out.append(np.corrcoef(rankdata(pred[w]),
rankdata(R[w]))[0, 1])
return np.array(out)
def resumen(ic):
et = ic.std(ddof=1) / np.sqrt(len(ic))
return {"IC": ic.mean(), "std. error": et, "t": ic.mean() / et}
base, betas_base = walk_forward(anual=True)
ic_base = ic_semanal(base)
print({k: round(float(v), 4) for k, v in resumen(ic_base).items()},
f"weeks: {len(ic_base)}")
```
```output
{'IC': 0.0185, 'std. error': 0.0038, 't': 4.8789} weeks: 574
```
Expanding, retrained every January: an IC of 0.0185 with t = 4.9 over 574 weeks, the same result the README publishes
for this model. It is the reference for the rest of the lesson.
## Retraining frequency
With the expanding window, four frequencies: every January, every 13 weeks (quarterly), every 4 (monthly) and every
week.
```python
esquemas = {"expanding, annual": ic_base}
for nombre, kw in [("expanding, every 13 weeks", {"cada": 13}),
("expanding, every 4 weeks", {"cada": 4}),
("expanding, weekly", {"cada": 1})]:
esquemas[nombre] = ic_semanal(walk_forward(**kw)[0])
pd.DataFrame({k: resumen(v) for k, v in esquemas.items()}).T.round(4)
```
```output
IC std. error t
expanding, annual 0.0185 0.0038 4.8789
expanding, every 13 weeks 0.0176 0.0038 4.6397
expanding, every 4 weeks 0.0172 0.0038 4.5324
expanding, weekly 0.0176 0.0038 4.6463
```
Retraining more often improves nothing: from 0.0185 to 0.0172-0.0176, differences of a few thousandths with a standard
error of almost four. It makes sense: with an expanding window, one new year on top of ten years of history barely
moves the coefficients, and one new week, even less.
## Expanding, rolling and weighted windows
Now the window. Rolling windows of five, three, two and one year retrained every January, and two weekly versions that
should be the most adaptive: a one-year rolling window and exponential weights with half-lives of two years and of
one.
```python
for nombre, kw in [
("rolling 5 years, annual", {"ventana": 260, "anual": True}),
("rolling 3 years, annual", {"ventana": 156, "anual": True}),
("rolling 2 years, annual", {"ventana": 104, "anual": True}),
("rolling 1 year, annual", {"ventana": 52, "anual": True}),
("rolling 1 year, weekly", {"ventana": 52, "cada": 1}),
("half-life 2 years, weekly", {"semivida": 104, "cada": 1}),
("half-life 1 year, weekly", {"semivida": 52, "cada": 1})]:
p, b = walk_forward(**kw)
esquemas[nombre] = ic_semanal(p)
if nombre == "rolling 1 year, weekly":
betas_movil = b
tabla = pd.DataFrame({k: resumen(v) for k, v in esquemas.items()}).T
print(tabla.round(4).to_string())
```
```output
IC std. error t
expanding, annual 0.0185 0.0038 4.8789
expanding, every 13 weeks 0.0176 0.0038 4.6397
expanding, every 4 weeks 0.0172 0.0038 4.5324
expanding, weekly 0.0176 0.0038 4.6463
rolling 5 years, annual 0.0143 0.0040 3.6273
rolling 3 years, annual 0.0102 0.0037 2.7680
rolling 2 years, annual 0.0085 0.0038 2.2282
rolling 1 year, annual 0.0084 0.0039 2.1652
rolling 1 year, weekly 0.0039 0.0038 1.0086
half-life 2 years, weekly 0.0120 0.0038 3.1412
half-life 1 year, weekly 0.0081 0.0038 2.1550
```
The figure at the top ranks the eleven schemes with their ±2 standard error interval. The pattern is clear: **the
shorter the window, the worse**. Five years lose a quarter of the IC (0.0143); one year, more than half (0.0084), and
with weekly retraining, almost all of it (0.0039, t = 1.0, indistinguishable from zero). The exponential weights fall
in between, worse the shorter the half-life.
The differences between schemes are not read from the intervals of each one, which overlap: they are read from the
week-by-week **paired difference**, which removes the common noise (the good and bad weeks are the same for every
model).
```python
def pareada(a, b):
dif = esquemas[a] - esquemas[b]
et = dif.std(ddof=1) / np.sqrt(len(dif))
return {"difference": dif.mean(), "t": dif.mean() / et}
pd.DataFrame({
"expanding weekly - rolling 1 year weekly":
pareada("expanding, weekly", "rolling 1 year, weekly"),
"expanding annual - expanding weekly":
pareada("expanding, annual", "expanding, weekly"),
"expanding annual - rolling 5 years annual":
pareada("expanding, annual", "rolling 5 years, annual"),
}).T.round(4)
```
```output
difference t
expanding weekly - rolling 1 year weekly 0.0138 3.8797
expanding annual - expanding weekly 0.0008 1.2332
expanding annual - rolling 5 years annual 0.0041 2.6339
```
Expanding against one-year rolling, both with weekly retraining: a difference of 0.014 with t = 3.9. It is the only
firm conclusion in the table. Annual against weekly frequency: t = 1.2, nothing. Expanding against five-year rolling,
both annual: 0.004 with t = 2.6, in the same direction.
## By regime: the momentum crash
The hope of the rolling window was to adapt to changes. The panel has known regime changes: in `estres` reversal
strengthens (×1.5) and in `rebote` momentum reverses (×−2.5). The IC by regime tells whether any window takes
advantage of it:
```python
reg_eval = regimen[evaluables]
por_regimen = {}
for nombre in ("expanding, annual", "rolling 3 years, annual",
"rolling 1 year, weekly", "half-life 1 year, weekly"):
ic = esquemas[nombre]
por_regimen[nombre] = {r: ic[reg_eval == r].mean()
for r in ("calma", "estres", "rebote")}
t = pd.DataFrame(por_regimen).T.round(4)
t.columns = [f"{r} ({(reg_eval == r).sum()} wk.)" for r in t.columns]
t
```
```output
calma (420 wk.) estres (89 wk.) rebote (65 wk.)
expanding, annual 0.0230 0.0026 0.0110
rolling 3 years, annual 0.0156 -0.0132 0.0075
rolling 1 year, weekly 0.0073 -0.0094 -0.0003
half-life 1 year, weekly 0.0128 -0.0077 -0.0003
```
No short window wins in the rebounds, which is where the advantage of adapting was expected: there the expanding window
remains the best (0.011, half of what it gets in calm) and the adaptive ones stay at zero. The panel's rebounds last
three to six months (March-June 2020, April-July 2025…): when a one-year window starts to reflect the change in the
sign of momentum, the rebound is over and the model is left with a distorted coefficient for the calm that follows. In
the stress weeks, moreover, the short windows have a negative IC. The adaptation that would work is not in the window
but in the model: a regime feature observable in time (and properly dated, lesson 5.2.2) with which the model can
learn the interaction.
## Bias versus variance: the path of a coefficient
The explanation lies in the coefficients. `panel5_coef_verdad.csv` contains the true momentum coefficient each week
(`c_mom`, in weekly return per unit of g). The ridge estimates the same object with the feature `mom_12_1` (with the
other 20 alongside, among them `ruido_4`, which inherits part of the momentum, so the level need not match). What
matters is how each estimate moves against the truth:
```python
j = FEATS.index("mom_12_1")
verdad_mom = coef["c_mom"].to_numpy()[semanas_prueba] * 1e4
_, betas_exp = walk_forward(cada=1)
filas = {}
for nombre, b in (("expanding weekly", betas_exp),
("rolling 1 year weekly", betas_movil)):
est = b[semanas_prueba, j] * 1e4
filas[nombre] = {
"mean (bp)": est.mean(), "std. (bp)": est.std(),
"corr. with truth": np.corrcoef(est, verdad_mom)[0, 1],
"root mean squared error (bp)":
np.sqrt(np.mean((est - verdad_mom) ** 2))}
filas["truth"] = {"mean (bp)": verdad_mom.mean(),
"std. (bp)": verdad_mom.std()}
pd.DataFrame(filas).T.round(2)
```
```output
mean (bp) std. (bp) corr. with truth root mean squared error (bp)
expanding weekly 4.88 0.93 -0.01 6.04
rolling 1 year weekly 4.37 6.37 -0.10 8.99
truth 2.40 5.42 NaN NaN
```
*Fig. — True momentum coefficient in the level 5 panel against its ridge estimate with an expanding window and a one-year rolling window (weekly retraining, every 4 weeks shown).* Three lines from 2015 to 2025. The truth (solid line) swings between about -10 and +9 basis points per week, dropping to -25 in the 2020 rebound and to -9 in those of 2016 and 2019. The one-year rolling estimate (dashed) swings between -13 and +16 without tracking the truth (correlation -0.10). The expanding one (highlighted) is almost flat, between 4 and 7.
The true coefficient fluctuates a lot from one week to the next (a standard deviation of 5.4 bp around a mean of 2.4)
and changes sign in the rebounds. The one-year estimate also fluctuates, with a standard deviation of 6.4 bp, but it
does not follow the truth: its correlation with it is −0.10. It chases the noise in the sample of the last twelve
months, not the effect. The expanding one is almost a straight line (a standard deviation of 0.9 bp, zero correlation):
it is biased, because it does not see the changes, but its root mean squared error against the truth is 6.0 bp and that
of the rolling one, 9.0. Tracking an effect that changes every few weeks with one year of noisy data adds more error
than it removes. It is the classic result of Pesaran and Timmermann (2007): with breaks, the optimal window depends on
the size of the break relative to the noise, and when the signal is small, using all the data usually wins.
| Situation | Window that usually works best |
|---|---|
| Weak signal (IC in the hundredths), effects that fluctuate without a trend | expanding, or weights with a long half-life |
| Large, permanent break (an effect that disappears) | rolling or weighted, after detecting it (lesson 5.10.2) |
| Strong, changing signal (high frequency, abundant data) | short rolling window and frequent retraining |
| Any | whichever a walk-forward fixed in advance says, with its paired difference |
## Exercises
1. Repeat the weekly expanding walk-forward and the 26-week rolling one forgetting the margin: train up to week w
itself, whose label is the one being predicted. How much does the IC rise in each one, and why more in the short
window?
**Solution.**
```python
def ultima_con_fuga(w):
return w # includes the label of week w itself: a leak
original = ultima_conocida
for nombre, kw in (("expanding weekly", {"cada": 1}),
("rolling 26 weeks", {"ventana": 26, "cada": 1})):
fila = []
for fn in (original, ultima_con_fuga):
ultima_conocida = fn
fila.append(resumen(ic_semanal(walk_forward(**kw)[0]))["IC"])
print(f"{nombre:<18} no leak {fila[0]:.4f} with leak "
f"{fila[1]:.4f}")
ultima_conocida = original
```
```output
expanding weekly no leak 0.0176 with leak 0.0272
rolling 26 weeks no leak 0.0051 with leak 0.0603
```
With the expanding window, the leaked row is one week out of the more than 500 in the training set, and even so the IC
rises. With 26 weeks, it is 4% of the training set: the model learns mostly from the week it has to predict and the IC
multiplies. Short windows are more sensitive to any leak at the edge, and that is why a rolling-window walk-forward
that gives an excellent result deserves a special review of the time rule.
2. Try exponential weights with half-lives of 13, 26, 52, 104, 208 and 416 weeks (weekly retraining). Which one wins?
Can you present its IC as an out-of-sample result?
**Solution.**
```python
barrido = {h: resumen(ic_semanal(walk_forward(semivida=h,
cada=1)[0]))["IC"]
for h in (13, 26, 52, 104, 208, 416)}
pd.Series(barrido, name="IC").rename_axis("half-life").round(4)
```
```output
half-life
13 0.0048
26 0.0057
52 0.0081
104 0.0120
208 0.0150
416 0.0165
Name: IC, dtype: float64
```
The longest half-life wins, which is the one most similar to the expanding window: the sweep confirms the lesson. But
the IC of the best one is not an out-of-sample result: it was chosen by looking at the six walk-forwards. It has to be
logged as the best of six attempts and the selection discounted (lesson 5.4.4), or the half-life chosen inside the
walk-forward with past data (a nested walk-forward). Here the sweep says something useful, that more history is
better; the specific figure of the winner does not.
3. The level's `deriva5_*` data (lesson 5.10.2) have a **permanent** sign change in the effect of a feature in July
2019. Without doing any calculations, would you expect the expanding or the rolling window to win there? What
would you need in order to choose without looking at the result?
**Solution.**
There a rolling or weighted window should win: the change is large (the effect goes from +8 to −8 basis points per
unit) and permanent, so after a few months the history before 2019 teaches the model exactly the opposite of what is
happening. The expanding window would take years to dilute it. To choose without looking at the result you need a
rule fixed in advance: a drift detector (the moving average of the model's IC, as in the level's README, or the
structural break tests of lesson 2.1.7) that triggers the switch to a short window, all of it evaluated together in a
walk-forward. The general lesson: the window is chosen by the nature of the changes (fluctuations versus breaks) and
by the amount of signal, and it is validated as a complete procedure.
## Self-assessment
1. In the panel, the label `ret_sig_semana` of date w ends at date w+1. Which weeks can a model that decides at the close of date w be trained on?
- A. All of them up to and including w
- B. Only those of previous years
- C. The weeks whose label has already ended: up to w−1 (which ends exactly at w) or, leaving a margin like the README, up to w−2
- D. Those from w+1 onward
*Answer C: The weeks whose label has already ended: up to w−1 (which ends exactly at w) or, leaving a margin like the README, up to w−2.* The label of week w itself is what you want to predict. Including it is a leak: in exercise 1 it raises the IC from 0.018 to 0.027 with an expanding window and from 0.005 to 0.060 with a 26-week window.
2. Expanding window retrained every year: IC 0.0185 (t = 4.9). One-year rolling window retrained every week: IC 0.0039 (t = 1.0). Why does the rolling window lose, if the true coefficients change week by week?
- A. Because the rolling window has a programming error
- B. Because with a true IC of a few hundredths, one year of data estimates the coefficients with so much noise that the estimation error far exceeds what is gained by tracking the changes
- C. Because retraining every week causes leaks
- D. Because rolling windows only work on daily data
*Answer B: Because with a true IC of a few hundredths, one year of data estimates the coefficients with so much noise that the estimation error far exceeds what is gained by tracking the changes.* It is the bias-variance trade-off. The path of the momentum coefficient with a one-year window fluctuates as much as the true one, but is not correlated with it (−0.10); the expanding one is almost constant, biased but with less error (6.0 bp against 9.0).
3. Retraining every week instead of every January (expanding window) changes the IC from 0.0185 to 0.0176, with a paired difference of t = 1.2. What do you conclude?
- A. That retraining every week makes the model worse
- B. That you have to retrain every day
- C. That the expanding window does not learn
- D. That with an expanding window the frequency barely matters: one more year of data on top of ten moves the coefficients very little, and the difference cannot be told apart from noise
*Answer D: That with an expanding window the frequency barely matters: one more year of data on top of ten moves the coefficients very little, and the difference cannot be told apart from noise.* A difference with t = 1.2 is evidence of nothing. With an expanding window, each retraining adds a small fraction of data; retraining often costs computation and buys no precision.
4. In the 65 weeks of the `rebote` regime (true momentum reverses, ×−2.5), which scheme protects against the momentum crash?
- A. None: rebounds last a few months and no retraining detects the sign change in time; protection would have to come from the model (the regime as a feature) or from risk management, not from the window
- B. The one-year rolling window, because it adapts
- C. The expanding window with weekly retraining
- D. Exponential weights with a 13-week half-life
*Answer A: None: rebounds last a few months and no retraining detects the sign change in time; protection would have to come from the model (the regime as a feature) or from risk management, not from the window.* By the time a year of data reflects a rebound, the rebound is over. Adapting through the window arrives late and pays for the noise in every other week.
5. Six half-lives are tried for the exponential weights and the IC of the best one is reported. What should be done with that number?
- A. Treat it as the best of six attempts: log them all and discount the selection (Deflated Sharpe, lesson 5.4.4), or choose the half-life inside the walk-forward itself with past data
- B. Nothing: it is out of sample because it comes from a walk-forward
- C. Multiply it by six
- D. Use the worst of the six
*Answer A: Treat it as the best of six attempts: log them all and discount the selection (Deflated Sharpe, lesson 5.4.4), or choose the half-life inside the walk-forward itself with past data.* A walk-forward is out of sample for a procedure fixed in advance. If the procedure is chosen by looking at the walk-forward result, the result no longer is.
## Key points
- Walk-forward rule on a panel: at the decision date, only train on rows whose label has already ended (`fin_sig_semana` before the date). Forgetting it with a 26-week rolling window multiplies the IC by twelve.
- With sufficient statistics per week (X′X, X′y), a ridge is retrained thousands of times in seconds; it matches scikit-learn.
- Expanding window retrained every January: IC 0.0185 (t = 4.9), the README figure. Retraining more often does not improve it (0.0172-0.0176).
- Rolling windows lose, and the shorter the window, the more they lose: 0.0143 with five years, 0.0084 with one. Expanding against one-year rolling, both weekly: paired difference of 0.014 with t = 3.9.
- No short window adapts to rebounds (momentum crash): the change lasts less than it takes to show up in the data, and the expanding window remains the best there too.
- With little signal, variance rules: the one-year estimate tracks a changing coefficient poorly; the expanding one, biased and stable, predicts better.
## Resources
### Research
- [Introduction to Quant Research](https://pythonparatrading.com/en/quant-research-2/) · Research-0023 · 8 min
### Documentation
- [scikit-learn: Ridge](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html) · scikit-learn.org
- [SciPy: scipy.stats.rankdata (nan_policy and axis)](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rankdata.html) · docs.scipy.org
### Books
- [Pesaran, M. H. and Timmermann, A. (2007): Selection of estimation window in the presence of breaks. Journal of Econometrics 137(1), 134-161](https://doi.org/10.1016/j.jeconom.2006.03.010) · doi.org
- [López de Prado, M. (2018): Advances in Financial Machine Learning. Wiley (chapter 11: the dangers of backtesting)](https://www.wiley.com/en-us/Advances+in+Financial+Machine+Learning-p-9781119482086) · wiley.com
### Data
- [Level 5 data: true coefficients and regime of each week (CSV)](https://pythonparatrading.com/curso/datos/nivel-5/panel5_coef_verdad.csv) · Download · panel5_coef_verdad.csv
- [Level 5 data: weekly panel features (Parquet)](https://pythonparatrading.com/curso/datos/nivel-5/panel5_features.parquet) · Download · panel5_features.parquet
Previous lesson: [5.4.2 · Purged k-fold with embargo and CPCV for ML models](https://pythonparatrading.com/en/python-trading-course/level-5/purged-k-fold-embargo-cpcv/)
Next lesson: [5.4.4 · Backtesting ML signals with costs and Deflated Sharpe](https://pythonparatrading.com/en/python-trading-course/level-5/backtesting-ml-signals-deflated-sharpe/)