Scheduled retraining, automatic validation and champion/challenger.

A cycle, not a training run

Lesson 5.4.3 decided when to retrain and with which window, and lesson 5.10.2, how to detect that the world has changed. Both work in a lab: a loop that retrains a thousand times with clean data. In production, every retraining is an event: a process that starts on the first business day of the month, asks for data from a pipeline that may have failed that week, trains, and has to decide with nobody watching whether the model that just came out replaces the one that moves money. This lesson builds that process. MLflow's experiment and model registry is introduced in lesson 7.3.1; here it is used for what is needed at the end: versions and aliases.

flowchart TB
  C[Calendar] --> D[Data snapshot]
  D --> E[Train]
  E --> P[Gates]
  P -->|fails| X[Rejection and alert]
  P -->|passes| R[Registry]
  R --> S[Shadow]
  S --> V[Champion]

In MLOps vocabulary, the champion is the model in production and the challenger the one that aims to replace it. For the handover to be automatic, three things are needed: knowing which data each candidate was trained on (the snapshot), a list of checks that a broken model does not pass (the gates) and a mechanism to switch models and undo the switch without deploying code (the registry aliases).

The lab: a changing regime and three faults

The data is synthetic, with a known answer. Fifty assets, five persistent features (an AR(1) with coefficient 0.9, like a momentum or a value signal that change slowly) and a next-day return that depends on them with coefficients that change on April 1, 2024: the first feature goes from adding to subtracting, the third starts to count, and the second and fourth stay the same.

import numpy as np
import pandas as pd
from scipy.stats import rankdata
from sklearn.linear_model import Ridge

rng = np.random.default_rng(732)
fechas = pd.bdate_range("2020-06-01", "2025-12-31")
T, N, K = len(fechas), 50, 5
CAMBIO = pd.Timestamp("2024-04-01")
F = np.zeros((T, N, K))                  # features at the close of t
F[0] = rng.standard_normal((N, K))
for t in range(1, T):                    # AR(1), persistence 0.9
    F[t] = 0.9 * F[t - 1] + np.sqrt(0.19) * rng.standard_normal((N, K))
B_ANTES = np.array([0.4, 0.3, 0.0, 0.2, 0.0])
B_DESPUES = np.array([-0.3, 0.3, 0.4, 0.2, 0.0])
B = np.where((fechas < CAMBIO)[:, None], B_ANTES, B_DESPUES)
VOL = 0.012
# R[t]: return from t to t+1, known at the close of t+1
R = VOL * (0.15 * np.einsum("tnk,tk->tn", F, B)
           + rng.standard_normal((T, N)))

def ic_dia(P, Y):
    """Spearman IC of each day (rows) between prediction and
    return."""
    a = rankdata(P, axis=1) - (N + 1) / 2
    b = rankdata(Y, axis=1) - (N + 1) / 2
    return (a * b).sum(1) / np.sqrt((a * a).sum(1) * (b * b).sum(1))

oraculo = pd.Series(ic_dia(np.einsum("tnk,tk->tn", F, B), R),
                    index=fechas)
print(f"{T} sessions x {N} assets x {K} features")
print("oracle IC before and after the change:",
      round(oraculo[:CAMBIO].iloc[:-1].mean(), 3),
      round(oraculo[CAMBIO:].mean(), 3))

1458 sessions x 50 assets x 5 features
oracle IC before and after the change: 0.077 0.078

The oracle's IC (the predictions with each day's true coefficients) is 0.077 before the change and 0.078 after: the problem does not get harder, it becomes a different one. A model that does not notice will keep betting on the first feature just as it starts to subtract.

On top of that, three faults in the data pipeline, each one visible only in the snapshot that a specific cycle receives, as happens in reality: the vendor fixes the data, or someone reverts the code change, and the following month everything is fine.

AVERIAS = {
    pd.Timestamp("2023-09-01"): "f1 frozen in August",
    pd.Timestamp("2023-12-01"): "target with the flipped sign",
    pd.Timestamp("2025-02-03"): "f5 built from the target",
}

def foto(ciclo):
    """Features and targets as the data pipeline delivers them on
    the day of the cycle (with the fault, if there is one that day)."""
    Fx, Rx = F.copy(), R.copy()
    a = AVERIAS.get(ciclo, "")
    if a.startswith("f1"):              # the vendor repeats the value
        i = np.where((fechas >= "2023-08-01")
                     & (fechas < "2023-09-01"))[0]
        Fx[i, :, 0] = F[i[0] - 1, :, 0]
    elif a.startswith("target"):        # precio_t / precio_t+1 - 1
        Rx = -R
    elif a.startswith("f5"):            # a join on the wrong date
        Fx[:, :, 4] = F[:, :, 4] + R / VOL
    return Fx, Rx

for c, a in AVERIAS.items():
    print(c.date(), a)

2023-09-01 f1 frozen in August
2023-12-01 target with the flipped sign
2025-02-03 f5 built from the target

• September 2023: the vendor repeated July's last value of f1 throughout August. It is the most common failure of data that arrives through an external service: there is no error, just a number that does not change.
• December 2023: a change in the code that computes the target inverts it (today's price divided by tomorrow's, instead of the other way around). Every column looks fine.
• February 2025: a merge on the wrong date puts information from the target itself into f5. It is leakage: the model learns from something it will not have live.

The cycle runs before the open on the first business day of each month, from February 2023 to December 2025. On that day, the last closed target is the return from the day before yesterday to yesterday: the row of the day before yesterday is the last one that can go into training. Behind it, 42 sessions (two months) are set aside for validation, and behind those, 252 for training:

VENT, VAL = 252, 42           # training and validation sessions
inicio = fechas.get_loc(pd.Timestamp("2023-01-02"))
ciclos = [fechas[fechas >= d][0]
          for d in pd.date_range("2023-02-01", "2025-12-01",
                                 freq="MS")]

def ajustar(Fx, Rx, a, b):
    X, y = Fx[a:b].reshape(-1, K), Rx[a:b].ravel()
    return Ridge(alpha=1.0).fit(X, y)

def predecir(m, Fx, a, b):
    return m.predict(Fx[a:b].reshape(-1, K)).reshape(b - a, N)

def ventanas(ciclo):
    """Training and validation indices of the cycle. The last row
    with a closed target is the one two sessions earlier."""
    i = fechas.get_loc(ciclo)
    fin = i - 1                      # rows < fin: target known
    return (fin - VAL - VENT, fin - VAL), (fin - VAL, fin)

(e0, e1), (v0, v1) = ventanas(pd.Timestamp("2024-05-01"))
print(len(ciclos), "cycles, from", ciclos[0].date(), "to",
      ciclos[-1].date())
print("cycle 2024-05-01 · trains", fechas[e0].date(), "to",
      fechas[e1 - 1].date(), "· validates", fechas[v0].date(), "to",
      fechas[v1 - 1].date())
champion_0 = ajustar(F, R, inicio - 1 - VENT, inicio - 1)
print("initial champion coefficients (x1000):",
      (champion_0.coef_ * 1e3).round(2))

35 cycles, from 2023-02-01 to 2025-12-01
cycle 2024-05-01 · trains 2023-03-15 to 2024-02-29 · validates 2024-03-01 to 2024-04-29
initial champion coefficients (x1000): [ 0.59  0.44 -0.1   0.39  0.08]

The initial champion was trained on the 252 sessions before January 2023 and recovers the structure of the first regime well: large positive coefficients on f1, f2 and f4, almost zero on f3 and f5. Two details of the cycle that matter in production:

• The snapshot (foto(ciclo)) is what the pipeline delivered that day, not what is in the database today. If the vendor corrects August in October, retraining “the September cycle” with today's data gives a different model. For a cycle to be reproducible, the snapshot is stored or identified by a fingerprint, as done further below.
• Each cycle's model is trained with a fixed recipe (Ridge, alpha=1, 252 sessions). Changing the recipe (other features, another algorithm) is not a retraining: it is a new model, and it is validated as in lessons 5.4.3 and 5.4.4 before entering the cycle.

Four gates

A gate is an automatic check with a threshold fixed in advance. If it fails, the candidate is not registered as a challenger, the champion stays and an alert fires (lesson 4.5.4). The order goes from cheap and general to expensive and specific:

The leakage gate rests on a domain fact: in daily data no legitimate feature explains the next day's return with a correlation of 0.3; in the lab, the largest in the cycles without faults is 0.085. The behavior gate rests on two models of the same recipe one month apart being very similar: a candidate that predicts the opposite of the champion is not an improvement, it is a mistake. And the performance gate uses the day-by-day IC difference over the same sessions, a paired test: both models see the same market, so most of the noise cancels out in the difference. Its standard error is computed with Newey-West, because with persistent features the ICs of consecutive days are correlated:

def t_hac(d, L=5):
    """t of the mean of d with Newey-West variance (L lags)."""
    m, e = d.mean(), d - d.mean()
    v = e @ e / len(d)
    for k in range(1, L + 1):
        v += 2 * (1 - k / (L + 1)) * (e[k:] @ e[:-k]) / len(d)
    return m / np.sqrt(v / len(d)) if v > 0 else 0.0

def puertas(ciclo, Fx, Rx, cand, champ, exigir_mejora):
    """Returns (the gate that fails or 'ok', detail)."""
    (e0, e1), (v0, v1) = ventanas(ciclo)
    # 1. data contract: repeated values in the last 63
    ult = Fx[v1 - 63:v1]
    rep = (ult[1:] == ult[:-1]).mean(axis=(0, 1))
    if np.isnan(Fx[e0:v1]).any():
        return "data", "NaN present"
    if rep.max() > 0.05:
        return "data", f"f{rep.argmax() + 1} repeated {rep.max():.0%}"
    # 2. leakage: no feature may explain the day's target
    X, y = Fx[e0:v1].reshape(-1, K), Rx[e0:v1].ravel()
    rho = np.array([np.corrcoef(X[:, k], y)[0, 1] for k in range(K)])
    if np.abs(rho).max() > 0.3:
        k = np.abs(rho).argmax()
        return "leakage", f"corr(f{k + 1}, target) = {rho[k]:.2f}"
    # 3. behavior: similar to the champion on the same inputs
    pc = predecir(cand, Fx, v0, v1)
    pp = predecir(champ, Fx, v0, v1)
    r = np.corrcoef(pc.ravel(), pp.ravel())[0, 1]
    if r < 0.5:
        return "behavior", f"corr. with champion {r:+.2f}"
    if not exigir_mejora:
        return "ok", f"corr. with champion {r:+.2f}"
    # 4. performance: daily IC against the champion, same sessions
    d = ic_dia(pc, Rx[v0:v1]) - ic_dia(pp, Rx[v0:v1])
    t = t_hac(d)
    det = f"dIC {d.mean():+.3f}, t {t:+.1f}"
    return ("ok" if d.mean() > 0.01 and t > 2 else "no improvement"), det

for c in [pd.Timestamp("2023-06-01"), *AVERIAS]:
    Fx, Rx = foto(c)
    (e0, e1), _ = ventanas(c)
    cand = ajustar(Fx, Rx, e0, e1)
    print(c.date(), *puertas(c, Fx, Rx, cand, champion_0, True))

2023-06-01 no improvement dIC +0.003, t +0.9
2023-09-01 data f1 repeated 35%
2023-12-01 behavior corr. with champion -0.95
2025-02-03 leakage corr(f5, target) = 0.70

All three faults are caught, each one by a different gate, and a normal cycle reaches the last one:

• September 2023: 35% of the f1 values in the last 63 sessions are equal to the previous day's. August is 22 of those 63 sessions; with real data that moves, the proportion is zero.
• December 2023: the features are fine and nothing explains the day's target, so it passes the first two. But its predictions have a correlation of −0.95 with the champion's: the model has learned the market backwards.
• February 2025: f5 has a correlation of 0.70 with the same day's target. No honest feature does that.
• June 2023: a cycle without faults in the initial regime. The candidate beats the champion by 0.003 of IC with t = 0.9: there is no reason to switch models.

Champion/challenger: require an improvement?

The underlying question is what a candidate has to prove to replace the champion. There are two reasonable answers, and the lab lets you measure both, alongside two baselines: never retraining, and retraining every month while deploying whatever comes out.

• Fixed: the January 2023 champion forever.
• Always: every cycle retrains on the last 252 sessions and deploys, with no gates.
• Superiority: classic champion/challenger. The candidate passes the four gates and, at the performance gate, has to beat the champion by more than 0.01 of IC with t > 2. The validated model is deployed.
• Gates: the candidate passes the first three gates; if it does, it is refitted on the last 252 sessions (including the validation ones) and deployed. It is not required to improve.

def politica(nombre):
    """Simulates the cycles and returns the live daily IC, the
    decision log and the deployed models."""
    champ, vigentes, reg = champion_0, [(inicio, champion_0)], []
    for c in ciclos:
        if nombre == "fixed":
            break
        Fx, Rx = foto(c)
        (e0, e1), (v0, v1) = ventanas(c)
        i = fechas.get_loc(c)
        if nombre == "always":             # no gates
            champ = ajustar(Fx, Rx, v1 - VENT, v1)
            vigentes.append((i, champ))
            reg.append((c, "deployed", ""))
            continue
        cand = ajustar(Fx, Rx, e0, e1)
        mejora = nombre == "superiority"
        g, det = puertas(c, Fx, Rx, cand, champ, mejora)
        reg.append((c, g, det))
        if g == "ok":                      # gates: refit on everything
            champ = cand if mejora else ajustar(Fx, Rx, v1 - VENT, v1)
            vigentes.append((i, champ))
    P = np.zeros((T, N))
    for j, (a, m) in enumerate(vigentes):
        b = vigentes[j + 1][0] if j + 1 < len(vigentes) else T
        P[a:b] = predecir(m, F, a, b)
    ic = pd.Series(ic_dia(P[inicio:], R[inicio:]),
                   index=fechas[inicio:])
    return ic, reg, len(vigentes) - 1

res = {n: politica(n) for n in
       ("fixed", "always", "superiority", "gates")}
filas = {}
for n, (ic, reg, k) in res.items():
    mes = ic.groupby(ic.index.to_period("M")).mean()
    filas[n] = {"IC": ic.mean(), "before": ic[:"2024-03"].mean(),
                "after": ic["2024-04":].mean(),
                "2023-12": mes["2023-12"], "2025-02": mes["2025-02"],
                "deployments": k}
pd.DataFrame(filas).T.round(3).astype({"deployments": int})

                IC  before  after  2023-12  2025-02  deployments
fixed        0.032   0.094 -0.012    0.079   -0.006            0
always       0.067   0.087  0.053   -0.091    0.021           35
superiority  0.064   0.094  0.042    0.079    0.095            4
gates        0.074   0.098  0.057    0.087    0.094           32

The figure at the top accumulates the daily IC of the four. Before the change they move together (from 0.087 to 0.098 of mean IC). Afterwards, the fixed model drops to −0.012: it keeps betting on f1. The other three recover, at different speeds:

• Always reacts before anyone else, because every month it deploys the freshest model, but it also deploys the broken ones: −0.091 of IC in December 2023 (the inverted model) and 0.021 in February 2025 (the leaky one, which live does not have the information it learned from and bets on noise).
• Superiority deploys no broken model and only switches models four times in three years, but it is slow to adapt: after the change, 0.042.
• Gates has the best IC before (0.098) and after (0.057) the change: the freshness of “always” without its two bad months.

The difference between “always” and “gates” comes almost entirely from those two months: the same retraining, with and without checking the result. The September 2023 fault, on the other hand, barely costs anything: one month of frozen f1 in a 252-session window only dampens one coefficient. That a fault does not cost much in this lab does not mean it will not cost anything in another; the gate is there so you do not have to find out.

Why is the classic policy so slow? The table of the cycles around the change tells the story:

ic, reg, _ = res["superiority"]
tabla = pd.DataFrame(reg, columns=["cycle", "gate", "detail"])
tabla["cycle"] = tabla["cycle"].dt.date.astype(str)
tabla[tabla["cycle"].between("2024-03-01", "2025-01-31")]

         cycle            gate              detail
13  2024-03-01  no improvement  dIC +0.009, t +1.5
14  2024-04-01  no improvement  dIC +0.001, t +0.1
15  2024-05-01  no improvement  dIC +0.001, t +0.1
16  2024-06-03  no improvement  dIC +0.001, t +0.1
17  2024-07-01  no improvement  dIC +0.005, t +0.5
18  2024-08-01  no improvement  dIC +0.014, t +0.9
19  2024-09-02  no improvement  dIC +0.018, t +1.2
20  2024-10-01  no improvement  dIC +0.031, t +1.8
21  2024-11-01              ok  dIC +0.052, t +3.3
22  2024-12-02              ok  dIC +0.013, t +3.2
23  2025-01-01              ok  dIC +0.021, t +2.5

In April, May and June 2024 the candidate's training window is still almost entirely from the old regime (the validation takes up the last two months), so candidate and champion predict almost the same (0.001 of IC difference). From August on, the candidate starts to see the new regime: it improves by 0.014 (t = 0.9), by 0.018 (t = 1.2), by 0.031 (t = 1.8). It is already better, but with 42 validation sessions it cannot prove it until November: 0.052 with t = 3.3. Seven months after the change.

The delay has three components, and it pays to know which one dominates before trying to reduce it:

That is why superiority makes sense when the challenger is a different recipe: another algorithm, other features, another target. There the question “is it better?” is the right one and getting it wrong is costly, because a different model can fail in ways the champion does not. When the challenger is the same recipe with one more month of data, what was validated was the recipe (in the walk-forward of lesson 5.4.3, with its Deflated Sharpe from 5.4.4), and what needs checking every month is that this month's model is not broken. That is what the gates policy does.

The shadow test: the challenger with the real inputs

A model that passes the gates has still never run with the production pipeline: the cycle's snapshot is a copy, and live the features arrive from another process, at another time and sometimes in another format. The shadow test (lesson 4.5.3, applied to the model instead of to orders) runs the challenger in parallel with the champion for a few days, with the same inputs, and stores its predictions without trading on them.

What the shadow test checks is not whether it wins. With 50 assets, the standard deviation of a single day's IC is about 0.14 ($1/\sqrt{50}$); over five sessions, the mean IC has an error of about 0.06, larger than any plausible improvement. What it checks is that the model behaves:

c = pd.Timestamp("2024-11-01")
i = fechas.get_loc(c)
Fx, Rx = foto(c)
(e0, e1), _ = ventanas(c)
challenger = ajustar(Fx, Rx, e0, e1)
champ_vigente = champion_0           # superiority: still the initial one

def comprobar_sombra(champ, chal, a, b, max_rotacion=0.6):
    """Operational checks, not statistical ones: the IC of five
    sessions is noise."""
    pc, pp = predecir(chal, F, a, b), predecir(champ, F, a, b)
    top = lambda p: set(np.argsort(p)[-10:])   # top 10
    rot = np.mean([1 - len(top(pc[j]) & top(pc[j - 1])) / 10
                   for j in range(1, b - a)])
    return {"finite": bool(np.isfinite(pc).all()),
            "corr_champion": round(float(np.corrcoef(
                pc.ravel(), pp.ravel())[0, 1]), 2),
            "rel_dispersion": round(float(pc.std() / pp.std()), 2),
            "top10_turnover": round(float(rot), 2),
            "ok": bool(np.isfinite(pc).all() and rot < max_rotacion)}

comprobar_sombra(champ_vigente, challenger, i, i + 5)

{'finite': True, 'corr_champion': 0.77, 'rel_dispersion': 0.87, 'top10_turnover': 0.28, 'ok': True}

The November 2024 challenger gives finite predictions, with a correlation of 0.77 with the champion (they are similar, but not the same model: it has already learned something about the new regime), 13% lower dispersion and a turnover of the top ten assets of 28% per session. These are numbers to compare with the champion's at the time and with a fixed maximum: a model that changed half the portfolio every day would generate transaction costs that the validation, without costs, does not see.

The registry: one run per cycle and aliases

What remains of each cycle, in the MLflow SQLite store of the ops environment (lesson 7.3.1): a run with the parameters, the gate that stopped the candidate or ok, and the SHA-256 fingerprint of the data snapshot it was trained on. With the fingerprint you can check years later that a cycle reproduces: same snapshot, same model.

import hashlib
import mlflow
from mlflow import MlflowClient

def huella(Fx, Rx, a, b):
    """SHA-256 of the data snapshot used for training."""
    h = hashlib.sha256(Fx[a:b].tobytes())
    h.update(Rx[a:b].tobytes())
    return h.hexdigest()[:12]

mlflow.set_experiment("senal-n7")
for c, g, det in reg:
    Fx, Rx = foto(c)
    (e0, e1), _ = ventanas(c)
    with mlflow.start_run(run_name=f"ciclo-{c.date()}"):
        mlflow.log_params({"ventana": VENT, "validacion": VAL,
                           "alpha": 1.0, "ciclo": str(c.date())})
        mlflow.set_tags({"puerta": g, "detalle": det,
                         "datos_sha256": huella(Fx, Rx, e0, e1)})
runs = mlflow.search_runs(filter_string="tags.puerta != 'no improvement'",
                          order_by=["params.ciclo"])
print(len(mlflow.search_runs()), "runs")
runs[["params.ciclo", "tags.puerta", "tags.datos_sha256"]]

35 runs
  params.ciclo tags.puerta tags.datos_sha256
0   2023-09-01        data      86b09d034105
1   2023-12-01    behavior      7805683890e5
2   2024-11-01          ok      e3a71b5c0551
3   2024-12-02          ok      9e01f0587dad
4   2025-01-01          ok      eff27095c2e9
5   2025-02-03     leakage      91fbc679f221
6   2025-08-01          ok      132ff59aad40

Thirty-five runs, one per cycle; the query filters out the “no improvement” ones and leaves in view what matters in an audit: the three rejections with their reason and the four cycles that passed. Models are only registered when they are going to be deployed: registering 35 versions that nobody will use fills the registry with noise. The tag and parameter keys (puerta, gate; ciclo, cycle) are the registry's identifiers and stay as they are.

Going to production is an alias. In the MLflow registry, an alias is a name that points to a specific version of a registered model (champion to 1, challenger to 2), and the live process loads models:/senal-n7@champion without knowing which version it is. The fixed stages MLflow used before (Staging, Production) have been deprecated since version 2.9.0, and the documentation proposes exactly this use of aliases11Note 1 · MLflow, “Model Registry Workflows” (https://mlflow.org/docs/latest/ml/model-registry/workflow/): “As of MLflow 2.9.0, Model Stages have been deprecated and will be removed in a future major release”; the documentation's example creates a champion alias with set_registered_model_alias and loads the model with models:/<name>@champion. Accessed on October 23, 2026.:

NOMBRE = "senal-n7"
ejemplo = F[i].astype("float64")[:2]
versiones = {}
for etiqueta, modelo in (("inicial", champion_0),
                         ("2024-11-01", challenger)):
    with mlflow.start_run(run_name=f"modelo-{etiqueta}"):
        info = mlflow.sklearn.log_model(modelo, name="modelo",
                                        input_example=ejemplo)
    versiones[etiqueta] = mlflow.register_model(info.model_uri,
                                                NOMBRE).version
cli = MlflowClient()
cli.set_registered_model_alias(NOMBRE, "champion", versiones["inicial"])
cli.set_registered_model_alias(NOMBRE, "challenger",
                               versiones["2024-11-01"])

def promover(cli, nombre):
    """The challenger becomes champion; the champion, 'anterior'."""
    antes = cli.get_model_version_by_alias(nombre, "champion")
    nuevo = cli.get_model_version_by_alias(nombre, "challenger")
    cli.set_registered_model_alias(nombre, "anterior", antes.version)
    cli.set_registered_model_alias(nombre, "champion", nuevo.version)
    cli.delete_registered_model_alias(nombre, "challenger")

def marcha_atras(cli, nombre):
    antes = cli.get_model_version_by_alias(nombre, "anterior")
    cli.set_registered_model_alias(nombre, "champion", antes.version)

def alias(cli, nombre):
    al = cli.get_registered_model(nombre).aliases
    return dict(sorted(al.items()))

print("before:", alias(cli, NOMBRE))
promover(cli, NOMBRE)
print("promoted:", alias(cli, NOMBRE))
vivo = mlflow.sklearn.load_model(f"models:/{NOMBRE}@champion")
print("the live process loads the challenger:",
      np.allclose(vivo.predict(F[i]), challenger.predict(F[i])))
marcha_atras(cli, NOMBRE)
print("rollback:", alias(cli, NOMBRE))

Successfully registered model 'senal-n7'.
Created version '1' of model 'senal-n7'.
Registered model 'senal-n7' already exists. Creating a new version of this model...
Created version '2' of model 'senal-n7'.
before: {'challenger': 2, 'champion': 1}
promoted: {'anterior': 1, 'champion': 2}
the live process loads the challenger: True
rollback: {'anterior': 1, 'champion': 1}

Promoting means moving two aliases: champion moves to version 2 and version 1 stays as anterior (previous). The live process, which loaded by alias, receives the challenger without changing a line of code. Rolling back means moving champion to anterior: one second, without retraining or deploying anything. That is the property that justifies the whole registry. If the new model misbehaves on its first day, the decision to undo cannot wait for someone to find the previous model's file.

The full lifecycle of a version:

stateDiagram-v2
  [*] --> CANDIDATE
  CANDIDATE --> REJECTED: gate fails
  CANDIDATE --> CHALLENGER: gates ok
  CHALLENGER --> CHAMPION: shadow ok
  CHALLENGER --> REJECTED: shadow fails
  CHAMPION --> PREVIOUS: promotion
  PREVIOUS --> CHAMPION: rollback
  PREVIOUS --> ARCHIVED

What gets automated and what does not

The lesson's cycle decides on its own, and that is what you want for repetitive work. Some decisions should not be automated:

• The first deployment of a new recipe goes through full validation and through a person. The gates check that this month's model is not broken; they do not decide whether the recipe is good.
• Rejection alerts reach someone. A rejection leaves the champion trading on older data; three rejections in a row are a system that has stopped learning, and that is also a failure.
• Gate thresholds only change through a review and stay in the repository history.
• Shutting down the model, if its live performance collapses between two cycles, cannot wait for the next cycle: that is lesson 7.3.3.

Exercises

1. Repeat the superiority policy with a 126-session training window instead of 252. When is the first challenger promoted after the change? How many times are models switched in total?

VENT = 126
ic126, reg126, k126 = politica("superiority")
VENT = 252
print("promotions:", [str(c.date()) for c, g, _ in reg126
                      if g == "ok"])
print(f"mean IC {ic126.mean():.3f} · after the change "
      f"{ic126['2024-04':].mean():.3f}")
desde_oct = [g for c, g, _ in reg126 if c >= pd.Timestamp("2024-10")]
print("gates since October 2024:",
      pd.Series(desde_oct).value_counts().to_dict())

promotions: ['2024-03-01']
mean IC 0.034 · after the change -0.009
gates since October 2024: {'behavior': 14, 'leakage': 1}

2. A fourth fault: the target is computed with a one-day lag (the return from tomorrow to the day after tomorrow instead of from today to tomorrow). Write its snapshot and run it through the gates in the May 2025 cycle. Does any gate stop it?

def foto_retrasada(ciclo):
    Fx, _ = foto(ciclo)
    return Fx, np.roll(R, -1, axis=0)   # return from t+1 to t+2

c = pd.Timestamp("2025-05-01")
_, (_, v1_previo) = ventanas(pd.Timestamp("2025-04-01"))
champ = ajustar(F, R, v1_previo - VENT, v1_previo)  # April's model
Fx, Rx = foto_retrasada(c)
(e0, e1), _ = ventanas(c)
cand = ajustar(Fx, Rx, e0, e1)
for mejora in (False, True):
    print(*puertas(c, Fx, Rx, cand, champ, mejora))
print("coefficients x1000:", (cand.coef_ * 1e3).round(2),
      (champ.coef_ * 1e3).round(2))

ok corr. with champion +1.00
no improvement dIC -0.003, t -1.1
coefficients x1000: [-0.52  0.27  0.54  0.31 -0.07] [-0.66  0.36  0.6   0.32 -0.08]

3. A colleague proposes speeding up the superiority policy by comparing the challenger with the champion on the validation sessions and, if it wins, deploying it refitted on all the data. What gets validated and what gets deployed in that case?