---
title: "5.3.3 Meta-labeling: primary and secondary models"
description: "A primary model decides the side and a secondary one learns when it is right: filtering, precision versus recall and bet sizing from its probability."
url: "https://pythonparatrading.com/en/python-trading-course/level-5/meta-labeling-side-and-size/"
lang: "en"
image: "https://pythonparatrading.com/og/en/python-trading-course/level-5/meta-labeling-side-and-size.png"
---

# 5.3.3 · Meta-labeling: one model for the side and another for the size

> The FNX primary model is right 50.7% of the time: 60.4% in trend and 44.6% in reversion, but the regime is not observed. Meta-labeling trains a second model, with information available at the time of the event, that estimates the probability that the primary is right. This lesson builds it, measures how much of the regime it recovers, compares thresholds with economic metrics and turns the probability into a bet size.

- Course: [Level 5: Machine learning and deep learning (Keras/PyTorch)](https://pythonparatrading.com/en/python-trading-course/level-5/)
- Module: 5.3 · Labelling and sampling
- Duration: 70 min
- Prerequisites: [5.3.2 · Triple-barrier method: labels that mimic a trade](https://pythonparatrading.com/en/python-trading-course/level-5/triple-barrier-labeling-method/), [5.1.3 · Economic metrics for trading models: beyond accuracy](https://pythonparatrading.com/en/python-trading-course/level-5/economic-metrics-for-trading-models/), [4.2.2 · Kelly criterion and volatility targeting: how much to risk](https://pythonparatrading.com/en/python-trading-course/level-4/kelly-criterion-and-volatility-targeting/)

## By the end of this lesson

- Separate the decision on the side (primary model) from the decision on whether to trade and how much (secondary model) and build the meta-label with the triple barrier.
- Build features observable at the time of the event without later information and validate with a split by sessions that cuts no label.
- Compare the meta-model with a one-line baseline and with an oracle that knows the regime, with standard errors.
- Choose the threshold with the trade-off between precision and recall and with economic metrics, not with the F1.
- Turn the meta-model's probability into a bet size with 2Φ(z) − 1 and measure its effect at equal exposure.

## Two decisions, two models

A trading system makes two decisions on every signal: **which side** to take and **how much** to bet (zero included).
Most ML models mix them: a classifier predicts “up” or “down” and trades with a fixed size. **Meta-labeling**
(López de Prado, 2018, section 3.6) separates them:

- The **primary model** decides the side. It can be a technical rule, an econometric model, a fundamental analyst or
  an ML model. It is designed to have high **recall**: to propose almost all the opportunities, even if many of them
  fail.
- The **secondary model**, or meta-model, receives each proposal from the primary and estimates the probability that
  it is right. It does not decide the side: it decides whether to trade and with what size. Its job is to raise the
  **precision**.

```mermaid
flowchart TB
  D[Data up to t] --> P[Primary: side]
  P --> TB[Triple barrier]
  TB --> Y[Meta-label]
  D --> F[Features at t]
  F --> S[Secondary: p]
  Y --> S
  S --> Z[Trade and size]
```

The separation has three advantages. The primary can be simple and interpretable, and ML only acts as a filter, which
reduces how much it can overfit. The size comes from a probability, not from a separate rule. And the method works
with any primary, including a discretionary one: the secondary learns under what conditions that analyst is right.

The case in this lesson has a known answer. In FNX (level 5 **synthetic** data), the primary, a momentum triggered by
a CUSUM filter (lesson 5.3.5), is right 60.4% of the time in the trend regime and 44.6% in the reversion regime.
Overall, 50.7%: it does not win. A good meta-model should get close to what someone who knew the regime would do.

## The primary model and the meta-label

The data, the volatility and the two functions come from lessons 5.3.2 and 5.3.5, in compact form:

```python
import numpy as np
import pandas as pd

barras = pd.read_parquet("hf5_barras_1min.parquet")
verdad = pd.read_parquet("hf5_verdad.parquet")
n_ses = len(barras) // 390


def matriz(col):
    """Bar column as a session x minute matrix."""
    return barras[col].to_numpy().reshape(n_ses, 390)


C, V = matriz("cierre"), matriz("volumen")
VC = matriz("volumen_comprador")
tend = (verdad["regimen"] == "tendencia").to_numpy()
tend = tend.reshape(n_ses, 390)        # the truth: only for measuring
lr = np.diff(np.log(C), axis=1)
sig = (pd.Series(lr.ravel()).ewm(span=100).std()
       .to_numpy().reshape(n_ses, 389))
print(C.shape, f"median σ {1e4 * np.nanmedian(sig):.2f} bp")
```
```output
(247, 390) median σ 4.22 bp
```

```python
def eventos_cusum(lr, sig, k=4.0):
    """Symmetric CUSUM filter per session (lesson 5.3.5)."""
    filas = []
    for d in range(lr.shape[0]):
        s_pos = s_neg = 0.0
        for m in range(lr.shape[1]):
            h = k * sig[d, m]
            if not np.isfinite(h):
                continue
            s_pos = max(0.0, s_pos + lr[d, m])
            s_neg = min(0.0, s_neg + lr[d, m])
            if s_pos > h or s_neg < -h:
                filas.append((d, m + 1, 1 if s_pos > h else -1,
                              sig[d, m]))
                s_pos = s_neg = 0.0
    return pd.DataFrame(
        filas, columns=["sesion", "minuto", "lado", "sigma"])


def triple_barrera(C, ev, k=1.0, h=30):
    """Barriers ±k·σ·√h and vertical at h minutes (lesson 5.3.2)."""
    fin, ret = [], []
    for d, m, s in ev[["sesion", "minuto", "sigma"]].itertuples(
            index=False):
        ult = min(m + h, C.shape[1] - 1)
        tramo = np.log(C[d, m + 1:ult + 1] / C[d, m])
        golpes = np.flatnonzero(np.abs(tramo) >= k * s * np.sqrt(h))
        j = golpes[0] if len(golpes) else len(tramo) - 1
        fin.append(m + 1 + j)
        ret.append(float(tramo[j]) if len(tramo) else 0.0)
    return ev.assign(fin=fin, ret=ret)


ev = triple_barrera(C, eventos_cusum(lr, sig))
ev["y"] = (ev.lado * ev.ret > 0).astype(int)   # meta-label
ev["tendencia"] = tend[ev.sesion, ev.minuto]
print(len(ev), "events; primary hit rate:", round(ev.y.mean(), 3))
ev.groupby("tendencia")["y"].agg(["size", "mean"]).round(3)
```
```output
6550 events; primary hit rate: 0.507
           size   mean
tendencia
False      4010  0.446
True       2540  0.604
```

The **meta-label** `y` is 1 if the return up to the first touch of the triple barrier ($\pm\sigma\sqrt{30}$,
vertical at 30 minutes) has the sign of the primary's side. These are the README figures: 6,550 events and a hit rate
of 50.7%, 60.4% in the 2,540 events in trend and 44.6% in the 4,010 in reversion. The `tendencia` column comes from
`hf5_verdad.parquet` and will not be used as a feature: it serves to check what the meta-model learns.

## Features known at the event

The meta-model can only use what is known at the close of the event minute. The README describes four reference
features, all over the 30-minute window that ends at that close:

- `imb_lado`: the flow **imbalance** (aggressive buys minus sells, over volume) with the sign of the side. Positive
  when the flow pushes in favor of the trade. In FNX, in trend, the buying fraction has a persistent bias toward the
  direction of the episode; in both regimes, in addition, it follows the price of each minute.
- `imb_abs`: its absolute value, the intensity of the flow without direction.
- `vol_rel`: the logarithm of the window's volume relative to that of the same time slot in the previous 20 sessions.
  Comparing it with the same time slot removes the intraday U of volume.
- `rv_rel`: the logarithm of the window's realized volatility relative to the one the EWMA predicted.

```python
d, m = ev.sesion.to_numpy(), ev.minuto.to_numpy()
cv = np.cumsum(V, axis=1)               # cumulative volume
ci = np.cumsum(2 * VC - V, axis=1)      # buys minus sells


def ultimos_30(acum, d, m):
    """Sum of the 30 minutes ending at minute m (inclusive)."""
    antes = np.where(m >= 30, acum[d, np.maximum(m - 30, 0)], 0)
    return acum[d, m] - antes


vol30 = ultimos_30(cv, d, m)
imb30 = ultimos_30(ci, d, m) / vol30
# volume of the same time slot in the previous 20 sessions
hist = np.array([
    ultimos_30(cv, np.arange(max(0, s - 20), s),
               np.full(min(s, 20), mm)).mean() if s > 0 else np.nan
    for s, mm in zip(d, m)])
lc = np.log(C)
rv30 = np.array([np.sqrt((np.diff(lc[s, max(mm - 30, 0):mm + 1])**2)
                         .sum()) for s, mm in zip(d, m)])
X = pd.DataFrame({
    "imb_lado": ev.lado * imb30,          # in favor of the side: > 0
    "imb_abs": np.abs(imb30),
    "vol_rel": np.log(vol30 / hist),
    "rv_rel": np.log(rv30 / (ev.sigma * np.sqrt(np.minimum(m, 30)))),
})
X.describe().loc[["count", "mean", "std"]].round(3)
```
```output
       imb_lado   imb_abs   vol_rel    rv_rel
count  6550.000  6550.000  6528.000  6550.000
mean      0.033     0.055    -0.012     0.015
std       0.062     0.045     0.265     0.239
```

`np.cumsum` per session allows any window to be summed with one subtraction. The historical volume is computed with
the 20 **previous** sessions, so the events of the first session have no `vol_rel` (they are the 22 that are missing).
The imbalance in favor of the side is positive on average (0.033): it makes sense, because the CUSUM fires after a
move and the flow follows the price. The meta-model has to tell when that flow is persistent.

## Validation: two halves by sessions

Lesson 5.4.2 explains why cross-validation with overlapping labels needs purging. Here a simpler split is enough:
train on the first 123 sessions and evaluate on the next 124. No label crosses a session close, so no training label
shares minutes with a test one.

```python
ok = np.isfinite(X).all(axis=1).to_numpy()
mitad = n_ses // 2
ent = ok & (d < mitad)
pru = ok & (d >= mitad)
y, t = ev.y.to_numpy(), ev.tendencia.to_numpy()
print("training:", ent.sum(), " test:", pru.sum())
print("primary hit rate in the test set:", round(y[pru].mean(), 3))
print("events in trend in the test set:", round(t[pru].mean(), 3))
print("labels that cross sessions:", (ev.fin > 389).sum())
```
```output
training: 3241  test: 3287
primary hit rate in the test set: 0.501
events in trend in the test set: 0.402
labels that cross sessions: 0
```

In the second half the primary is right 50.1% of the time and 40.2% of the events fall in trend: that is the
population on which everything that follows is measured.

## The one-line baseline and the ceiling

Before training anything it is worth knowing how much the simplest rule that would occur to someone who has read the
README gives, and how much the one who knows the regime would give:

```python
def resumen(sel, nombre):
    """Fraction traded, precision and its standard error, mean in bp."""
    yy = y[pru][sel]
    r = (ev.lado * ev.ret).to_numpy()[pru][sel]
    return {"filtro": nombre, "N": int(sel.sum()),
            "fraccion": sel.mean(), "precision": yy.mean(),
            "error": np.sqrt(yy.mean() * (1 - yy.mean()) / len(yy)),
            "media_pb": 1e4 * r.mean()}


filas = [resumen(np.ones(pru.sum(), bool), "all"),
         resumen(X.imb_lado.to_numpy()[pru] > 0, "imb_lado > 0"),
         resumen(t[pru], "oracle: trend")]
pd.DataFrame(filas).set_index("filtro").round(3)
```
```output
                  N  fraccion  precision  error  media_pb
filtro
all            3287     1.000      0.501  0.009     0.467
imb_lado > 0   2301     0.700      0.535  0.010     1.885
oracle: trend  1323     0.402      0.597  0.013     4.925
```

The rule `imb_lado > 0` trades 70% of the events with 53.5% precision (standard error of 1.0 points) and a mean of
1.9 bp per trade. The oracle, which only trades in trend, reaches 59.7% and 4.9 bp: it is the ceiling. A meta-model
that beat it in the second half would be measuring noise or using information it does not have.

## The secondary model

A logistic regression with the standardized features (lesson 5.5.1), trained on the first half:

```python
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

meta = make_pipeline(StandardScaler(), LogisticRegression())
meta.fit(X[ent], y[ent])
p = meta.predict_proba(X[pru])[:, 1]
coef = meta[-1].coef_[0].round(3).tolist()
print("coefficients:", dict(zip(X.columns, coef)))
print("AUC against the hit:", round(roc_auc_score(y[pru], p), 3))
print("AUC against the regime:", round(roc_auc_score(t[pru], p), 3))
filas.insert(2, resumen(p > 0.5, "logistic p > 0.5"))
pd.DataFrame(filas).set_index("filtro").round(3)
```
```output
coefficients: {'imb_lado': 0.244, 'imb_abs': 0.028, 'vol_rel': 0.074, 'rv_rel': -0.011}
AUC against the hit: 0.575
AUC against the regime: 0.707
                     N  fraccion  precision  error  media_pb
filtro
all               3287     1.000      0.501  0.009     0.467
imb_lado > 0      2301     0.700      0.535  0.010     1.885
logistic p > 0.5  1729     0.526      0.545  0.012     2.502
oracle: trend     1323     0.402      0.597  0.013     4.925
```

The numbers match the README reference ones. The coefficient of the imbalance in favor of the side (0.244) dominates
the other three: relative volume contributes something (0.074) and realized volatility and unsigned intensity, almost
nothing. The AUC against the hit is 0.575, modest, but against the **true regime** it is 0.707: the model does not see
the regime and has learned to recognize it considerably better than it predicts each trade, because within each
regime the outcome is still noisy.

Trading only when $p > 0.5$, precision goes to 54.5% (error of 1.2 points) with half of the events, and the mean per
trade from 0.5 to 2.5 bp. It sits between the primary and the oracle, as it should, and barely above the one-line
rule: the difference between 53.5% and 54.5% cannot be told apart from noise.

The proof that it has learned the regime shows up by deciles of its probability:

```python
dec = pd.DataFrame({"p": p, "tendencia": t[pru], "acierto": y[pru],
                    "decil": pd.qcut(p, 10, labels=False) + 1})
dec.groupby("decil").mean().round(3).T
```
```output
decil         1      2      3      4      5      6      7      8      9      10
p          0.407  0.443  0.462  0.479  0.496  0.513  0.533  0.556  0.589  0.650
tendencia  0.286  0.195  0.235  0.246  0.316  0.348  0.407  0.503  0.705  0.784
acierto    0.401  0.432  0.503  0.429  0.498  0.448  0.520  0.527  0.647  0.605
```

*Fig. — The meta-model does not see the regime but learns it: in the top decile of its probability 78% of the events are in a trend, against 40% overall.* Bars by decile of the meta-model probability in the second half: share of events in a trend of 29%, 20%, 23%, 25%, 32%, 35%, 41%, 50%, 70% and 78% (decile 10, highlighted). A dashed line marks the 40% of all events.

In the low deciles, between 20% and 29% of the events are in trend; in the highest, 78%. Precision rises with them:
from 40% to 60-65%. And the probability is reasonably **calibrated**: in decile 10 the model says 0.65 and is right
60.5% of the time; in decile 1 it says 0.41 and is right 40.1% of the time. If it is going to be used as a size, that
calibration matters as much as the AUC; with tree models it is worth checking and correcting it if needed.

## Precision, recall and the threshold

The 0.5 threshold has nothing special about it. Raising it trades less and with more precision; lowering it, the
opposite. The figure in the header goes through all the thresholds; the table, some of them:

```python
yp = y[pru]
r = (ev.lado * ev.ret).to_numpy()[pru]
filas_u = []
for umbral in (0.45, 0.48, 0.50, 0.52, 0.55, 0.58):
    s = p > umbral
    prec, rec = yp[s].mean(), yp[s].sum() / yp.sum()
    filas_u.append({"umbral": umbral, "operadas": s.mean(),
                    "precision": prec, "recall": rec,
                    "F1": 2 * prec * rec / (prec + rec),
                    "media_pb": 1e4 * r[s].mean(),
                    "error_pb": 1e4 * r[s].std() / np.sqrt(s.sum())})
pd.DataFrame(filas_u).set_index("umbral").round(3)
```
```output
        operadas  precision  recall     F1  media_pb  error_pb
umbral
0.45       0.814      0.521   0.847  0.645     1.311     0.415
0.48       0.647      0.532   0.687  0.600     2.006     0.468
0.50       0.526      0.545   0.572  0.558     2.502     0.525
0.52       0.411      0.570   0.468  0.514     3.917     0.594
0.55       0.275      0.602   0.330  0.427     5.128     0.734
0.58       0.172      0.631   0.217  0.323     6.423     0.969
```

- With a 0.45 threshold, 81% is traded with 52.1%; with 0.55, 27.5% with 60.2%, already at the oracle's level but
  with less than a third of the events and a larger standard error (0.73 bp on the mean).
- The **F1** drops as the threshold rises: 0.645 with 0.45 and 0.427 with 0.55. Trading everything (recall 1,
  precision 0.501) would give the highest F1 of all, 0.668. The F1 weighs a missed winning trade the same as a losing
  trade taken; in trading, with costs, they do not weigh the same. The threshold is chosen with the economic metric
  (lesson 5.1.3): net mean per trade, number of trades and risk.
- The mean per trade rises from 1.3 to 6.4 bp, but the number of trades falls to a fifth. The net total, which is
  what gets paid, has an intermediate maximum.

<aside class="callout" data-kind="aviso">

Choosing the threshold by looking at the second half and then presenting the result of that same half is overfitting.
The threshold is set with the training data (or with a validation within them, lesson 5.4.3) and the second half is
looked at once. Here the table is shown in full to understand the trade-off, not to choose.

</aside>

## A more flexible model?

If the logistic regression learns the regime with four features, a nonlinear model might learn more:

```python
import lightgbm as lgb

gbm = lgb.LGBMClassifier(
    n_estimators=200, learning_rate=0.02, num_leaves=8,
    min_child_samples=50, subsample=0.8, subsample_freq=1,
    random_state=0, n_jobs=4, deterministic=True,
    force_row_wise=True, verbose=-1)
gbm.fit(X[ent], y[ent])
p_gbm = gbm.predict_proba(X[pru])[:, 1]
print("AUC against the hit:", round(roc_auc_score(y[pru], p_gbm), 3))
print("AUC against the regime:", round(roc_auc_score(t[pru], p_gbm), 3))
pd.DataFrame([resumen(p_gbm > 0.5, "LightGBM p > 0.5")]).set_index(
    "filtro").round(3)
```
```output
AUC against the hit: 0.567
AUC against the regime: 0.653
                     N  fraccion  precision  error  media_pb
filtro
LightGBM p > 0.5  1676      0.51      0.548  0.012     2.509
```

LightGBM (lesson 5.5.3), with 200 small trees, comes out **worse** than the logistic regression in AUC (0.567 versus
0.575) and in recognizing the regime (0.653 versus 0.707), and almost the same in precision with $p > 0.5$ (54.8%).
The relationship that matters here is almost linear in one feature; flexibility only adds variance. It is the usual
result with little signal: the simple baseline and the linear model are the bar a complex model has to clear on data
it has not seen, not on the training data.

## From probability to size

The meta-model's probability can decide the size, not just the yes or no. López de Prado's rule (2018, chapter 10)
treats $p$ as the result of a test of the hypothesis “the trade is a coin flip”:

$$
z = \frac{p - \tfrac{1}{2}}{\sqrt{p(1-p)}}, \qquad m = 2\,\Phi(z) - 1 \in [0, 1],
$$

with $\Phi$ the normal cumulative distribution function and $m = 0$ when $p \le 0.5$. The size is rounded to multiples
of a step so as not to change the position over minimal differences in probability:

```python
from scipy.stats import norm


def tamano(p, paso=0.05):
    """2·Φ(z) - 1 with z = (p - 1/2) / √(p(1 - p)); 0 if p ≤ 1/2;
    rounded to multiples of `paso`."""
    z = (p - 0.5) / np.sqrt(p * (1 - p))
    m = np.clip(2 * norm.cdf(z) - 1, 0, None)
    return np.round(m / paso) * paso


for q in (0.50, 0.55, 0.60, 0.65, 0.70):
    print(f"p = {q:.2f}  size = {tamano(np.array([q]))[0]:.2f}")
tam = tamano(p)
print("mean size:", round(tam.mean(), 3), " with size > 0:",
      (tam > 0).sum())
```
```output
p = 0.50  size = 0.00
p = 0.55  size = 0.10
p = 0.60  size = 0.15
p = 0.65  size = 0.25
p = 0.70  size = 0.35
mean size: 0.056  with size > 0: 1441
```

The rule is cautious: with $p = 0.6$ it bets 0.15 of the maximum size and with 0.7 only 0.35. With the FNX
probabilities, almost all between 0.40 and 0.65, the mean size is 0.056 and 1,441 events receive a size greater than
zero. To compare it with the filter it has to be brought to the **same total exposure**: multiply the sizes by 9.4.
Lesson 4.2.2 explains why the absolute size is a separate risk decision.

## The economic result in the second half

The P&L of each trade is the size times the side's return up to the exit, minus a cost. The assumed cost is 0.6 bp
per round-trip trade and unit of size: one FNX tick is about 0.47 bp at the year's average price (a one-tick bid-ask
spread, paid half on entry and half on exit) plus a commission. It is an assumption, not market data; lesson 2.4.7
explains how to model it.

```python
COSTE = 0.6e-4        # per round-trip trade and unit of size
ses_pru = d[pru]


def pyg(tam_op):
    """Net P&L per trade and annualized Sharpe of the daily P&L."""
    neto = tam_op * r - COSTE * tam_op
    diario = pd.Series(neto).groupby(ses_pru).sum()
    sr = diario.mean() / diario.std() * np.sqrt(252)
    return {"exposicion": tam_op.sum(),
            "bruto_pb_por_unidad": 1e4 * (tam_op * r).sum()
            / tam_op.sum(),
            "neto_total_pb": 1e4 * neto.sum(), "sharpe": sr,
            "error_sharpe": np.sqrt((1 + sr**2 / 504) / len(diario)
                                    * 252)}


escala = (p > 0.5).sum() / tam.sum()   # same total exposure
res = pd.DataFrame({
    "all": pyg(np.ones(len(p))),
    "logistic p > 0.5": pyg((p > 0.5).astype(float)),
    "size 2Φ(z) - 1": pyg(tam * escala),
    "oracle": pyg(t[pru].astype(float)),
}).T
print(f"size scale: x{escala:.1f}")
res.round(2)
```
```output
size scale: x9.4
                  exposicion  bruto_pb_por_unidad  neto_total_pb  sharpe  error_sharpe
all                   3287.0                 0.47        -436.04   -0.45          1.43
logistic p > 0.5      1729.0                 2.50        3288.91    3.75          1.45
size 2Φ(z) - 1        1729.0                 4.51        6768.89    5.12          1.46
oracle                1323.0                 4.93        5722.38    8.01          1.51
```

*Fig. — Cumulative net P&L in the second half (0.6 bp cost per trade and unit): the primary alone loses; filtered by the meta-model it wins, and sized by its probability at the same total exposure it wins more. Planted effect on synthetic data.* Four cumulative P&L lines in basis points over 124 sessions. All primary trades end at −436 bp. With the logistic filter (highlighted), 3,289 bp. With the 2Φ(z) − 1 size scaled to the same exposure, 6,769 bp. The oracle trading only in trends, with less exposure, 5,722 bp.

- Trading everything loses: 0.47 bp gross per trade does not pay for 0.6 of cost.
- Filtering with $p > 0.5$ earns 2.5 bp gross per trade and 3,289 bp net over 124 sessions, with an annualized Sharpe
  of 3.75.
- Sizing with $2\Phi(z) - 1$, at the same total exposure, earns 4.5 bp per unit of size, very close to the oracle
  (4.9), and 6,769 bp net: it concentrates the size on the clearest events.
- The oracle trades less (1,323 units) with the best result per unit and a Sharpe of 8.0.

Two warnings. The standard error of each Sharpe is about 1.45 with 124 sessions: the difference between the filter
and the sizing (3.75 versus 5.12) is one standard error and does not allow choosing between them with this sample.
And the regime effect is **planted** with a large size so that the method shows: in a real market the signal is
weaker, it changes over time and others exploit it. What the lesson shows is that meta-labeling recovers a structure
that exists; not that it exists in any real future.

## Common mistakes

- **Features with later information.** Any window that includes the minute after the event, or a full “daily”
  volume, leaks the outcome. In the meta-model it is especially easy, because the features are computed after the
  events are known.
- **Training the secondary with the same labels the primary was tuned on.** If the CUSUM threshold or the barriers
  were chosen by looking at the hit rate, the secondary learns on a selected sample.
- **Using the probability as a size without checking calibration.** A poorly calibrated probability gives
  systematically large or small sizes.
- **Measuring with accuracy or F1.** The goal is economic; the meta-model can worsen the F1 and improve the result.
- **Forgetting the overlap.** The meta-labels of nearby events share minutes; cross-validation within the training set
  needs purging and embargo (5.4.2), and bagging needs uniqueness weights (5.3.4).

## Exercises

1. Swap the halves: train the meta-model on the second and evaluate it on the first. Do the AUC and the precision
   with $p > 0.5$ hold?

**Solution.**

```python
meta2 = make_pipeline(StandardScaler(), LogisticRegression())
meta2.fit(X[pru], y[pru])
p2 = meta2.predict_proba(X[ent])[:, 1]
print("AUC against the hit:", round(roc_auc_score(y[ent], p2), 3))
print("AUC against the regime:", round(roc_auc_score(t[ent], p2), 3))
s2 = p2 > 0.5
print(f"traded {s2.mean():.3f}  precision {y[ent][s2].mean():.3f}"
      f"  (all: {y[ent].mean():.3f})")
```
```output
AUC against the hit: 0.574
AUC against the regime: 0.674
traded 0.535  precision 0.564  (all: 0.513)
```

With the halves swapped the result is almost the same: an AUC of 0.574 against the hit and 0.674 against the regime,
and a precision of 56.4% trading 53.5% of the events, versus 51.3% for all of those in the first half. That the
improvement holds in both directions is a cheap stability check; it does not replace a validation with retraining
(lesson 5.4.3), but it rules out that the result depends on a lucky half.

2. Add as a feature whether the previous event in the same session had the same side (`+1`, `−1` or `0` if it is the
   first). In trend, consecutive events tend to repeat the side. Does the out-of-sample AUC improve?

**Solution.**

```python
misma = (ev.sesion == ev.sesion.shift()).to_numpy()
prev = np.where(misma, ev.lado * ev.lado.shift().fillna(0), 0)
print("same side as the previous one, in trend / reversion:",
      round((prev[t] == 1).mean(), 3), round((prev[~t] == 1).mean(), 3))
X5 = X.assign(prev_igual=prev)
meta5 = make_pipeline(StandardScaler(), LogisticRegression())
meta5.fit(X5[ent], y[ent])
p5 = meta5.predict_proba(X5[pru])[:, 1]
print("AUC with the feature:", round(roc_auc_score(y[pru], p5), 3),
      " without it:", round(roc_auc_score(y[pru], p), 3))
```
```output
same side as the previous one, in trend / reversion: 0.6 0.434
AUC with the feature: 0.574  without it: 0.575
```

In trend, 60% of the events repeat the side of the previous one; in reversion, 43.4%. The feature has information,
but the AUC does not improve (0.574 versus 0.575): what it says is already said by the imbalance in favor of the side,
which also measures the persistence of the flow. Adding features that repeat the same information does not help and
adds variance; lesson 5.2.5 shows how to measure it with importance by groups of correlated features.

3. An asset manager has an analyst who recommends buying or selling sector indices once a week. How would you apply
   meta-labeling to their recommendations and what would you be especially careful about?

**Solution.**

The analyst is the primary model: their recommendations give the side. Each recommendation is labeled with a triple
barrier consistent with how the position would be managed (for example, ±1.5 weekly standard deviations and a
vertical at four weeks) and the meta-label is whether it won. The secondary uses only information available on the
date of each recommendation: volatility regime, sector momentum, dispersion, time since the last recommendation, even
the analyst's recent track record. Three things need care. Few observations: with 52 recommendations a year, a linear
model with few features and a validation with purging (four-week labels overlap). That the analyst changes: their
process is not stationary, and the meta-model must be retrained and monitored (5.10.2). And that the date of each
recommendation is the real one, not that of its later recording.

## Self-assessment

1. What exactly does the secondary model predict in meta-labeling?

   - A. Whether the price will go up or down
   - B. The market regime
   - C. The optimal width of the barriers
   - D. The probability that the trade proposed by the primary model, with its side, is a winner according to the triple barrier

   *Answer D: The probability that the trade proposed by the primary model, with its side, is a winner according to the triple barrier.* The primary sets the side; the meta-label is 1 if its trade is right and 0 if not. The secondary decides whether to trade and how much. That its probability ends up resembling the regime (AUC of 0.707 against it) is a consequence, not its goal.

2. In the second half, the rule `imb_lado > 0` trades 70% of the events with a precision of 53.5% and the logistic regression (p > 0.5) 52.6% with 54.5%, with standard errors of 1.0 and 1.2 points. What do you conclude?

   - A. That the logistic regression is much better
   - B. That almost all the improvement comes from one feature and a one-line rule; the logistic regression adds little and its advantage over the rule is within the noise
   - C. That the rule is better because it trades more
   - D. That meta-labeling does not work

   *Answer B: That almost all the improvement comes from one feature and a one-line rule; the logistic regression adds little and its advantage over the rule is within the noise.* The simple baseline is the yardstick: the standardized coefficient of the imbalance in favor of the side (0.244) dominates the other three. A LightGBM with the same features gives an AUC of 0.567, worse than the logistic regression (0.575).

3. With the threshold at 0.5, the meta-model's F1 is 0.558; that of the primary alone (trade everything: recall 1, precision 0.501) is 0.668. Why is that not an argument against the meta-model?

   - A. Because the F1 weighs precision and recall equally, and in trading missing doubtful trades costs little: net P&L goes from −436 to +3,289 bp in the second half
   - B. Because the F1 is miscalculated
   - C. Because the F1 is only valid for three classes
   - D. Because recall never matters

   *Answer A: Because the F1 weighs precision and recall equally, and in trading missing doubtful trades costs little: net P&L goes from −436 to +3,289 bp in the second half.* Trading everything maximizes recall with coin-flip precision; with costs, that loses. The metric that decides is the economic one (lesson 5.1.3): net result per trade and Sharpe with its error.

4. With p = 0.60, what size does the 2Φ(z) − 1 rule give with z = (p − 0.5)/√(p(1 − p))?

   - A. 0.60
   - B. 1, because p > 0.5
   - C. About 0.16 (0.15 after rounding to multiples of 0.05): the rule is cautious with probabilities close to 0.5
   - D. 0.20

   *Answer C: About 0.16 (0.15 after rounding to multiples of 0.05): the rule is cautious with probabilities close to 0.5.* z = 0.1/0.49 = 0.204 and 2Φ(0.204) − 1 = 0.162. In the second half the mean size is 0.056: to compare it with the filter it has to be scaled to the same total exposure (×9.4).

5. The meta-model gives a Sharpe of 3.75 in the second half (standard error of 1.45). How should it be read?

   - A. As the expected return of the strategy in a real future
   - B. As an overfitted result that must be discarded
   - C. As the maximum possible: the oracle gives less
   - D. As confirmation that the method recovers an effect planted in synthetic data, with a huge standard error from having 124 sessions, no slippage and no competition

   *Answer D: As confirmation that the method recovers an effect planted in synthetic data, with a huge standard error from having 124 sessions, no slippage and no competition.* The regime effect is planted with a known size; the oracle that knows it gives a Sharpe of 8.0. In a real market the effects are smaller, they change and others compete for them. And six months of data leave an interval of ±2.9 on the Sharpe.

## Key points

- Meta-labeling separates two decisions: the primary chooses the side with high recall; the secondary estimates the probability that it is right and decides whether to trade and how much.
- The meta-label comes from the triple barrier with the primary's side: 1 if the trade wins. The features have to be known at the close of the event minute.
- In FNX the meta-model goes from 50.1% precision to 54.5% trading half of the events (the regime oracle, 59.7%), with an AUC of 0.575 against the hit and 0.707 against the regime it does not see.
- The one-line baseline (`imb_lado > 0`) gives 53.5%: almost all the improvement comes from one feature. A LightGBM does not improve on it.
- The threshold is chosen with economic metrics: the F1 drops, but net P&L goes from −436 to +3,289 bp in the second half.
- The probability is turned into a size with 2Φ(z) − 1: at equal exposure it improves the result per unit (4.5 bp versus 2.5), with a Sharpe whose standard error (1.45) prevents telling the variants apart.

## Resources

### Research

- [Alpha Factors and Alpha Trading in Python](https://pythonparatrading.com/en/alpha-factors-and-alpha-trading-in-python/) · Research-0007 · 5 min

### Documentation

- [scikit-learn: LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) · scikit-learn.org
- [scikit-learn: precision_recall_curve](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_curve.html) · scikit-learn.org
- [scikit-learn: probability calibration](https://scikit-learn.org/stable/modules/calibration.html) · scikit-learn.org

### Books

- [Marcos López de Prado: Advances in Financial Machine Learning (Wiley, 2018), chapter 3 (3.6, “Meta-Labeling”) and chapter 10, “Bet Sizing”](https://www.wiley.com/en-us/Advances+in+Financial+Machine+Learning-p-9781119482086) · wiley.com

### Data

- [Level 5 data: FNX 1-minute bars (Parquet)](https://pythonparatrading.com/curso/datos/nivel-5/hf5_barras_1min.parquet) · Download · hf5_barras_1min.parquet
- [Level 5 data: true FNX regime minute by minute (Parquet)](https://pythonparatrading.com/curso/datos/nivel-5/hf5_verdad.parquet) · Download · hf5_verdad.parquet

Previous lesson: [5.3.2 · Triple-barrier method: labels that mimic a trade](https://pythonparatrading.com/en/python-trading-course/level-5/triple-barrier-labeling-method/)

Next lesson: [5.3.4 · Label uniqueness, overlap weights and the sequential bootstrap](https://pythonparatrading.com/en/python-trading-course/level-5/label-uniqueness-weights-sequential-bootstrap/)