Lesson 2.6.2 · 65 min

Win rate, payoff, expectancy, alpha, beta, information ratio, turnover and capacity

The metrics in lesson 2.6.1 look at the equity curve. These look at its parts: each trade, the share of the return the market explains, and what it costs to move the portfolio. With a control that has no edge, wins 63% of the time and loses money, and a placebo that clears the t = 2 bar.

Fig. — Win rate against payoff, with iso-expectancy curves (in units of the average loss L): the reversion rule wins most often and is the only one that loses money
Level 2 lessons
  1. 2.1 · Statistics and econometrics for trading

    1. 2.1.1 Hypothesis tests, p-values, bootstrap and regression with robust standard errors
    2. 2.1.2 Stationarity: ADF, KPSS, differencing and fractional differencing
    3. 2.1.3 ARMA and ARIMA models: autocorrelation (ACF) and partial autocorrelation (PACF)
    4. 2.1.4 Volatility models: ARCH, GARCH, EGARCH and GJR with the arch library
    5. 2.1.5 Cointegration: Engle-Granger, Johansen and the half-life of mean reversion
    6. 2.1.6 Kalman filter for dynamic hedge ratios
    7. 2.1.7 Regime detection: hidden Markov models and structural breaks
    8. 2.1.8 The multiple testing problem: Bonferroni, FDR and the “factor zoo”
  2. 2.2 · Anatomy of a trading system

    1. 2.2.1 Edge hypotheses: risk, behavior, structure and information
    2. 2.2.2 Universe, signal, filter, entry, exit, sizing and risk management
    3. 2.2.3 Frequencies: intraday, swing and position trading, and the capacity of a strategy
    4. 2.2.4 Research journal and experiment tracking against data snooping
  3. 2.3 · Strategy families

    1. 2.3.1 Trend following and momentum
    2. 2.3.2 Mean reversion: RSI and Bollinger, intraday reversal and gaps
    3. 2.3.3 Pairs trading and statistical arbitrage: distance, cointegration and Kalman
    4. 2.3.4 Carry in currencies, futures and bonds; seasonality and calendar effects
    5. 2.3.5 Factors (value, quality, low volatility, size) and event strategies
    6. 2.3.6 Introduction to market making: inventory, spread and adverse selection (Avellaneda-Stoikov)
    7. 2.3.7 Crypto-specific strategies: momentum, funding arbitrage and basis
  4. 2.4 · Backtesting: engines and paradigms

    1. 2.4.1 Vectorized backtest from scratch: matrices, lag, costs and limits
    2. 2.4.2 Event-driven backtest from scratch: loop, portfolio, orders and fills
    3. 2.4.3 vectorbt: massive parameter sweeps and portfolio simulation
    4. 2.4.4 backtesting.py and backtrader: two teaching engines and their status
    5. 2.4.5 NautilusTrader: architecture, data, strategies and the same code in backtest and live
    6. 2.4.6 LEAN/QuantConnect and Zipline Reloaded; bt for asset allocation
    7. 2.4.7 Cost modeling: commissions, bid-ask spread, impact, financing and securities borrowing
    8. 2.4.8 Fill modeling: limit orders, queue, partial fills, gaps and stops
  5. 2.5 · Biases and validation

    1. 2.5.1 Backtest biases: look-ahead, survivorship, selection and narrative
    2. 2.5.2 In-sample and out-of-sample: anchored and rolling walk-forward
    3. 2.5.3 Combinatorial purged cross-validation for strategies
    4. 2.5.4 Deflated Sharpe Ratio and Probability of Backtest Overfitting (PBO)
    5. 2.5.5 Monte Carlo: trade resampling, block bootstrap and signal permutations
    6. 2.5.6 Parameter sensitivity: plateaus versus peaks, robustness across assets and periods
    7. 2.5.7 Synthetic data for stress testing: GBM, GARCH and bootstrapping
    8. 2.5.8 “Honest backtest” checklist before moving to paper trading
  6. 2.6 · Performance metrics

    1. 2.6.1 CAGR, Sharpe, Sortino, Calmar, Omega, drawdown and Ulcer Index
    2. 2.6.2 Win rate, payoff, expectancy, alpha, beta, information ratio, turnover and capacity
    3. 2.6.3 Reports with QuantStats or pyfolio-reloaded and custom tearsheets

From the curve to the trade list

Lesson 2.6.1 summarized three equity curves. A curve is the result of many decisions, and to know what it is made of you have to look at them one by one. This lesson works at three levels: the trades (how much is won and lost on each one), the portfolio against an index (which part of the return is market and which is not) and execution (how much the portfolio turns over and how much capital it can take).

Daily positionsTradesWin rate pAverage W and LExpectancyProfit factorReturnAlpha and betaTurnoverCapacity
Mermaid source
flowchart TB
  P[Daily positions] --> O[Trades]
  O --> WR[Win rate p]
  O --> WL[Average W and L]
  WR --> E[Expectancy]
  WL --> E
  WL --> PF[Profit factor]
  P --> R[Return]
  R --> AB[Alpha and beta]
  P --> T[Turnover]
  T --> C[Capacity]

We start by rebuilding the two strategies of lesson 2.6.1 on the level 2 synthetic data: TSMOM on 24 futures and the ten-currency carry, from 1 February 2005 to 2025. The effects are planted (a momentum term in each future and a carry that earns 70% of the rate differential with five crashes): the figures teach how to measure, they do not say that those strategies work outside this data.

IN [01]python
import numpy as np
import pandas as pd

fut = pd.read_parquet("futuros2_continuos.parquet")
px = fut.pivot(index="fecha", columns="mercado",
               values="cierre_ajustado")
r_fut = px.pct_change()
vol = r_fut.ewm(span=60, min_periods=60).std() * np.sqrt(252)
senal = np.sign(px / px.shift(252) - 1)
peso = (senal * 0.40 / px.shape[1] / vol).shift(1)

fx = pd.read_parquet("fx2_carry.parquet")
tabla = lambda c: fx.pivot(index="fecha", columns="divisa", values=c)
spot, dif = tabla("spot"), tabla("tipo_divisa") - tabla("tipo_usd")
dias = spot.index.to_series().diff().dt.days
interes = dif.shift(1) / 100 * dias.to_numpy()[:, None] / 360
r_fx = spot.pct_change() + interes
fin_mes = dif.index.to_series().groupby(dif.index.to_period("M")).max()
rango = dif.loc[fin_mes].rank(axis=1)
w_fx = ((rango >= 8).astype(float) - (rango <= 3).astype(float)) / 3
w_fx = w_fx.reindex(dif.index).ffill().shift(1)

inicio = "2005-02-01"
peso, r_fut = peso.loc[inicio:], r_fut.loc[inicio:]
w_fx, r_fx = w_fx.loc[inicio:], r_fx.loc[inicio:]
print(peso.shape, w_fx.shape)
OUT [01]TSMOM and carry from lesson 2.6.1
(5263, 24) (5457, 10)

Win rate, payoff and expectancy

A TSMOM trade is a run of positions with the same sign in one market: it starts when the signal changes sign and ends when it changes again. Its result is the sum of what it contributes to the portfolio each day, as a fraction of capital. The size of each position changes with volatility, but the trade is the decision to be long or short.

IN [02]python
def operaciones(peso, r):
    """One trade: a run of the same sign in one market."""
    aporte = peso * r       # return contributed to the portfolio
    filas = []
    for c in peso.columns:
        lado = np.sign(peso[c]).fillna(0)
        racha = (lado != lado.shift()).cumsum()
        d = pd.DataFrame({"lado": lado, "pnl": aporte[c],
                          "racha": racha})
        d = d[d["lado"] != 0].reset_index()
        t = d.groupby("racha").agg(
            entrada=("fecha", "first"), salida=("fecha", "last"),
            lado=("lado", "first"), pnl=("pnl", "sum"),
            sesiones=("pnl", "size"))
        filas.append(t.assign(mercado=c))
    return pd.concat(filas, ignore_index=True)

op_tsmom = operaciones(peso, r_fut)
print(len(op_tsmom), "trades")
op_tsmom.sort_values("pnl").iloc[[0, -1]]
OUT [02]TSMOM trade list
2576 trades
      entrada     salida  lado       pnl  sesiones mercado
1  2005-03-24 2005-08-24  -1.0 -0.021429       107     AGA
69 2019-04-03 2024-12-02  -1.0  0.123242      1427     AGA

That is 2,576 trades in 24 markets, about 107 per market over 21 years. The two extremes are both in the fictitious corn contract AGA: the worst takes 2.1% off capital (short from March to August 2005) and the best adds 12.3% (short from April 2019 to December 2024, 1,427 sessions, almost six years). Trades still open on 31 December 2025 and those that started before February 2005 count only for their part inside the sample.

From the list we compute four metrics. If pp is the fraction of winning trades (win rate), WW the average gain of the winners and LL the average loss of the losers (as a positive number):

payoff=WL,E=p W−(1−p) L,PF=∑winnerspnl∣∑loserspnl∣=p W(1−p) L\text{payoff} = \frac{W}{L}, \qquad E = p\,W - (1 - p)\,L, \qquad PF = \frac{\sum_{\text{winners}} \text{pnl}}{\left|\sum_{\text{losers}} \text{pnl}\right|} = \frac{p\,W}{(1 - p)\,L}

The expectancy EE is what is earned on average per trade. The profit factor PFPF compares what is won with what is lost: it is greater than 1 if and only if E>0E > 0. Setting EE to zero gives the break-even win rate, the minimum needed not to lose with a given payoff:

p∗=11+W/Lp^{*} = \frac{1}{1 + W/L}

For the carry, which rebalances at each month-end, the natural trade is the month: each month the six-currency basket is opened (in practice, with one-month forwards) and closed the next.

IN [03]python
def metricas_op(pnl):
    gana, pierde = pnl[pnl > 0], pnl[pnl <= 0]
    p = len(gana) / len(pnl)
    W, L = gana.mean(), -pierde.mean()
    return {"operaciones": len(pnl), "win_rate": p,
            "ganancia_media_pb": 1e4 * W,
            "perdida_media_pb": 1e4 * L, "payoff": W / L,
            "expectativa_pb": 1e4 * (p * W - (1 - p) * L),
            "profit_factor": gana.sum() / -pierde.sum(),
            "win_rate_equilibrio": 1 / (1 + W / L)}

aporte_fx = (w_fx * r_fx).sum(axis=1)
mes_carry = aporte_fx.groupby(aporte_fx.index.to_period("M")).sum()
pd.DataFrame({"tsmom": metricas_op(op_tsmom["pnl"]),
              "carry_mes": metricas_op(mes_carry)}).round(3)
OUT [03]Win rate, payoff, expectancy and profit factor
                        tsmom  carry_mes
operaciones          2576.000    251.000
win_rate                0.330      0.574
ganancia_media_pb      72.069    227.469
perdida_media_pb       20.389    223.261
payoff                  3.535      1.019
expectativa_pb         10.083     35.325
profit_factor           1.738      1.371
win_rate_equilibrio     0.221      0.495

TSMOM wins only 33.0% of its trades, but each winner is worth on average 3.54 times what each loser costs: its break-even win rate is 22.1% and it clears it comfortably. The carry wins 57.4% of months with a payoff of 1.02; its break-even is at 49.5%. Both have positive expectancy (10.1 bp per trade and 35.3 bp per month) and a profit factor above 1 (1.74 and 1.37). They are two opposite ways of making money.

The average hides how the profit is distributed. Sorting the trades from best to worst:

IN [04]python
def concentracion(pnl):
    orden = pnl.sort_values(ascending=False)
    n = len(pnl) // 10
    return {"mejor_10%": orden.iloc[:n].sum() / pnl.sum(),
            "peor_10%": orden.iloc[-n:].sum() / pnl.sum(),
            "sin_extremos": orden.iloc[n:-n].sum() / pnl.sum()}

pd.DataFrame({"tsmom": concentracion(op_tsmom["pnl"]),
              "carry_mes": concentracion(mes_carry)}).round(2)
OUT [04]Where the money comes from
              tsmom  carry_mes
mejor_10%      2.12       1.45
peor_10%      -0.63      -1.60
sin_extremos  -0.49       1.15
Fig. — Where the profit comes from: contribution of the best, middle and worst trades to the total profit of TSMOM and FX carry

In TSMOM, the best 10% of trades contributes 212% of the total profit: without them, the strategy loses. The middle 80% subtracts 49% and the worst 10%, 63%. It is the shape of a trend follower: it lives on a few long trends, and any rule that trims the winners (a profit target, a filter that skips some signals) takes away exactly what sustains it. The carry is the mirror image: the middle 80% of months makes money (115% of the total) and the worst 10% takes 160%, the crash months. A strategy with negative skewness seems to work every month until the month it does not.

Why the win rate is misleading

A high win rate is easy to manufacture: just put the profit target close and the stop far away. To see it we need a strategy with no edge, and the level 2 data has one: in the stocks of the fictitious universe, short-term reversion has no planted effect (it is a control). The rule buys at the close after a fall of more than 2 standard deviations, and exits at the close as soon as the cumulative gain exceeds +1σ (target) or the loss −3σ (stop), or after 10 sessions.

IN [05]python
aj = pd.read_parquet("universo2_ajustados.parquet")
pa = aj.pivot(index="fecha", columns="ticker", values="cierre_ajustado")
lp = np.log(pa)
sig = lp.diff().ewm(span=60, min_periods=60).std().shift(1)

def reversion(objetivo=1.0, stop=3.0, max_ses=10, caida=2.0):
    """Buy at the close after a fall of `caida` sigmas; exit at the
    close on hitting +objetivo or -stop sigmas, or after max_ses sessions."""
    filas = []
    for tk in pa.columns:
        x = lp[tk].dropna()
        v, s = x.to_numpy(), sig[tk].reindex(x.index).to_numpy()
        i = 61
        while i < len(v) - max_ses:
            if v[i] - v[i - 1] < -caida * s[i]:
                for k in range(1, max_ses + 1):
                    c = v[i + k] - v[i]
                    if (c >= objetivo * s[i] or c <= -stop * s[i]
                            or k == max_ses):
                        break
                filas.append((tk, x.index[i], k, np.expm1(c)))
                i += k
            i += 1
    return pd.DataFrame(filas, columns=["ticker", "entrada",
                                        "sesiones", "pnl"])

op_rev = reversion()
op_rev.head(3)
OUT [05]Reversion with no edge: close target, distant stop
  ticker    entrada  sesiones       pnl
0   QXAC 2006-05-01         4  0.008333
1   QXAC 2006-06-27         4 -0.035329
2   QXAC 2006-07-05         5  0.023794

In a random walk with no drift, a target at +a+a and a stop at −b-b is hit first with probability b/(a+b)b/(a+b): with 1σ and 3σ, the target wins 75% of the time. But the gain is three times smaller than the loss, so the expectancy is zero: the rule redistributes the result, it does not create it. With the data and its costs:

IN [06]python
ch = pd.read_parquet("universo2_caracteristicas.parquet")
spr = ch.pivot(index="fecha", columns="ticker", values="horquilla_pb")
spr.index = spr.index.to_period("M")
mes_ant = op_rev["entrada"].dt.to_period("M") - 1
coste = [spr.at[m, t] if m in spr.index else np.nan
         for m, t in zip(mes_ant, op_rev["ticker"])]
op_rev["coste"] = pd.Series(coste).fillna(spr.stack().median()) / 1e4

pd.DataFrame({"bruto": metricas_op(op_rev["pnl"]),
              "neto": metricas_op(op_rev["pnl"] - op_rev["coste"])}
             ).round(3)
OUT [06]With each stock's bid-ask spread
                         bruto       neto
operaciones          12633.000  12633.000
win_rate                 0.634      0.633
ganancia_media_pb      321.795    315.899
perdida_media_pb       620.908    627.096
payoff                   0.518      0.504
expectativa_pb         -23.333    -29.709
profit_factor            0.897      0.871
win_rate_equilibrio      0.659      0.665

The rule wins 63.4% of 12,633 trades (less than the theoretical 75%, because many exit on time and closes jump past the thresholds) with a payoff of 0.52. Its break-even win rate is 65.9%: it falls 2.5 points short and loses 23.3 bp per trade before costs. Each stock's bid-ask spread (the one for the month before entry, from universo2_caracteristicas.parquet) adds 6.4 bp, down to −29.7 bp.

Why does it lose even without costs, if short-term reversion has no planted effect? Falls of more than 2σ are not spread evenly over time: they cluster in the stress periods of the script (the crises with fixed dates in the data README; eight since 2006), during which the market has a very negative drift for weeks.

IN [07]python
tramos = [("2008-09-02", "2009-03-09"), ("2010-04-26", "2010-07-05"),
          ("2011-07-25", "2011-10-03"), ("2015-08-17", "2016-02-11"),
          ("2018-10-01", "2018-12-25"), ("2020-02-20", "2020-03-23"),
          ("2022-01-04", "2022-10-12"), ("2025-02-20", "2025-04-08")]
en_estres = np.zeros(len(op_rev), dtype=bool)
for a, b in tramos:
    en_estres |= op_rev["entrada"].between(a, b).to_numpy()
pd.DataFrame({"fuera": metricas_op(op_rev["pnl"][~en_estres]),
              "estres": metricas_op(op_rev["pnl"][en_estres])}).round(3)
OUT [07]Inside and outside the stress periods
                        fuera    estres
operaciones          9542.000  3091.000
win_rate                0.667     0.532
ganancia_media_pb     304.080   390.319
perdida_media_pb      548.349   780.430
payoff                  0.555     0.500
expectativa_pb         20.086  -157.369
profit_factor           1.110     0.569
win_rate_equilibrio     0.643     0.667

Outside the stress periods, the rule wins 66.7% with a payoff of 0.555 and earns 20.1 bp per trade: it is long for a few sessions in a market with positive drift. Inside them, 3,091 trades (one in four) win only 53.2% and lose 157 bp each: it buys dips in the middle of a crisis and hits the stop. The whole loss comes from there. It is the shape of the carry in the previous section: small, frequent gains and large losses clustered in crises. A high win rate is often exactly that: unknowingly selling crash insurance.

The header figure places the three strategies on the win rate-payoff plane. The solid curve is zero expectancy; above it, you make money. TSMOM (and the cloud of its 24 markets) sits at the bottom right: it is rarely right and wins a lot when it is. The carry, in the middle. The reversion, at the top left: it is right most often and is the only one below the curve.

Alpha and beta against an index

The second question is a different one: of what a portfolio earns, which part would anyone exposed to the market have earned. The regression of excess returns on those of the index separates it:

rt−rf,t=α+β (rm,t−rf,t)+εtr_t - r_{f,t} = \alpha + \beta\,(r_{m,t} - r_{f,t}) + \varepsilon_t

β\beta is the exposure to the index and α\alpha the average return it does not explain. To test them we need four portfolios on the universe of 120 fictitious stocks, with point-in-time data (lesson 1.4.4) and the delisting return of the bankruptcies:

  • Index: each month's index members, equally weighted. It is the benchmark.
  • Factors: the 20 stocks with the highest composite score of 12-1 momentum, value and low volatility, the three premia planted in the data. The effect exists.
  • ESG: the 20 with the highest ESG score, a pure placebo (an AR(1) independent of everything else). The truth is zero.
  • Quality (ROE) and size: the two controls with no premium. Quality has a trap documented in the data: ROE is E/P times P/B, it is highly correlated with value and in a univariate sort it inherits its premium with the opposite sign.
IN [08]python
from scipy import stats

ev = pd.read_csv("universo2_eventos.csv")
desl = ev.dropna(subset=["rentabilidad_deslistado"])
rd = pa.pct_change(fill_method=None)
for tk, r_desl in zip(desl["ticker"], desl["rentabilidad_deslistado"]):
    i = pa.index.get_loc(pa[tk].last_valid_index())
    if i + 1 < len(pa.index):          # delisting return
        rd.loc[pa.index[i + 1], tk] = r_desl
mes = rd.index.to_period("M")
rm = ((1 + rd.fillna(0)).groupby(mes).prod() - 1).where(
    rd.notna().groupby(mes).any())

def piv(col):     # characteristic at month-end (point-in-time)
    t = ch.pivot(index="fecha", columns="ticker", values=col)
    return t.set_axis(rm.index, axis=0).reindex(columns=rm.columns)

miembro = piv("miembro").fillna(False).astype(bool)
fin = pa.index.to_series().groupby(mes).max()
vol252 = rd.rolling(252, min_periods=126).std().loc[fin]
vol252.index = rm.index
mom = (1 + rm).rolling(11).apply(np.prod, raw=True).shift(1) - 1
value = np.log(piv("patrimonio_neto") / piv("capitalizacion"))

def z(s):           # rank-based normal score among members
    s = s.where(miembro)
    q = (s.rank(axis=1) - 0.5).div(s.notna().sum(axis=1), axis=0)
    return pd.DataFrame(stats.norm.ppf(q), s.index, s.columns)

def top(puntos, n=20):          # top 20, equally weighted
    rk = puntos.where(miembro).rank(axis=1, ascending=False)
    w = (rk <= n).astype(float)
    return w.div(w.sum(axis=1), axis=0)

compuesto = (z(mom).fillna(0) + z(value).fillna(0)
             - z(vol252).fillna(0))
pesos = {"indice": miembro.astype(float).div(miembro.sum(axis=1),
                                             axis=0),
         "factores": top(compuesto),
         "esg": top(piv("puntuacion_esg")),
         "calidad": top(piv("roe")),
         "tamano": top(-np.log(piv("capitalizacion")))}
rent = pd.DataFrame({k: (w.shift(1) * rm).sum(axis=1, min_count=1)
                     for k, w in pesos.items()}).loc["2007-01":]
tl = pd.read_csv("tipo_libre2.csv", parse_dates=["fecha"],
                 index_col="fecha")["tipo_3m"]
rf = (tl / 1200).groupby(tl.index.to_period("M")).mean()
exceso = rent.sub(rf.reindex(rent.index), axis=0)
pd.DataFrame({"media": 12 * exceso.mean(),
              "vol": np.sqrt(12) * exceso.std(),
              "sharpe": np.sqrt(12) * exceso.mean() / exceso.std()}
             ).round(3)
OUT [08]Factor portfolio, ESG placebo and equal-weighted index
          media    vol  sharpe
indice    0.068  0.220   0.308
factores  0.109  0.196   0.558
esg       0.092  0.218   0.421
calidad   0.023  0.223   0.105
tamano    0.072  0.241   0.299

The monthly returns rm come from the adjusted prices plus the delisting return in universo2_eventos.csv. universo2_caracteristicas.parquet also has one last row for each of the ten stocks that stop trading, with no price, with miembro = False and with that same return chained into rentabilidad_mes (−100% for bankruptcies). Here they change nothing: the signals and weights are formed only from members, and the delisting month is already in rm. Anyone who uses rentabilidad_mes instead of computing it has to keep those rows for performance and filter them out (by a non-empty cierre) for the signals: without them, a portfolio holding one of the bankruptcies would not see its −100%.

For the error of α\alpha we use Newey-West errors1Note 1 · Whitney K. Newey and Kenneth D. West, “A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix”, Econometrica 55(3), 703-708 (1987). In statsmodels, fit(cov_type="HAC", cov_kwds={"maxlags": 5}). (lesson 2.1.1), robust to heteroskedasticity and to autocorrelation of the residuals, with 5 lags (a common rule of thumb, 4 (T/100)2/94\,(T/100)^{2/9}, gives 4.8 with 228 months):

IN [09]python
import statsmodels.api as sm

def alfa_beta(y, x, lags=5):
    X = sm.add_constant(x.rename("beta"))
    ols = sm.OLS(y, X).fit()
    hac = sm.OLS(y, X).fit(cov_type="HAC", cov_kwds={"maxlags": lags})
    return {"beta": hac.params["beta"],
            "alfa_anual": 12 * hac.params["const"],
            "t_ols": ols.tvalues["const"],
            "t_hac": hac.tvalues["const"],
            "ee_alfa_anual": 12 * hac.bse["const"]}

pd.DataFrame({k: alfa_beta(exceso[k], exceso["indice"])
              for k in ["factores", "esg", "calidad", "tamano"]}
             ).T.round(3)
OUT [09]Alpha and beta with Newey-West errors
           beta  alfa_anual  t_ols  t_hac  ee_alfa_anual
factores  0.847       0.052  3.670  4.093          0.013
esg       0.960       0.027  2.152  2.006          0.013
calidad   0.986      -0.043 -3.529 -2.907          0.015
tamano    1.065      -0.000 -0.014 -0.013          0.014
Fig. — Annual alpha of four 20-stock portfolios against the equal-weighted index, 2007-2025, with Newey-West errors: the ESG placebo clears the t = 2 bar by chance
  • Factors: beta 0.85 and an alpha of 5.2% a year with t = 4.1. The method recovers an effect that exists.
  • ESG: alpha of 2.7% with t = 2.0. A placebo clears the classic bar of two standard errors. It is not a failure of the method: with a t = 2 threshold, roughly one in twenty tests with no effect passes by chance. That is why a t of 2 on the first characteristic you test is not enough, and on the twentieth it means nothing (lesson 2.1.8).
  • Quality: alpha of −4.3% with t = −2.9. “Significant” and false: there is no quality premium in the data; the high-ROE portfolio has a high P/B (low value) and pays the value premium with the opposite sign. Without controlling for value, the regression attributes to quality what belongs to value.
  • Size: alpha of 0.0% and t = 0.0. The control behaves as it should.

The Newey-West errors change the t-statistics little in these portfolios (from 3.67 to 4.09 in the factor portfolio): their monthly residuals have almost no autocorrelation. In portfolios with overlaps (12-month returns computed every month) or with illiquid assets, the difference is large.

ReturnBeta x marketAlphaResidualAppraisal ratioActive vs indexInformation ratio
Mermaid source
flowchart TB
  R[Return] --> M[Beta x market]
  R --> A[Alpha]
  R --> E[Residual]
  A --> AP[Appraisal ratio]
  E --> AP
  R --> ACT[Active vs index]
  ACT --> IR[Information ratio]

Information ratio and tracking error

A manager competing with an index is measured by their active return, the difference from the index. Its standard deviation is the tracking error and the ratio of the two, the information ratio:

IR=r−rm‾σ(r−rm) 12IR = \frac{\overline{r - r_m}}{\sigma(r - r_m)}\,\sqrt{12}

The IR does not adjust for beta: a portfolio with a beta of 0.85 loses against the index in up months because of its lower exposure, and that goes into its tracking error. The appraisal ratio divides the regression alpha by the standard deviation of the residual: it measures the quality of the part the market does not explain.

IN [10]python
def ratio_informacion(r, ref, n=12):
    activa = r - ref
    te = activa.std() * np.sqrt(n)
    return {"activa_anual": activa.mean() * n, "tracking_error": te,
            "ir": activa.mean() * n / te}

def appraisal(y, x, n=12):
    fit = sm.OLS(y, sm.add_constant(x)).fit()
    return n * fit.params.iloc[0] / (fit.resid.std() * np.sqrt(n))

pd.DataFrame({k: {**ratio_informacion(rent[k], rent["indice"]),
                  "appraisal": appraisal(exceso[k], exceso["indice"])}
              for k in ["factores", "esg"]}).T.round(3)
OUT [10]Information ratio and tracking error
          activa_anual  tracking_error     ir  appraisal
factores         0.042           0.070  0.595      0.847
esg              0.024           0.054  0.440      0.497

The factor portfolio beats the index by 4.2% a year with a tracking error of 7.0%: an IR of 0.60. Its appraisal ratio is higher, 0.85, because removing the beta of 0.85 takes the noise the market adds out of the denominator. The ESG placebo has an IR of 0.44: with 19 years, it is also compatible with zero. The rule for reading them is the same as for the Sharpe in lesson 2.6.1: an IR of 0.5 over 19 years has a standard error of about 0.23.

Turnover and capacity

The metrics above are gross of costs. To turn them into net figures you need to know how much is traded. The turnover of a rebalance is half the sum of the absolute changes in the weights (each buy has a sell on the other side) and the annual turnover, its sum over a year:

TO=12∑i∣wi,t−wi,t−1∣TO = \frac{1}{2}\sum_i \left| w_{i,t} - w_{i,t-1} \right|
IN [11]python
def turnover_anual(w, n=12):
    cambio = w.fillna(0).diff().abs().sum(axis=1) / 2
    return cambio.loc["2007-01":].mean() * n

pd.Series({k: turnover_anual(w) for k, w in pesos.items()}).round(2)
OUT [11]Turnover
indice      0.01
factores    2.28
esg         2.65
calidad     0.40
tamano      0.64
dtype: float64

The index barely turns over (1% a year: only the members change). The factor portfolio turns over 228%: each year it renews its holdings more than twice, because momentum and volatility change every month. The ESG placebo turns over even more (265%), because its score is almost noise. Quality (40%) and size (64%) are slow characteristics. The calculation uses the target weights at each month-end and ignores the drift of the weights between rebalances: it is a lower-bound approximation.

Capacity is the capital beyond which costs eat the edge (lesson 2.2.3). It depends on how much is traded, on the size of each order relative to the stock's volume and on how the cost grows with that size. The cell's model, deliberately simple, charges half the bid-ask spread plus a square-root impact for each unit traded:

ci=si2+σiΔwi⋅KADVic_i = \frac{s_i}{2} + \sigma_i \sqrt{\frac{\Delta w_i \cdot K}{ADV_i}}

with sis_i the bid-ask spread, σi\sigma_i the daily volatility, KK the capital under management and ADViADV_i the stock's average daily dollar volume (volumen_medio_21d times the close). The impact coefficient is 1, an assumption: lesson 2.4.7 explains how it is calibrated and why real impact depends on how the order is spread over time.

IN [12]python
adv = piv("volumen_medio_21d") * piv("cierre")    # USD per session
spr_m = piv("horquilla_pb") / 1e4
dw = pesos["factores"].fillna(0).diff().abs().loc["2007-01":]
adv, spr_m, sd = (t.reindex_like(dw) for t in (adv, spr_m, vol252))

def capacidad(capital):
    part = dw * capital / adv              # fraction of ADV
    coste = dw * (spr_m / 2 + sd * np.sqrt(part))
    operadas = part.where(dw > 0).stack().dropna()
    return {"coste_anual": 12 * coste.sum(axis=1).mean(),
            "participacion_p95": operadas.quantile(0.95)}

pd.DataFrame({f"{c:.0e}": capacidad(c)
              for c in [1e6, 1e7, 1e8, 3e8, 1e9, 3e9]}).T.round(4)
OUT [12]Capacity: annual cost by capital under management
       coste_anual  participacion_p95
1e+06       0.0030             0.0019
1e+07       0.0068             0.0190
1e+08       0.0191             0.1902
3e+08       0.0322             0.5707
1e+09       0.0577             1.9024
3e+09       0.0992             5.7072
Fig. — Capacity of the factor portfolio: annual alpha net of spread and impact by assets under management (square-root model assumed in the lesson)

With one million dollars, the cost is almost only the bid-ask spread: 0.30% a year against a gross alpha of 5.2%. With 100 million, 1.91%, and the 95th percentile of orders is already 19% of its stock's daily volume. With 1 billion, 5.8%, more than the whole alpha, and the 95th-percentile order is almost twice the daily volume: it could not be executed in one session. With this model, the portfolio loses half its alpha at around 190 million and all of it at around 808 million. A practical execution limit (95% of orders not exceeding 10% of ADV) puts it at about 53 million.

CapitalOrder per stockTurnoverADV participationImpactBid-ask spreadCostNet alpha
Mermaid source
flowchart TB
  K[Capital] --> O[Order per stock]
  TO[Turnover] --> O
  O --> PA[ADV participation]
  PA --> IM[Impact]
  S[Bid-ask spread] --> CO[Cost]
  IM --> CO
  CO --> AN[Net alpha]

Common mistakes when reading these metrics

  • Judging by the win rate. It only says something together with the payoff: always compare it with 1/(1+payoff)1/(1 + \text{payoff}).
  • Mixing units. A profit factor of days, one of months and one of trades are three different figures.
  • Forgetting open trades and delistings. Trades still open at the end of the sample and those in stocks that stop trading change the expectancy.
  • Calling a beta alpha. A portfolio with a beta of 1.2 beats the index in a bull market without any alpha. Regress on the right benchmark with robust errors.
  • Believing the first t of 2. The ESG placebo gets one. Count how many characteristics you have tested.
  • Ignoring turnover. A gross alpha of 5% with 228% turnover is a different thing with 500 million than with one.

Exercises

  1. Group the TSMOM trades by sector (the sector column of futuros2_especificaciones.csv) and compute the win rate, payoff and expectancy of each one. Is any sector below break-even?
Solution
IN [13]python
esp = pd.read_csv("futuros2_especificaciones.csv",
                  index_col="mercado")["sector"]
por_sector = op_tsmom.groupby(op_tsmom["mercado"].map(esp))["pnl"]
pd.DataFrame({s: metricas_op(g) for s, g in por_sector}).T[
    ["operaciones", "win_rate", "payoff", "expectativa_pb",
     "win_rate_equilibrio"]].round(3)
OUT [13]TSMOM by sector
                operaciones  win_rate  payoff  expectativa_pb  win_rate_equilibrio
agricolas             349.0     0.344   5.180          22.025                0.162
bonos                 488.0     0.357   4.677          19.575                0.176
divisas               460.0     0.328   2.312           1.537                0.302
energia               421.0     0.280   2.532          -0.183                0.283
metales               414.0     0.343   2.635           5.262                0.275
renta_variable        444.0     0.324   3.635          13.350                0.216

Five of the six sectors are above their break-even win rate, with similar profiles: they win between 28.0% and 35.7%, and what changes is the payoff. Agriculturals have the highest payoff (5.18) and the highest expectancy (22.0 bp); bonds, the highest win rate (35.7%) and a payoff of 4.68. Energy sits just below break-even (28.0% against 28.3%, expectancy of −0.18 bp) and currencies, just above (1.5 bp). However, all 24 markets have the same planted momentum term (0.15 per unit of volatility, data README): the differences between sectors are mostly noise. With 350 to 490 trades per sector they cannot be told apart; deciding which sectors to trade would require their standard error, not just the figure, and here dropping energy would mean dropping an effect that exists.

  1. Repeat the reversion with a symmetric target and stop (2σ and 2σ). What happens to the win rate, the payoff and the expectancy?
Solution
IN [14]python
op_sim = reversion(objetivo=2.0, stop=2.0)
pd.Series(metricas_op(op_sim["pnl"])).round(3)
OUT [14]Symmetric reversion
operaciones            12522.000
win_rate                   0.495
ganancia_media_pb        459.519
perdida_media_pb         484.926
payoff                     0.948
expectativa_pb           -17.832
profit_factor              0.927
win_rate_equilibrio        0.513
dtype: float64

With the target and stop at the same distance, the win rate drops to 49.5% and the payoff rises to 0.95: the rule no longer “wins a lot”. The expectancy stays negative (−17.8 bp per trade, break-even at 51.3%), for the same reason as before: buying in the middle of a crisis. Moving the thresholds changes the shape of the results, not their mean; only a real edge (or avoiding the stress periods, if they could be known in advance) changes the expectancy.

  1. Estimate the beta of TSMOM against the IDA index future with monthly returns and Newey-West errors. What role would TSMOM play in a portfolio that already holds the index?
Solution
IN [15]python
tsmom_d = (peso * r_fut).sum(axis=1)
m = pd.DataFrame({"tsmom": tsmom_d, "ida": r_fut["IDA"]})
m = (1 + m).resample("ME").prod() - 1
ab = alfa_beta(m["tsmom"], m["ida"])
print(f"beta {ab['beta']:.3f} · alpha {ab['alfa_anual']:.2%} "
      f"(HAC t {ab['t_hac']:.2f})")
print(f"monthly correlation: {m.corr().iloc[0, 1]:.3f}")
OUT [15]Beta of TSMOM against the index
beta 0.089 · alpha 11.38% (HAC t 4.36)
monthly correlation: 0.162

The beta of TSMOM against the index is 0.09 and the monthly correlation, 0.16: it hardly depends on the market. Its alpha against the index (11.38% a year, t of 4.36) is practically all of its return. In a portfolio with the index, TSMOM would act as a diversifier: it adds return with little correlation, and in this data it makes money in the long crises (2008, +26.9%; 2011, +8.1%; 2022, +8.4%), but not in 2020 (−1.7%), a two-month fall too fast for a 12-month signal. It is a result on synthetic data with the effect planted; Moskowitz, Ooi and Pedersen (2012) study the real relationship of time series momentum with bull and bear markets.

Footnotes

  1. Whitney K. Newey and Kenneth D. West, “A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix”, Econometrica 55(3), 703-708 (1987). In statsmodels, fit(cov_type="HAC", cov_kwds={"maxlags": 5}). ↩

Each answer is folded under its question

Self-assessment

  1. A strategy wins 63% of its trades with a payoff of 0.52. Does it make money?

    • ANo: with a payoff of 0.518 the break-even win rate is 1 / (1 + 0.518) = 65.9%, and 63.4% falls short of it
    • BYes: it is right almost two times out of three
    • CIt depends only on the number of trades
    • DYes, if the profit factor is greater than the win rate
    Show answer

    Answer A. No: with a payoff of 0.518 the break-even win rate is 1 / (1 + 0.518) = 65.9%, and 63.4% falls short of it

    It is the lesson's reversion rule, applied to stocks with no reversion effect: target at +1σ and stop at −3σ. It wins often because the target is close, and it loses when it hits the stop, which is three times farther away. Its expectancy is −23 bp per trade before costs, and the whole loss comes from dips bought in the middle of a crisis.

  2. In TSMOM, the best 10% of trades contributes 212% of the total profit. What does that mean?

    • AThat there is an error: a percentage cannot exceed 100
    • BThat the strategy has negative skewness
    • CThat the large trades should be removed
    • DThat the remaining 90% loses as a whole (−112% of the total): the strategy lives on a few long trends and cannot skip any of them
    Show answer

    Answer D. That the remaining 90% loses as a whole (−112% of the total): the strategy lives on a few long trends and cannot skip any of them

    It is the typical shape of a trend follower: many small losses and a few large gains. Any filter that removes some of the large ones (for example, a profit target) destroys the strategy, and a period without trends produces long losing streaks.

  3. The portfolio of the 20 stocks with the highest ESG score (a placebo with no premium in the data) has an alpha of 2.7% a year with t = 2.0. What do you conclude?

    • AThat ESG has a 2.7% premium
    • BThat the Newey-West errors are miscalculated
    • CThat it is a false positive: with a t = 2 threshold, roughly one in twenty portfolios with no effect passes by chance, and here we know the truth is zero
    • DThat the index is not a good benchmark
    Show answer

    Answer C. That it is a false positive: with a t = 2 threshold, roughly one in twenty portfolios with no effect passes by chance, and here we know the truth is zero

    Synthetic data lets us know the answer: the ESG score is an AR(1) independent of everything else. A t of 2 in a single test is already fragile; if several characteristics have been tried, you have to correct for multiple testing (lesson 2.1.8).

  4. What is the difference between the information ratio and the appraisal ratio?

    • ANone
    • BThe IR divides the active return (portfolio minus index) by its volatility, without adjusting for beta; the appraisal ratio divides the regression alpha by the volatility of the residual, after removing the part explained by the market
    • CThe IR is only valid for index funds
    • DThe appraisal ratio uses log returns
    Show answer

    Answer B. The IR divides the active return (portfolio minus index) by its volatility, without adjusting for beta; the appraisal ratio divides the regression alpha by the volatility of the residual, after removing the part explained by the market

    A portfolio with a beta of 0.85 has an active return contaminated by the market: in bull years it loses against the index because of its lower beta. In the factor portfolio, IR 0.60 and appraisal 0.85.

  5. The factor portfolio turns over 228% a year. With USD 1 million the estimated annual cost is 0.30%; with USD 1 billion, 5.8%. Why does it grow so much?

    • ABecause the bid-ask spread costs the same at any capital, but impact grows with the participation in each stock's volume: with more capital, each order is a larger fraction of the ADV and moves the price more
    • BBecause commissions are proportional
    • CBecause more capital means trading more often
    • DBecause the risk-free rate goes up
    Show answer

    Answer A. Because the bid-ask spread costs the same at any capital, but impact grows with the participation in each stock's volume: with more capital, each order is a larger fraction of the ADV and moves the price more

    The bid-ask spread is a fixed cost per unit traded; impact, with the square-root model, grows with the square root of participation. With USD 1 billion, the 95th percentile of orders is almost twice its stock's average daily volume: impossible to execute in one day.

Key points

  • Expectancy = p·W − (1 − p)·L. The break-even win rate is 1 / (1 + payoff): with a payoff of 3, winning 25% is enough; with 0.5, you need 67%.
  • The win rate alone is misleading: TSMOM wins 33% and makes money; a reversion with no edge wins 63% and loses 23 bp per trade, all of it in crises: a high win rate often hides crash risk.
  • Shape matters: in TSMOM the best 10% of trades contributes 212% of the profit; in carry, the worst 10% of months takes 160%.
  • Alpha and beta are estimated with robust errors (Newey-West). In the data, the planted factor portfolio gives t = 4.1; the ESG placebo, t = 2.0 by chance; quality, −2.9 through confounding with value.
  • Information ratio = active return / tracking error; appraisal ratio = alpha / residual volatility. Report both, and the benchmark.
  • Capacity comes from turnover, the bid-ask spread and participation in volume. The factor portfolio loses half its alpha at around USD 190 million and all of it at around USD 800 million, under the lesson's assumptions.

Resources

Data