--- title: "3.1.4 Diversified futures systems (Carver)" description: "Carver's approach on 23 futures: continuous forecasts, scaled to 10 and capped at 20, combined with FDM, IDM, whole contracts, buffering and costs." url: "https://pythonparatrading.com/en/python-trading-course/level-3/diversified-futures-systems-carver/" lang: "en" image: "https://pythonparatrading.com/og/en/python-trading-course/level-3/diversified-futures-systems-carver.png" --- # 3.1.4 · Diversified futures systems: Carver's approach > Robert Carver proposes a futures system that does not buy or sell on signals but on forecasts: continuous numbers, scaled so that their average absolute value is 10, combined across rules and turned into contracts with the account's risk target. This lesson builds it in full on the 23 synthetic futures of the level, with trend and carry, and measures each piece: the scalars, the diversification multipliers, buffering, costs and what combining adds. - Course: [Level 3: Derivatives: futures, spreads, forex and options](https://pythonparatrading.com/en/python-trading-course/level-3/) - Module: 3.1 · Futures in depth - Duration: 75 min - Prerequisites: [3.1.3 · Roll management, margins, SPAN and implied leverage](https://pythonparatrading.com/en/python-trading-course/level-3/futures-roll-margins-span-leverage/), [2.2.2 · Universe, signal, filter, entry, exit, sizing and risk management](https://pythonparatrading.com/en/python-trading-course/level-2/universe-signal-entry-exit-position-sizing/), [2.3.1 · Trend following and momentum](https://pythonparatrading.com/en/python-trading-course/level-2/trend-following-and-momentum/), [2.3.4 · Carry in currencies, futures and bonds; seasonality and calendar effects](https://pythonparatrading.com/en/python-trading-course/level-2/carry-seasonality-calendar-effects/) ## By the end of this lesson - Calculate an EWMAC forecast and a carry forecast the way Carver does: in risk units, scaled so that their average absolute value is 10 and capped at ±20. - Estimate the forecast scalars with data from several markets at once and compare them with the reference values in pysystemtrade. - Combine forecasts with weights and a forecast diversification multiplier (FDM), and allocate risk across markets with an IDM. - Go from the forecast to the number of contracts with the risk target, the notional and the currency of each market, and apply a no-trade buffer. - Measure the system net of costs against each rule alone and against the known answer of the data, without confusing what is being validated (the code) with what is not (the strategy in real markets). ## Forecasts, not signals The systems of lessons 2.2.2 and 2.3.1 decided “long, short or flat” and sized by volatility. Robert Carver, in *Systematic Trading* (2015) and *Advanced Futures Trading Strategies* (2023), separates those two things differently: each rule produces a **forecast**, a continuous number that says how much is expected and with how much conviction, and a single formula turns any forecast into contracts. The pieces: ```mermaid flowchart TB P[Adjusted price] --> R[Raw rule] V[Volatility] --> R R --> E[× scalar] E --> T[Cap ±20] T --> C[Weights × FDM] C --> N[Contracts] N --> B[Buffer and order] ``` The conventions are those of his open-source library, pysystemtrade (GPL v3 license): the average absolute forecast is 10 (`average_absolute_forecast: 10.0` in its default configuration), it is capped at ±20 (`forecast_cap: 20.0`), and the diversification multipliers are capped at 2.5 (`dm_max: 2.5`). With a forecast of +10 the system has the “average” position in a market; with +20, twice that; with −5, half of it short. The goal of this lesson is to build the whole system and measure what each piece adds, not to reproduce pysystemtrade line by line: where the lesson departs from its configuration, it says so. The starting data: Carver calculates trend rules on the price adjusted by **difference** (the “Panama” method of lesson 1.4.6): the daily change in points of the contract held, accumulated. It preserves price differences, which is what a contract gains or loses, and volatility is measured in the same units: ```python import numpy as np import pandas as pd curvas = pd.read_parquet("futuros3_curvas.parquet") ctr = pd.read_csv("futuros3_contratos.csv", parse_dates=["vencimiento", "fecha_roll"]) esp = pd.read_csv("futuros3_especificaciones.csv", index_col="mercado") fx = pd.read_parquet("fx3_diario.parquet") usd_qev = fx[fx["divisa"] == "QEV"].set_index("fecha")[ "usd_por_unidad"] MERCADOS = list(esp.index[esp["mercado_base"].isna()]) def serie(mercado): """Contract held (roll on fecha_roll), its price, the daily change in points of the same contract and the annual carry.""" c = curvas[curvas["mercado"] == mercado] P = c.pivot(index="fecha", columns="contrato", values="liquidacion") cm = ctr[ctr["mercado"] == mercado].reset_index(drop=True) pos = np.searchsorted(cm["fecha_roll"].to_numpy(), P.index.to_numpy(), side="right") tiene = cm["contrato"].to_numpy()[pos] sig = cm["contrato"].to_numpy()[np.minimum(pos + 1, len(cm) - 1)] i, j = P.columns.get_indexer(tiene), P.columns.get_indexer(sig) f, Pv = np.arange(len(P)), P.to_numpy() ayer = np.r_[i[0], i[:-1]] dif = np.r_[np.nan, Pv[f[1:], ayer[1:]] - Pv[f[1:] - 1, ayer[1:]]] v = cm.set_index("contrato")["vencimiento"] dt = (v.reindex(sig).to_numpy() - v.reindex(tiene).to_numpy()).astype("timedelta64[D]") F2 = np.where(j >= 0, Pv[f, np.maximum(j, 0)], np.nan) carry = np.log(Pv[f, i] / F2) / (dt.astype(float) / 365) return pd.DataFrame({"contrato": tiene, "precio": Pv[f, i], "dif": dif, "carry": carry}, index=P.index) S = {m: serie(m) for m in MERCADOS} fechas = S["RVA"].index tabla = lambda col: pd.DataFrame( {m: S[m][col] for m in MERCADOS}).reindex(fechas) precio, dif, contrato = tabla("precio"), tabla("dif"), tabla("contrato") carry = tabla("carry") ajustado = dif.fillna(0).cumsum() # difference adjustment sigma_p = dif.ewm(span=35, min_periods=20).std() # points per day print(precio.shape, fechas[0].date(), fechas[-1].date()) pd.DataFrame({"precio": precio.iloc[-1], "sigma_dia_pts": sigma_p.iloc[-1], "sigma_anual_pct": 100 * np.sqrt(252) * sigma_p.iloc[-1] / precio.iloc[-1]}).loc[["RVA", "TPC", "PEC", "GRM"] ].round(3) ``` ```output (4024, 23) 2010-01-04 2025-12-31 precio sigma_dia_pts sigma_anual_pct RVA 6747.000 131.746 30.998 TPC 112.547 0.460 6.481 PEC 60.900 1.544 40.252 GRM 340.250 4.196 19.579 ``` 23 markets and 4,024 sessions. Volatility in points is a 35-session exponential average of the daily changes, the same window that pysystemtrade's EWMAC rule uses by default (`vol_days=35`); Carver also uses variants that blend recent volatility with long-term volatility, which are not applied here. On the last day, `RVA` moves 131.7 points a day and the 10-year note, 0.46: three orders of magnitude that have to be removed before comparing rules across markets. ## EWMAC: the rule and its scaling Carver's trend rule is the difference between two exponential averages of the adjusted price, fast and slow (with the slow one four times longer), divided by the daily volatility of the price in points: $$ \text{EWMAC}(r, l)_t = \frac{\mathrm{EMA}_r(P)_t - \mathrm{EMA}_l(P)_t}{\sigma_{P,t}} $$ It is what `ewmac.py` does in pysystemtrade: `raw_ewmac = fast_ewma - slow_ewma` and `raw_ewmac / vol.ffill()`. The result is in daily standard deviations and has no natural scale, so it is multiplied by a **scalar** that brings its average absolute value to 10. The scalar is estimated by pooling all the markets, so that it does not depend on one, and with an initial window that is not used afterwards to judge the rule: ```python def ewmac(rapida, lenta): m_r = ajustado.ewm(span=rapida, min_periods=rapida).mean() m_l = ajustado.ewm(span=lenta, min_periods=lenta).mean() return (m_r - m_l) / sigma_p ESTIMA = slice("2011", "2013") # window for the scalars brutos = {f"ewmac{a}_{4 * a}": ewmac(a, 4 * a) for a in (8, 16, 32, 64)} escalar = {} for k, b in brutos.items(): media_abs = b.loc[ESTIMA].abs().stack().mean() resto = b.loc["2014":].abs().stack().mean() escalar[k] = 10 / media_abs print(f"{k:11s} scalar {10 / media_abs:5.2f} " f"(with 2014-2025: {10 / resto:5.2f})") ``` ```output ewmac8_32 scalar 5.65 (with 2014-2025: 5.94) ewmac16_64 scalar 3.74 (with 2014-2025: 4.01) ewmac32_128 scalar 2.45 (with 2014-2025: 2.65) ewmac64_256 scalar 1.76 (with 2014-2025: 1.76) ``` The scalars estimated with 2011-2013 are 5.65, 3.74, 2.45 and 1.76 for the four speeds; with 2014-2025, 5.94, 4.01, 2.65 and 1.76. The chapter 15 example configuration of pysystemtrade uses 5.3, 3.75, 2.65 and 1.87: almost the same, on completely different data. It is not a coincidence. With the rule in risk units, its scale depends on the windows of the averages and on how volatility is measured, and little on the market. That is why a scalar estimated with many markets works for all of them, and why it is reasonable to fix it once and not re-estimate it every day. What the scalar does not say is whether the rule makes money. ```python def escala(bruto, k, tope=20): return (bruto * k).clip(-tope, tope) prev = {"ewmac16_64": escala(brutos["ewmac16_64"], escalar["ewmac16_64"]), "ewmac64_256": escala(brutos["ewmac64_256"], escalar["ewmac64_256"])} for k, p in prev.items(): x = p.loc["2011":].stack() print(k, f"mean |f| {x.abs().mean():.1f},", f"at the cap {100 * (x.abs() == 20).mean():.1f} %,", f"std. dev. {x.std():.1f}") ``` ```output ewmac16_64 mean |f| 8.9, at the cap 9.3 %, std. dev. 10.8 ewmac64_256 mean |f| 9.1, at the cap 11.3 %, std. dev. 11.0 ``` Over the whole sample, the average absolute value ends up at 8.9 and 9.1, a little below 10 because the cap clips the extremes: 9.3% and 11.3% of market-days are at ±20. The cap exists for two reasons: a huge forecast is usually a data error or a market in a move that has already happened, and it limits how much a single market can weigh in the portfolio. ## Carry as a forecast The carry of lesson 3.1.2 (the annualized slope between the contract held and the next one) is turned into a forecast the same way: in risk units, divided by the annual volatility of the return. pysystemtrade calculates raw carry as “the annualized roll divided by the annualized volatility” (`rawdata.py`) and smooths it with a 90-day exponential average (`carry.py`); here it is smoothed with 30 sessions, a choice of this lesson: ```python vol_anual = (dif / precio.shift(1)).ewm(span=35, min_periods=20 ).std() * np.sqrt(252) bruto_carry = carry.ewm(span=30, min_periods=10).mean() / vol_anual escalar["carry"] = 10 / bruto_carry.loc[ESTIMA].abs().stack().mean() prev["carry"] = escala(bruto_carry, escalar["carry"]) print(f"carry scalar: {escalar['carry']:.1f}") ultimo = prev["carry"].loc["2025-11-26"] ultimo.sort_values().round(1).iloc[[0, 1, 2, -3, -2, -1]] ``` ```output carry scalar: 16.7 GRS -20.0 GRH -20.0 GRM -20.0 GNA 15.6 PEC 19.5 PEH 20.0 Name: 2025-11-26 00:00:00, dtype: float64 ``` The carry scalar comes out at 16.7, different from the 30 in the pysystemtrade configuration, because the data and the way of calculating carry are not the same (here, in logs and with different smoothing): the scalar is estimated for each definition. On November 26, 2025, the soybean and corn agriculturals are at −20 (steep contango) and energy close to +20 (backwardation in crude oil and heating oil, lesson 3.1.2). A detail that the code does not handle and a real system does: in markets with a seasonal shape in the curve (gas, electricity), the carry between consecutive months jumps with the calendar, and that part is not carry (lesson 3.1.2). ## Combining: weights and the diversification multiplier The combined forecast is a weighted average of the forecasts of each rule. Since the rules are not perfectly correlated, the average moves less than each one and its average absolute value ends up below 10. The **forecast diversification multiplier** (FDM) brings it back to its scale: $$ \text{FDM} = \frac{1}{\sqrt{w'\rho\,w}} $$ with $w$ the weights and $\rho$ the correlation matrix of the forecasts. The lesson's weights are 0.25 for each EWMAC and 0.50 for carry, similar to those of the pysystemtrade chapter 15 example (0.21, 0.08 and 0.21 for three EWMACs and 0.50 for carry): ```python apiladas = pd.DataFrame({k: p.loc[ESTIMA].stack() for k, p in prev.items()}).dropna() rho = apiladas.corr() w = pd.Series({"ewmac16_64": 0.25, "ewmac64_256": 0.25, "carry": 0.50}) fdm = min(2.5, 1 / np.sqrt(w @ rho @ w)) print(rho.round(2)) print(f"FDM = {fdm:.3f}") combinada = sum(prev[k] * w[k] for k in w.index) * fdm combinada = combinada.clip(-20, 20) x = combinada.loc["2011":].stack() print(f"combined: mean |f| {x.abs().mean():.1f}, " f"at the cap {100 * (x.abs() == 20).mean():.1f} %") ``` ```output ewmac16_64 ewmac64_256 carry ewmac16_64 1.00 0.61 0.34 ewmac64_256 0.61 1.00 0.36 carry 0.34 0.36 1.00 FDM = 1.263 combined: mean |f| 7.9, at the cap 7.5 % ``` The two EWMACs have a correlation of 0.61 (they are the same idea at two speeds) and carry, 0.34 and 0.36 with them. The FDM is 1.263. The combined forecast has an average absolute value of 7.9, below 10: the FDM was estimated with 2011-2013 and the cap clips 7.5% of the days. A system that wanted exactly its risk target would re-estimate the FDM with an expanding window; the consequence, here, is that the system will have somewhat less volatility than requested. *Fig. — Synthetic crude, 2019-2023: difference-adjusted price and the system's combined forecast, capped at ±20.* Two panels sharing the time axis, 2019 to 2023. Top, the difference-adjusted price of synthetic crude: around 120 in 2019, falling to about 35 in 2020, rising to near 100 in mid-2022 and dropping to almost 0 at the end of 2023. Bottom, the combined forecast (highlighted) between the caps of −20 and +20: it moves between −15 and +10 in 2019, spends almost all of 2020 at −20, rises to +10 in 2021-2022 without reaching the cap and returns to −20 in 2023. The figure shows the combined forecast for crude oil from 2019 to 2023 below its adjusted price. It spends almost all of 2020 at the −20 cap (downtrend and contango at the same time), rises with the 2021-2022 recovery without going above +11 (the trend is positive but carry, in the data, is still negative for a good part of that time) and goes back to −20 in 2023. The difference-adjusted price reaches values close to zero at the end of 2023: with this adjustment, old levels are not real prices, and that is why the rules use price differences and not percentages on the adjusted price. ## From forecast to number of contracts The position in each market comes from a single formula. With capital $K$, an annual risk target $\tau$, the weight of the market in the portfolio $w_i$ and the annual money risk of one contract (lesson 3.1.1): $$ N_i = \frac{f_i}{10} \cdot \text{IDM} \cdot w_i \cdot \frac{K \, \tau}{\sigma_{P,i}\sqrt{252} \cdot M_i \cdot X_i} $$ where $M_i$ is the multiplier and $X_i$ the exchange rate to USD. The **instrument diversification multiplier** (IDM) does with markets what the FDM does with rules: since markets do not move together, a portfolio with $\tau / n$ in each one has less risk than $\tau$, and the IDM compensates for it. ```python normal = (dif / sigma_p.shift(1)).loc[ESTIMA] rho_i = normal.corr() w_i = pd.Series(1 / len(MERCADOS), index=MERCADOS) idm_bruto = 1 / np.sqrt(w_i @ rho_i @ w_i) idm = min(2.5, idm_bruto) print(f"IDM uncapped {idm_bruto:.2f}, capped {idm:.2f}") ``` ```output IDM uncapped 2.81, capped 2.50 ``` With 23 markets, equal weights and the correlations of the data (high within each sector and almost zero across sectors, lesson 3.1.1), the uncapped IDM would be 2.81; with the 2.5 cap, it stays at 2.5. The cap is a precaution: the IDM is estimated with calm-period correlations, and in a crisis correlations rise and so does the real risk of the portfolio. The pysystemtrade example configuration uses 1.89 with its universe. Equal weights per market are a simplification: Carver allocates by sector and then within each one, so that five correlated indices do not weigh five times as much as one currency. ```python CAPITAL, TAU = 5_000_000, 0.20 mult = esp.loc[MERCADOS, "multiplicador"] cambio = pd.DataFrame( {m: (usd_qev.reindex(fechas).ffill() if esp.loc[m, "divisa"] == "QEV" else pd.Series(1.0, index=fechas)) for m in MERCADOS}) riesgo_contrato = sigma_p * np.sqrt(252) * mult * cambio # USD/year media = idm * w_i * CAPITAL * TAU / riesgo_contrato optimo = combinada / 10 * media d = "2025-11-26" pd.DataFrame({"prevision": combinada.loc[d], "riesgo_contrato": riesgo_contrato.loc[d], "posicion_media": media.loc[d], "optimo": optimo.loc[d]}).loc[ ["RVA", "TPA", "PEC", "GNA", "MTO", "GRM"]].round(2) ``` ```output prevision riesgo_contrato posicion_media optimo RVA 8.44 83822.01 1.30 1.09 TPA 0.89 4835.93 22.48 1.99 PEC 20.00 20918.48 5.20 10.39 GNA 14.79 22301.88 4.87 7.21 MTO 7.27 26192.67 4.15 3.02 GRM -17.47 2768.80 39.26 -68.58 ``` With 5 million USD and a 20% risk target, the average position (that of a +10 forecast) is 1.30 contracts in `RVA`, 22.48 in the 2-year note and 39.26 in corn. On November 26, 2025, the crude oil forecast is at the cap (+20) and the system would want 10.39 contracts, twice its average position; corn's is −17.47 and it would want 68.58 contracts short. `RVA`, at +8.44, would want 1.09: with a large index contract, rounding matters. ## Buffering and costs An optimal position of 1.09 contracts one day and 0.96 the next does not justify a trade. Carver avoids those trades with a **buffer**: a band is calculated around the optimum (in pysystemtrade, `buffer_size: 0.10`, 10% of the average position) and you only trade if the current position falls outside it, and only up to the edge of the band (`buffer_trade_to_edge: True`): ```mermaid flowchart LR O[Optimum ± buffer] --> Q{Position inside?} Q -- yes --> X[Do not trade] Q -- no --> Y[Trade to the edge] ``` The daily loop uses the position decided with the previous session's data, executed at the close of the session (lesson 2.2.2), in whole contracts. Each change pays half a tick and the commission per contract, and each roll, two sides on the whole position: ```python def contratos(optimo, media, buffer=0.10): """Whole position: trade only if the current one falls outside optimum ± buffer·average position, and only up to the edge.""" P = optimo.shift(1).to_numpy() # decided with data from t − 1 B = buffer * media.shift(1).to_numpy() N, actual = np.zeros_like(P), np.zeros(P.shape[1]) for t in range(len(P)): bajo, alto = np.round(P[t] - B[t]), np.round(P[t] + B[t]) nuevo = np.clip(actual, bajo, alto) actual = np.where(np.isnan(P[t]), 0.0, nuevo) N[t] = actual return pd.DataFrame(N, index=optimo.index, columns=optimo.columns) coste_lado = (0.5 * esp.loc[MERCADOS, "valor_tick"] + esp.loc[MERCADOS, "comision"]) * cambio # USD roll = contrato.ne(contrato.shift(1)) def backtest(N): pyg = (N.shift(1) * dif * mult * cambio).sum(axis=1) c = (N.diff().abs() * coste_lado + 2 * N.shift(1).abs() * roll * coste_lado).sum(axis=1) return (pyg / CAPITAL).loc["2011":], (c / CAPITAL).loc["2011":] def resumen(r): return {"sharpe": r.mean() / r.std() * np.sqrt(252), "vol_pct": 100 * r.std() * np.sqrt(252), "rent_pct": 100 * r.mean() * 252} filas = {} for nombre, b in [("no buffer", 0.0), ("buffer 10%", 0.10)]: N = contratos(optimo, media, b) bruto, coste = backtest(N) cambios = (N.diff().abs() > 0).loc["2011":].sum().sum() / 15 filas[nombre] = {**resumen(bruto - coste), "coste_pct": 100 * coste.sum() / 15, "operaciones_ano": cambios} pd.DataFrame(filas).T.round(2) ``` ```output sharpe vol_pct rent_pct coste_pct operaciones_ano no buffer 0.92 16.53 15.19 0.91 2134.67 buffer 10% 0.86 16.34 14.11 0.69 1132.07 ``` The buffer almost halves the trades (from 2,135 to 1,132 position changes a year across the 23 markets) and costs from 0.91% to 0.69% a year. But the net Sharpe falls from 0.92 to 0.86: what is saved in costs is less than what is lost by being farther from the optimal position. It is not a general rule. With 5 million USD, costs of half a tick and a volatility of 16%, costs are small (0.7-0.9 points a year against 14-15 of return); with more expensive markets, less capital or a faster system, the balance changes, and the first exercise checks it. The 0.06 difference between the two rows, moreover, is small relative to the standard error of a 15-year Sharpe (around 0.25). ## Each rule, the system and the known answer ```python def sistema(prevision): N = contratos(prevision / 10 * media, media) bruto, coste = backtest(N) return bruto - coste netos = {k: sistema(p) for k, p in prev.items()} netos["trend (two EWMACs)"] = sistema( (0.5 * prev["ewmac16_64"] + 0.5 * prev["ewmac64_256"]).clip( -20, 20) * min(2.5, 1 / np.sqrt( np.array([0.5, 0.5]) @ rho.iloc[:2, :2].to_numpy() @ np.array([0.5, 0.5])))) netos["combined"] = sistema(combinada) pd.DataFrame({k: resumen(r) for k, r in netos.items()}).T.round(2) ``` ```output sharpe vol_pct rent_pct ewmac16_64 0.99 19.75 19.47 ewmac64_256 0.60 20.55 12.28 carry 0.53 14.69 7.82 trend (two EWMACs) 0.86 20.14 17.42 combined 0.86 16.34 14.11 ``` *Fig. — Net Sharpe of each rule and of the combination in the lesson's system, with its realised volatility (23 synthetic futures, 2011-2025).* Horizontal bars of net Sharpe: EWMAC 16/64, 0.99 with 19.8% volatility; EWMAC 64/256, 0.60 and 20.6%; carry, 0.53 and 14.7%; trend with the two EWMACs, 0.86 and 20.1%; combined (highlighted), 0.86 and 16.3%; buy and hold with the same sizing, 0.61. Net of costs, from 2011 to 2025: the fast EWMAC alone gives a Sharpe of 0.99, the slow one 0.60, carry 0.53, the two EWMACs together 0.86 and the combination of all three, 0.86 with a volatility of 16.3% (below the requested 20%, because of the combined forecast's average absolute value of 7.9). The header figure plots the curves: 1 USD in 2011 ends at 13.8 with the fast EWMAC, 4.6 with the slow one, 2.7 with carry and 6.8 with the combination. That a single rule came out better than the combination is not an argument for keeping it. With 15 years, the standard error of a Sharpe is about 0.25, and picking the best of several correlated rules after the fact is the multiple testing bias (lesson 2.1.8). The combination has less volatility and does not depend on a single rule continuing to work. ```python neto = netos["combined"] tramos = {f"{a}-{b}": float(resumen(neto.loc[str(a):str(b)])["sharpe"]) for a, b in [(2011, 2015), (2016, 2020), (2021, 2025)]} capital = (1 + neto).cumprod() caida = (capital / capital.cummax() - 1).min() print({k: round(v, 2) for k, v in tramos.items()}) print(f"maximum drawdown {100 * caida:.1f} %") comprar = sistema(pd.DataFrame(10.0, index=fechas, columns=MERCADOS)) print(f"buy and hold: Sharpe " f"{resumen(comprar)['sharpe']:.2f}, correlation with the " f"system {neto.corr(comprar):.2f}") ``` ```output {'2011-2015': 0.96, '2016-2020': 0.61, '2021-2025': 1.02} maximum drawdown -30.1 % buy and hold: Sharpe 0.61, correlation with the system 0.12 ``` By five-year period, the Sharpe of the combination is 0.96, 0.61 and 1.02; the maximum drawdown, 30.1%. Buy and hold with the same sizing (a fixed forecast of +10 in every market) gives 0.61, with a correlation of 0.12 with the system. The data README measures simpler portfolios on the same files (weights of 10% over the 63-session volatility, no costs, simple returns) and gives 0.99 for the EWMAC 16/64 + 64/256 trend and 0.53 for buy and hold, close to this lesson's 0.61. That measure hides a trap worth seeing once: with the same weights, just changing the type of return being averaged is enough to sink the buy-and-hold Sharpe: ```python log_r = np.log(precio / (precio - dif)) # ln(P_t / P_t−1) vol63 = log_r.rolling(63, min_periods=40).std() * np.sqrt(252) pesos = (0.10 / vol63).shift(1) for nombre, r in [("logarithmic", log_r), ("arithmetic", np.exp(log_r) - 1)]: cartera = (pesos * r).mean(axis=1).iloc[260:] print(f"{nombre:12s} Sharpe " f"{cartera.mean() / cartera.std() * np.sqrt(252):.2f}") ``` ```output logarithmic Sharpe 0.23 arithmetic Sharpe 0.54 ``` With the same weights, averaging log returns gives a Sharpe of 0.23 and averaging arithmetic returns, 0.54. The return of a portfolio is the weighted average of the **arithmetic** returns; the log return of each market is lower than the arithmetic one by half the variance ($\sigma^2/2$) on average, about 6 to 9 points a year in a commodity with 35-42% volatility, and that difference is subtracted from each volatile market in the portfolio even though nobody pays it. The README tables use simple returns for that reason. This lesson's system calculates the result in money, contract by contract, and does not have that problem. ## What Carver's approach solves, and what it does not - **It separates the idea from the size.** Each rule only has to produce a forecast in risk units; the size is decided by a common formula. Adding a rule does not force you to rewrite the system. - **It makes markets comparable.** With volatility in points and the risk of one contract, a bond and an index enter the same formula. - **It puts diversification in numbers.** FDM and IDM say how much extra risk can be taken per rule and per market because they do not move together, with a cap in case the correlations lie. - **It does not solve** the choice of rules or weights (that is research, with all the biases of module 2.5), nor the tail risk of large bond positions (lesson 3.1.3), nor granularity with little capital. ## Exercises 1. Repeat the buffer comparison with costs of two ticks per side instead of half a tick. Does the Sharpe still fall with the buffer? **Solution.** ```python coste_lado = (2 * esp.loc[MERCADOS, "valor_tick"] + esp.loc[MERCADOS, "comision"]) * cambio filas = {} for nombre, b in [("no buffer", 0.0), ("buffer 10%", 0.10)]: bruto, coste = backtest(contratos(optimo, media, b)) filas[nombre] = {**resumen(bruto - coste), "coste_pct": 100 * coste.sum() / 15} coste_lado = (0.5 * esp.loc[MERCADOS, "valor_tick"] + esp.loc[MERCADOS, "comision"]) * cambio pd.DataFrame(filas).T.round(2) ``` ```output sharpe vol_pct rent_pct coste_pct no buffer 0.80 16.53 13.15 2.95 buffer 10% 0.77 16.34 12.56 2.24 ``` With two ticks per side, costs go from 0.91% to 2.95% a year without a buffer and from 0.69% to 2.24% with it, and the net Sharpe falls to 0.80 and 0.77. The difference in favor of not using a buffer shrinks (from 0.06 to 0.03), but it does not change sign: in this system, with average positions of several contracts in almost every market and forecasts that move slowly, what is lost by moving away from the optimum weighs more than what is saved. The buffer is not an automatic improvement: its width is chosen by looking at the turnover and the cost per trade of each market, and it is tested like any other parameter. The cell leaves `coste_lado` as it was so as not to change the following ones. 2. With a capital of 250,000 USD, how many markets would have an average position of less than one contract? What happens to the realized risk of the system? **Solution.** ```python pequena = idm * w_i * 250_000 * TAU / riesgo_contrato print("markets with average position < 1:", int((pequena.loc["2025-11-26"] < 1).sum()), "of", len(MERCADOS)) N = contratos(combinada / 10 * pequena, pequena) pyg = (N.shift(1) * dif * mult * cambio).sum(axis=1) / 250_000 r = pyg.loc["2011":] print(f"realized vol {100 * r.std() * np.sqrt(252):.1f} %, " f"days without a position in RVA " f"{100 * (N['RVA'].loc['2011':] == 0).mean():.0f} %") ``` ```output markets with average position < 1: 19 of 23 realized vol 13.9 %, days without a position in RVA 78 % ``` With 250,000 USD, 19 of the 23 markets have an average position of less than one contract: their whole-contract position jumps between 0 and 1 (or 2), and they spend a good part of the time with no position (in `RVA`, 78% of the days). Realized volatility falls to 13.9%, far from the 20% target, and the real diversification is much lower than the IDM assumes. The solutions are micros and minis (lesson 3.1.1), fewer markets with more weight each, or accepting the tracking error. 3. Why is a rule's scalar estimated by pooling all the markets and not one per market? What would happen if each market had its own, estimated with the whole sample? **Solution.** Because the scalar should only normalize the scale of the rule, which is almost the same in every market when the rule is in risk units. A per-market scalar estimated with the whole sample would have two problems. The first, noise: in a market with few long trends, the average absolute value is low and the scalar high, and the system would give more weight exactly where the rule has been weakest. The second, look-ahead: if the scalar is calculated with future data, the 2012 position depends on how the rule did in 2020. Pooling markets and fixing the scalar with an initial window (or with an expanding window) avoids both. ## Self-assessment 1. Why does Carver divide the difference of the averages of an EWMAC rule by the volatility of the price in points? - A. So that the forecast measures the strength of the trend in risk units, comparable between a bond that moves 0.46 points a day and an index that moves 131 - B. So that the forecast is always positive - C. So that the system trades less - D. Because the broker requires it *Answer A: So that the forecast measures the strength of the trend in risk units, comparable between a bond that moves 0.46 points a day and an index that moves 131.* Without normalizing, the difference of averages of RVA would be hundreds of times that of TPC just because of the price scale. Divided by the daily volatility in points, both are measured in standard deviations and the same scalar can be used in every market. 2. The EWMAC 16/64 scalar estimated with 23 markets from 2011 to 2013 is 3.74 and the pysystemtrade example configuration uses 3.75. What does it indicate? - A. That the data is copied from pysystemtrade - B. That the scalar depends mostly on the shape of the rule and little on the market: with the forecast in risk units, the average absolute value of the raw rule is similar in any series whose volatility is estimated in a similar way - C. That the scalar has to be re-estimated every day - D. That the rule works *Answer B: That the scalar depends mostly on the shape of the rule and little on the market: with the forecast in risk units, the average absolute value of the raw rule is similar in any series whose volatility is estimated in a similar way.* The scalar only normalizes the scale of the rule, and that scale depends on the windows of the averages and on how volatility is measured. That is why it is estimated by pooling markets and is stable: with 2014-2025 it comes out at 4.01. It says nothing about whether the rule makes money. 3. Three forecasts with correlations from 0.34 to 0.61 and weights 0.25, 0.25 and 0.50 give an FDM of 1.263. What is it for? - A. To leverage the system more for no reason - B. To reduce costs - C. So that the combination, which has less dispersion than each forecast separately because they are weakly correlated, gets back to an average absolute value close to 10 - D. To correct survivorship bias *Answer C: So that the combination, which has less dispersion than each forecast separately because they are weakly correlated, gets back to an average absolute value close to 10.* A weighted average of weakly correlated forecasts moves less than each one. Without the FDM, the system would have less risk than the target. The FDM is 1/√(w′ρw), with a cap (2.5 in pysystemtrade). 4. In the lesson, the 10% buffer reduces trades from 2,135 to 1,132 a year and costs from 0.91% to 0.69% a year, but the net Sharpe falls from 0.92 to 0.86. How should it be interpreted? - A. That the buffer is a mistake - B. That a 50% buffer should be used - C. That the backtest is wrong - D. That with these costs, low relative to volatility, what is saved does not make up for what is lost by moving away from the optimal position; the buffer gains value when costs are higher *Answer D: That with these costs, low relative to volatility, what is saved does not make up for what is lost by moving away from the optimal position; the buffer gains value when costs are higher.* The buffer trades costs for tracking error. Here costs take less than one point a year and the Sharpe difference (0.06) is small relative to its standard error. With higher costs (more expensive markets, smaller capital) the balance changes. 5. The combination (Sharpe 0.86) returns less than the EWMAC 16/64 rule alone (0.99). What conclusion is reasonable? - A. Keep only EWMAC 16/64 - B. That carry is useless in every market - C. No strong one: with 15 years the standard error of a Sharpe is about 0.25, the rules are correlated and picking the best one after the fact is overfitting; the combination has less volatility and does not depend on a single rule - D. That the scalars have to be changed *Answer C: No strong one: with 15 years the standard error of a Sharpe is about 0.25, the rules are correlated and picking the best one after the fact is overfitting; the combination has less volatility and does not depend on a single rule.* Picking the rule that came out best in the backtest is the multiple testing error (lesson 2.1.8). Carver combines rules because nobody knows in advance which one will do better; the combination reduces the dependence on a single one. ## Key points - A forecast is a continuous number in risk units: EWMAC = (fast average − slow average) / price volatility in points; carry = annual carry / annual volatility. It is scaled so that its average absolute value is 10 and capped at ±20. - The scalars are estimated by pooling markets and are stable: 3.74 for EWMAC 16/64 and 1.76 for 64/256 with 2011-2013, close to the 3.75 and 1.87 of the pysystemtrade example configuration. - The combination is rescaled with the FDM (1.263 with three forecasts) and the allocation across markets with the IDM (2.81 uncapped, 2.5 capped): diversification allows more risk per market for the same total risk. - Contracts = forecast/10 × IDM × weight × capital × risk target / annual risk of one contract, in whole contracts and with a buffer of 10% of the average position. - Net of costs, the system gives a Sharpe of 0.86 with a volatility of 16.3% and a maximum drawdown of 30.1% (2011-2025), and 0.96, 0.61 and 1.02 by five-year period. The data is synthetic with a planted trend: the result validates the code, not the strategy. - A portfolio return is calculated with arithmetic returns: averaging weighted log returns gives a Sharpe of 0.23 for the same buy and hold that, measured properly, gives 0.54. ## Resources ### Research - [Volatility Targeting in Python](https://pythonparatrading.com/en/volatility-targeting-2/) · Research-0004 · 7 min ### Documentation - [pysystemtrade: default values (cap of 20, average absolute value of 10, multipliers capped at 2.5, 10% buffer)](https://github.com/robcarver17/pysystemtrade/blob/a62269d27707806ba61f516791b71a9cd14cf268/sysdata/config/defaults.yaml) · github.com - [pysystemtrade: chapter 15 example configuration (EWMAC and carry scalars, FDM, IDM)](https://github.com/robcarver17/pysystemtrade/blob/a62269d27707806ba61f516791b71a9cd14cf268/systems/provided/futures_chapter15/futuresconfig.yaml) · github.com - [pysystemtrade: the EWMAC rule (ewmac.py)](https://github.com/robcarver17/pysystemtrade/blob/a62269d27707806ba61f516791b71a9cd14cf268/systems/provided/rules/ewmac.py) · github.com ### Books - [Robert Carver: Systematic Trading (Harriman House, 2015)](https://www.harriman-house.com/authors/robert-carver/systematic-trading/9780857194459) · harriman-house.com - [Robert Carver: Advanced Futures Trading Strategies (Harriman House, 2023)](https://www.harriman-house.com/authors/robert-carver/advanced-futures-trading-strategies/9780857199683) · harriman-house.com ### Data - [Level 3 data: futures curves (Parquet)](https://pythonparatrading.com/curso/datos/nivel-3/futuros3_curvas.parquet) · Download · futuros3_curvas.parquet Previous lesson: [3.1.3 · Roll management, margins, SPAN and implied leverage](https://pythonparatrading.com/en/python-trading-course/level-3/futures-roll-margins-span-leverage/) Next lesson: [3.2.1 · Calendar spreads, butterflies and condors](https://pythonparatrading.com/en/python-trading-course/level-3/calendar-spreads-butterflies-condors/)