--- title: "6.5.2 Automated backtest review and code migrations" description: "Static and dynamic reviewer tested on six backtests with known bugs; pandas 2 to 3 migration with a differential test, and backtrader to Nautilus." url: "https://pythonparatrading.com/en/python-trading-course/level-6/automated-backtest-review-and-code-migration/" lang: "en" image: "https://pythonparatrading.com/og/en/python-trading-course/level-6/automated-backtest-review-and-code-migration.png" --- # 6.5.2 · Automated review of backtests and code migrations > An assistant writes a backtest in a minute and ports it from one library to another in another minute. Reviewing it by hand takes longer. This lesson builds a two-layer reviewer, one layer that reads the code and another that runs it with the future cut off and with signal-free data, and tests it against six backtests with planted bugs and their fixed versions. Then it applies the same idea to two migrations: from pandas 2 to pandas 3, running the same script with both versions, and from backtrader to NautilusTrader, reconciling the orders. - Course: [Level 6: Generative AI, LLMs, agents, big data and alternative data](https://pythonparatrading.com/en/python-trading-course/level-6/) - Module: 6.5 · AI as the quant's copilot - Duration: 75 min - Prerequisites: [2.5.1 · Backtest biases: look-ahead, survivorship, selection and narrative](https://pythonparatrading.com/en/python-trading-course/level-2/backtest-biases/), [2.4.4 · backtesting.py and backtrader: two teaching engines and their status](https://pythonparatrading.com/en/python-trading-course/level-2/backtesting-py-and-backtrader/), [2.4.5 · NautilusTrader: architecture, data, strategies and the same code in backtest and live](https://pythonparatrading.com/en/python-trading-course/level-2/nautilustrader-architecture-and-strategies/), [6.5.1 · Coding assistants with a test-first workflow](https://pythonparatrading.com/en/python-trading-course/level-6/test-first-ai-coding-assistants/) ## By the end of this lesson - Write a static reviewer with Python's syntax tree that detects negative shifts, full-sample normalizations, backward fills, execution on the signal bar and survivor universes. - Run two dynamic tests on any backtest: future truncation and a placebo with signal-free prices, and know which bugs each one sees. - Measure the coverage of each layer against a bank of backtests with known bugs and their fixed versions. - Migrate code from pandas 2 to pandas 3 with the release notes, a pattern scanner and a differential test that runs the same script with both versions. - Port a strategy from backtrader to NautilusTrader with a table of equivalences and reconcile its orders with a reference, including the difference in execution price. ## Reviewing faster than code is written A coding assistant (lesson 6.5.1) or a research agent (lesson 6.4.6) produces backtests at a pace that no person can review line by line. And the bugs that matter in a backtest are not the ones that make the program fail, but the ones that make it look good: using data from the future, filling at a price that was not available, choosing the universe by looking at who survives. Lesson 2.5.1 presented them one by one; this one looks for them automatically. A useful reviewer has two layers, with opposite properties: | Layer | What it does | Advantage | Limit | |---|---|---|---| | Static | Reads the code without running it and looks for known patterns | Fast, gives the exact line, needs no data | Only sees the forms someone wrote into a rule | | Dynamic | Runs the backtest with modified data and compares | Does not depend on how the code is written | Needs data and time; says there is a problem, not where | ```mermaid flowchart TB C[Code] --> E[Static review] C --> D[Execution] D --> T[Truncation] D --> P[Signal-free placebo] E --> I[Report] T --> I P --> I I --> H[Person who decides] ``` The report does not approve or reject anything: it tells the reviewer where to look. The second half of the lesson uses the same idea, run and compare, for something an assistant does often and that is almost never reviewed well: **migrating** code to a new version of a library (pandas 2 to pandas 3) or to another library (backtrader to NautilusTrader). ## A bank of backtests with known bugs To measure a reviewer you need to know the answer: backtests with planted bugs and their fixed versions. The six in the lesson use the universe of 120 fictitious stocks from level 2 (`universo2_ajustados.parquet`: adjusted prices from 2006 to 2025, with ten stocks that stop trading, four of them through bankruptcy). The data are synthetic: they are not market data. ```python import ast import textwrap import numpy as np import pandas as pd ajustados = pd.read_parquet("universo2_ajustados.parquet") P = ajustados.pivot(index="fecha", columns="ticker", values="cierre_ajustado") ultima = P.apply(pd.Series.last_valid_index) print(P.shape, "· sessions", P.index[0].date(), "to", P.index[-1].date()) print("stop trading before the end:", int((ultima < P.index[-1]).sum())) ``` ```output (5031, 120) · sessions 2006-01-03 to 2025-12-31 stop trading before the end: 10 ``` All the backtests follow the same contract: they receive the price table `P` (sessions in rows, stocks in columns, empty when the stock is not trading) and return the daily return of the strategy. Each one has **one** one-line bug, of the kind that shows up in generated code or code written in a hurry, and its fixed version changes only that line. | Backtest | Strategy | Planted bug | |---|---|---| | B1 | 20-session momentum, long-short | `shift(-1)` on the signal: aligns it with the label instead of with the decision | | B2 | Long the stocks with a momentum z above 1 | z with the mean and standard deviation of the full sample | | B3 | 12-month momentum, top quintile | Monthly signal turned daily with `bfill()` instead of `ffill()` | | B4 | 5-session trend, long-short | Weights not lagged: trades on the close that generates the signal | | B5 | Equal-weighted | Universe = the stocks trading on the last day (survivors) | | B6 | Month's return, long-short | Return of the full month spread over all its days | ```python BASE = """ def backtest(P): R = P.pct_change(fill_method=None) {cuerpo} w = w.div(w.abs().sum(axis=1), axis=0) return (w.shift(1) * R).sum(axis=1) """ LARGO_CORTO = " w = s.sub(s.mean(axis=1), axis=0)" # correct version of the body, correct line and line with the bug BANCO = { "B1": (""" s = P.pct_change(20, fill_method=None) """ + LARGO_CORTO, "(20, fill_method=None)", "(20, fill_method=None).shift(-1)"), "B2": (""" m = P.pct_change(120, fill_method=None) z = (m - m.expanding(250).mean()) / m.expanding(250).std() w = (z > 1).astype(float).where(P.notna())""", "(m - m.expanding(250).mean()) / m.expanding(250).std()", "(m - m.mean()) / m.std()"), "B3": (""" s = P.resample("ME").last().pct_change(12, fill_method=None) s = s.reindex(P.index).ffill() w = s.rank(axis=1, pct=True).ge(0.8).astype(float) w = w.where(P.notna())""", ".ffill()", ".bfill()"), "B4": (""" s = P.pct_change(5, fill_method=None) """ + LARGO_CORTO, "(w.shift(1) * R)", "(w * R)"), "B5": (""" vivas = P.columns w = P[vivas].notna().astype(float)""", "vivas = P.columns", "vivas = P.columns[P.iloc[-1].notna()]"), "B6": (""" s = P.resample("ME").last().pct_change(fill_method=None) s = s.reindex(P.index, method="ffill") """ + LARGO_CORTO, """s = P.resample("ME").last().pct_change(fill_method=None) s = s.reindex(P.index, method="ffill")""", """g = P.groupby(P.index.to_period("M")) s = g.transform("last") / g.transform("first") - 1"""), } def codigos(nombre): """(code with the bug, fixed code) of a backtest.""" cuerpo, buena, mala = BANCO[nombre] bien = BASE.format(cuerpo=cuerpo) return bien.replace(buena, mala), bien def compilar(codigo): espacio = {"np": np, "pd": pd} exec(textwrap.dedent(codigo), espacio) return espacio["backtest"] print(codigos("B3")[0]) ``` ```output def backtest(P): R = P.pct_change(fill_method=None) s = P.resample("ME").last().pct_change(12, fill_method=None) s = s.reindex(P.index).bfill() w = s.rank(axis=1, pct=True).ge(0.8).astype(float) w = w.where(P.notna()) w = w.div(w.abs().sum(axis=1), axis=0) return (w.shift(1) * R).sum(axis=1) ``` The template is the one any assistant would produce: daily returns `R`, weights `w` normalized so that gross exposure is 1, and the P&L as the previous day's weights times today's returns. First, how much each bug changes the result: ```python def sharpe(r): r = r.loc["2007":] # without the first year return float(r.mean() / r.std() * np.sqrt(252)) coste = {} for nombre in BANCO: mal, bien = (compilar(c)(P) for c in codigos(nombre)) coste[nombre] = (sharpe(mal), sharpe(bien)) print(f"{nombre} Sharpe with bug {coste[nombre][0]:6.2f}" f" fixed {coste[nombre][1]:6.2f}") ``` ```output B1 Sharpe with bug 18.77 fixed 0.29 B2 Sharpe with bug 0.41 fixed 0.36 B3 Sharpe with bug 2.12 fixed 0.48 B4 Sharpe with bug 30.69 fixed 0.34 B5 Sharpe with bug 0.41 fixed 0.38 B6 Sharpe with bug 17.67 fixed 0.78 ``` The bugs fall into two very different groups, as the figure at the top shows. Those that use the information of a day to trade that same day (B1, B4) or that of the whole month (B6) turn strategies with a Sharpe of 0.3–0.8 into Sharpe ratios of 18 to 31: so absurd that nobody would believe them. B3, with 2.12 against 0.48, is already a result someone could show. And the two most dangerous are the ones that are barely noticeable: the full-sample normalization (B2) raises the Sharpe from 0.36 to 0.41 and the survivor universe (B5), from 0.38 to 0.41. Nobody suspects a 0.41. A reviewer has to find all of them, and especially those two, without raising warnings on the six fixed versions. ## The static layer: reading the syntax tree Searching for `shift(-` with a regular expression works until someone writes `shift(periods=-1)` or `shift( -1 )`. The standard library's `ast` module turns the code into a **syntax tree**: every call, every attribute and every argument is a node, however the author writes it. The reviewer walks the tree and applies six rules: | Rule | Pattern | Why it is suspicious | |---|---|---| | Negative shift | `shift(-k)` | Brings a value from t + k to day t | | Centered window | `rolling(..., center=True)` | The window includes future sessions | | Backward fill | `bfill()`, `backfill()`, `fillna(method="bfill")` | Copies a later value backwards | | Full sample | `.mean()`, `.std()`, `.min()`, `.max()`, `.quantile()` with no window and no `axis=1` | The statistic uses every date | | Same bar | `* R` with no `shift` on the other factor | Trades on the close that generates the signal | | Survivors | `.iloc[-1]` or `dropna(axis=1)` | Chooses the universe with the last data point | ```python VENTANAS = {"rolling", "expanding", "ewm", "groupby", "resample"} TODA_MUESTRA = {"mean", "std", "min", "max", "quantile", "median"} def metodo(nodo): """Name of the method if the node is a call x.method(...).""" if (isinstance(nodo, ast.Call) and isinstance(nodo.func, ast.Attribute)): return nodo.func.attr return None def negativo(nodo): return (isinstance(nodo, ast.UnaryOp) and isinstance(nodo.op, ast.USub)) def revision_estatica(codigo): avisos = [] def aviso(nodo, texto): avisos.append(f"l.{nodo.lineno - 1} {texto}") for n in ast.walk(ast.parse(textwrap.dedent(codigo))): m = metodo(n) kw = {k.arg: k.value for k in getattr(n, "keywords", [])} if m == "shift" and (n.args and negativo(n.args[0]) or negativo(kw.get("periods"))): aviso(n, "negative shift") if m == "rolling" and getattr(kw.get("center"), "value", False): aviso(n, "centered window") if m in ("bfill", "backfill") or getattr( kw.get("method"), "value", "") in ("bfill", "backfill"): aviso(n, "backward fill") if m in TODA_MUESTRA and metodo(n.func.value) not in VENTANAS: eje = getattr(kw.get("axis"), "value", 0) if eje not in (1, "columns"): aviso(n, f".{m}() over the full sample") if (isinstance(n, ast.BinOp) and isinstance(n.op, ast.Mult) and any(isinstance(x, ast.Name) and x.id == "R" for x in (n.left, n.right))): otro = n.right if isinstance(n.left, ast.Name) else n.left if metodo(otro) != "shift": aviso(n, "weights not lagged (same bar)") if (isinstance(n, ast.Subscript) and isinstance(n.value, ast.Attribute) and n.value.attr == "iloc" and negativo(n.slice)): aviso(n, "universe from the last data point") if m == "dropna" and getattr(kw.get("axis"), "value", 0) in (1, "columns"): aviso(n, "universe from the last data point") return avisos estatica = {} for nombre in BANCO: mal, bien = codigos(nombre) estatica[nombre] = (revision_estatica(mal), revision_estatica(bien)) print(f"{nombre} with bug: {estatica[nombre][0] or '-'}") print(f" fixed: {estatica[nombre][1] or '-'}") ``` ```output B1 with bug: ['l.4 negative shift'] fixed: - B2 with bug: ['l.5 .std() over the full sample', 'l.5 .mean() over the full sample'] fixed: - B3 with bug: ['l.5 backward fill'] fixed: - B4 with bug: ['l.7 weights not lagged (same bar)'] fixed: - B5 with bug: ['l.4 universe from the last data point'] fixed: - B6 with bug: - fixed: - ``` Five out of six, with the exact line (counted from `def backtest`): the `shift(-1)` of B1 on line 4, the full-sample mean and standard deviation of B2 on line 5, the `bfill()` of B3 on line 5, the multiplication without `shift` of B4 on line 7 and the `iloc[-1]` of B5 on line 4. No warnings on the fixed versions: the full-sample rule does not fire with `s.mean(axis=1)` (a cross-sectional mean of each day, which is legitimate) or with `m.expanding(250).mean()`. B6 passes clean. Its bug, `g.transform("last")` on a monthly grouper, gives every day of the month the last price of the month, which is not yet known. There is nothing suspicious about the form: grouping and spreading a value is everyday pandas. A rule could be written for that case (exercise 1), but there will always be another way of looking at the future that has no rule. That is what the second layer is for. Rules come at a price: false positives. The same rule that catches the survivor universe fires on harmless code that reads the last value for a report: ```python informe = """ def resumen(curva): final = curva.iloc[-1] return {"final capital": final, "mean": curva.mean()} """ revision_estatica(informe) ``` ```output ['l.2 universe from the last data point', 'l.3 .mean() over the full sample'] ``` Two warnings and no bug: the last value of an equity curve and its mean are legitimate in a report. That is why the reviewer is applied only to the code that decides (the `backtest` function), not to the whole repository, and why its warnings are questions for a person, not verdicts. A reviewer that warns about everything ends up ignored. ## The dynamic layer: cutting off the future and removing the signal The dynamic layer does not read the code: it runs it twice with different data and compares. The first test is the **truncation** of lessons 0.2.6 and 6.5.1, applied to a complete backtest: what the backtest computes up to date T cannot change if the data after T are removed. It is tested with several cutoffs, because a bug can depend on what happens after each one (the survivor universe only changes if some stock stops trading after the cutoff). The second is a **placebo**: prices with the same shape as the real ones (the same stocks, the same dates on which they trade, the same volatility for each one) but with independent returns, in which there is nothing to predict. With them, the Sharpe of any bug-free strategy has to be compatible with zero: with 19 years, its standard error is $1/\sqrt{19} \approx 0.23$. A Sharpe above 1 on the placebo is not a finding, it is a bug. ```python CORTES = ["2010-11-15", "2014-06-13", "2018-12-14", "2022-06-15"] def prueba_truncado(f, P): """Sessions up to each cutoff in which the P&L changes.""" completo = f(P) distintas = 0 for corte in CORTES: parcial = f(P.loc[:corte]) a, b = completo.loc[:corte], parcial distintas += int((~np.isclose(a, b, equal_nan=True)).sum()) return distintas def placebo(P, semilla=652): """Independent zero-mean returns with the volatility of each stock and its same trading period.""" rng = np.random.default_rng(semilla) vol = P.pct_change(fill_method=None).std().to_numpy() r = rng.standard_normal(P.shape) * vol falso = 50 * np.cumprod(1 + r, axis=0) return pd.DataFrame(falso, P.index, P.columns).where(P.notna()) P0 = placebo(P) dinamica = {} for nombre in BANCO: fila = [] for codigo in codigos(nombre): f = compilar(codigo) fila.append((prueba_truncado(f, P), sharpe(f(P0)))) dinamica[nombre] = fila (t_mal, s_mal), (t_bien, s_bien) = fila print(f"{nombre} with bug: truncation {t_mal:5d} sessions " f"changed, placebo {s_mal:6.2f} | fixed: " f"{t_bien} and {s_bien:5.2f}") ``` ```output B1 with bug: truncation 0 sessions changed, placebo 33.89 | fixed: 0 and 0.27 B2 with bug: truncation 7900 sessions changed, placebo -0.42 | fixed: 0 and -0.22 B3 with bug: truncation 7853 sessions changed, placebo 6.03 | fixed: 0 and 0.27 B4 with bug: truncation 0 sessions changed, placebo 69.73 | fixed: 0 and -0.10 B5 with bug: truncation 10752 sessions changed, placebo -0.08 | fixed: 0 and 0.06 B6 with bug: truncation 37 sessions changed, placebo 29.17 | fixed: 0 and 0.51 ``` The two tests see different things: - **Truncation** finds B2, B3, B5 and B6, all the ones that use data after the cutoff: the full-sample mean changes when the sample is cut (7,900 sessions changed across the four cutoffs), as does the backward fill from the next month (7,853) and the universe of the stocks trading on the last day (10,752). B6 only changes in 37 sessions: those of the months split by the cutoffs, which is why the cutoffs fall mid-month. With month-end cutoffs it would not have seen it. - Truncation **does not see B1 or B4**: both use the close of session t for the P&L of session t itself, and when cutting at T session T is still there. The **placebo** catches them: on prices that have nothing to predict, B1 gives a Sharpe of 33.89 and B4 of 69.73. B3 (6.03) and B6 (29.17) too. The fixed versions pass both tests: zero sessions changed and placebo Sharpe ratios between −0.22 and 0.51, within what chance gives. The placebo uses zero-mean arithmetic returns; with prices following a random walk in logarithms, the expected arithmetic return would be positive ($\sigma^2/2$ per day) and a long-only portfolio would give a positive Sharpe without any bug. ## Two layers together The final report brings together the three tests for each backtest, with the false alarms on the fixed versions: ```python filas = [] for nombre in BANCO: (t_mal, s_mal), (t_bien, s_bien) = dinamica[nombre] filas.append({ "backtest": nombre, "static": bool(estatica[nombre][0]), "truncation": t_mal > 0, "placebo": abs(s_mal) > 1, "false alarms": int(bool(estatica[nombre][1])) + int(t_bien > 0) + int(abs(s_bien) > 1), }) matriz = pd.DataFrame(filas).set_index("backtest") matriz["any"] = matriz[["static", "truncation", "placebo"]].any(axis=1) matriz ``` ```output static truncation placebo false alarms any backtest B1 True False True 0 True B2 True True False 0 True B3 True True True 0 True B4 True False True 0 True B5 True True False 0 True B6 False True True 0 True ``` *Fig. — Which reviewer layer catches each bug in the bank: the static layer catches five of six; truncation and the placebo, four each; together, all six, with no false alarm on the fixed versions.* Table of 6 rows by 3 columns. B1 shift(-1): static yes, truncation no, placebo yes. B2 full-sample z: yes, yes, no. B3 bfill: yes, yes, yes. B4 same bar: yes, no, yes. B5 survivors: yes, yes, no. B6 whole month: static no (highlighted), truncation yes, placebo yes. The static layer catches five out of six; truncation, four; the placebo, four; together, all six, and none raises a false alarm on the fixed versions. No layer is redundant: B6 is seen only by the dynamic ones, B1 and B4 only by the static layer and the placebo, and B2 and B5, the two subtle bugs, only by the static layer and truncation. The placebo cannot see them: in B2 and B5 the edge does not come from knowing future returns, but from using future data to choose which stocks or which threshold to use, and with signal-free returns that gives nothing (exercise 2). It is a small, tailor-made bank: a real reviewer is measured with many more bugs, and its thresholds (the placebo Sharpe of 1, the four cutoffs) are tuned on fixed versions so as not to drown whoever uses it in warnings. ## pandas 2 to 3: what changes pandas 3.0.0 was released on January 21, 2026 and the course uses it from level 0 (in the main environment, version 3.0.6). Its release notes recommend a specific path: first upgrade to pandas 2.3 and get the code free of warnings, because 3.0 removes much of what 2.x marked as deprecated. The changes that most affect trading code, according to those notes and the official migration guide: | Change | What happened in pandas 2 | What happens in pandas 3 | |---|---|---| | Copy-on-write | A selection could be a view: modifying it modified the original | Every selection behaves as a copy; chained assignment does nothing and emits `ChainedAssignmentError` | | String type | Strings were `object` | `str` type by default (with PyArrow if installed); the missing value is always `NaN` | | Date resolution | Always nanoseconds | Inferred: when parsing text, microseconds | | Frequency aliases | `M`, `Q`, `Y`, `H`, `T` | `ME`, `QE`, `YE`, `h`, `min` (the old ones raise an error) | | Removed APIs | `fillna(method=…)`, `applymap`, `first`/`last` by date, `swapaxes`… | Removed: use `ffill`/`bfill`, `map`, `loc` | | Integer keys | `s[0]` on a series with dates was positional (with a warning) | An integer key is always a label: `KeyError` | | Time zones | `pytz` objects | The standard library's `zoneinfo`; `pytz` is no longer a dependency | A rule from the migration guide sums up the first row: if you want to modify an object, the only way is to modify that object directly. `df["a"][mascara] = x` modifies a temporary copy and does nothing; `df.loc[mascara, "a"] = x` modifies `df`. ### The same script with both versions The most reliable way to know what changes is to run the same code with both versions and compare. The course has both interpreters: the one in the main environment (pandas 3.0.6) and the one in the Zipline environment (pandas 2.3.3, lesson 2.4.6). On your machine any virtual environment with `pandas==2.3.3` will do. The cell writes a script with nine constructs that are common in old code and runs it with each interpreter; warnings are collected and shown by category: ```python import subprocess import sys from pathlib import Path PY3 = sys.executable PY2 = str(Path(sys.executable).parents[2] / "entornos" / "zipline" / ".venv" / "bin" / "python") Path("casos.py").write_text(''' import warnings import numpy as np import pandas as pd def caso(nombre, f): with warnings.catch_warnings(record=True) as w: warnings.simplefilter("always") try: r = f() except Exception as e: r = type(e).__name__ avisos = sorted({x.category.__name__ for x in w}) print(f"{nombre}|{r}|{' '.join(avisos)}") idx = pd.date_range("2024-01-02", periods=6, freq="B") s = pd.Series([1.0, np.nan, 3.0, np.nan, 5.0, 6.0], idx) df = pd.DataFrame({"tk": ["AAA", "BBB", None, "AAA", "BBB", "AAA"], "p": [10.0, 11.0, 12.0, 13.0, 14.0, 15.0]}, idx) def encadenada(): d = df.copy() d["p"][d["p"] > 13] = 0.0 return d["p"].tolist()[-3:] caso("fillna(method=)", lambda: s.fillna(method="ffill").iloc[1]) caso("applymap", lambda: df[["p"]].applymap(abs).iloc[0, 0]) caso("resample('M')", lambda: len(s.resample("M").last())) caso("first('3D')", lambda: len(s.first("3D"))) caso("s[0] with dates", lambda: s[0]) caso("chained assignment", encadenada) caso("dtype == object", lambda: df["tk"].dtype == object) caso("astype(str) of None", lambda: repr(df["tk"].astype(str).iloc[2])) caso("date to int64", lambda: int(pd.to_datetime( ["2024-01-02"]).astype("int64")[0])) print(f"version|pandas {pd.__version__}|") ''') def ejecutar(python, script): r = subprocess.run([python, "-I", script], capture_output=True, text=True, check=True) return [linea.split("|") for linea in r.stdout.splitlines()] def celda(valor, aviso): aviso = aviso.removesuffix("Warning") return f"{valor} ({aviso})" if aviso else valor for a, b in zip(ejecutar(PY2, "casos.py"), ejecutar(PY3, "casos.py")): print(f"{a[0]:<22}{celda(a[1], a[2]):<35}{celda(b[1], b[2])}") ``` ```output fillna(method=) 1.0 (Future) TypeError applymap 10.0 (Future) AttributeError resample('M') 1 (Future) ValueError first('3D') 3 (Future) AttributeError s[0] with dates 1.0 (Future) KeyError chained assignment [13.0, 0.0, 0.0] (SettingWithCopy) [13.0, 14.0, 15.0] (ChainedAssignmentError) dtype == object True False astype(str) of None 'None' nan date to int64 1704153600000000000 1704153600000000 version pandas 2.3.3 pandas 3.0.6 ``` Five constructs fail in pandas 3 with an exception: `fillna(method=)` with `TypeError`, `applymap` and `first()` with `AttributeError`, the `M` alias with `ValueError` and `s[0]` on a date index with `KeyError`. They are the easy ones: the program stops and says where. pandas 2.3 warned about all of them with a `FutureWarning`, the path the release notes recommend. The other four do not fail. The chained assignment does not modify the DataFrame (the last three values are still 13, 14 and 15) and only leaves a warning, `ChainedAssignmentError`, which in pandas 2.3 was `SettingWithCopyWarning`. And three changes give **no warning in either version**: `dtype == object` goes from `True` to `False` because the ticker column is now `str`; `astype(str)` leaves the missing value as `nan` instead of turning it into the text `'None'`; and a date parsed from text converts to integers in microseconds, not nanoseconds: 1,000 times smaller. A filter on text columns that stops filtering, a `None` that is no longer a string and timestamps divided by a thousand break nothing; they change results. ### Migrating a backtest and checking it An old backtest of the monthly portfolio of `KAPA` (the synthetic ETF of lessons 2.4.4 and 2.4.5): long the next month if the month was positive, with a cost of 10 bp on each change of position. It is written for pandas 2 and does not work in pandas 3. Before touching it, a **migration scanner** built on the syntax tree, like the bias reviewer, lists what needs to change: ```python b5 = pd.read_parquet("indice2_5min.parquet") sesion = (b5["ts"].dt.tz_convert("America/New_York") .dt.tz_localize(None).dt.normalize()) kapa = b5.groupby(sesion)["cierre"].last().rename("close") kapa.index.name = "fecha" kapa.to_csv("kapa_diario.csv") ANTIGUO = '''import pandas as pd precios = pd.read_csv("kapa_diario.csv", index_col=0, parse_dates=True)["close"] precios = precios.fillna(method="ffill") mensual = precios.resample("M").last() res = pd.DataFrame({"ret": mensual.pct_change()}) res["pos"] = (res["ret"] > 0).astype(float).shift(1) res["pnl"] = res["pos"] * res["ret"] cambios = res["pos"].diff().abs() > 0 res["pnl"][cambios] -= 0.001 print(f"first close {precios[0]:.2f}, months {len(res)}, " f"capital {(1 + res['pnl'].fillna(0)).prod():.6f}") ''' ALIAS = {"M": "ME", "Q": "QE", "Y": "YE", "A": "YE", "H": "h", "T": "min"} def escaner_migracion(codigo): avisos = [] for n in ast.walk(ast.parse(codigo)): m = metodo(n) kw = {k.arg: k.value for k in getattr(n, "keywords", [])} if m == "fillna" and "method" in kw: avisos.append((n.lineno, "fillna(method=) removed: " "ffill() / bfill()")) if m in ("resample", "asfreq", "date_range") and n.args: f = getattr(n.args[0], "value", None) if f in ALIAS: avisos.append((n.lineno, f"alias '{f}' removed: " f"'{ALIAS[f]}'")) if m == "applymap": avisos.append((n.lineno, "applymap removed: map")) if isinstance(n, ast.Assign): destinos = n.targets else: destinos = [getattr(n, "target", None)] if any(isinstance(d, ast.Subscript) and isinstance(d.value, ast.Subscript) for d in destinos): avisos.append((n.lineno, "chained assignment: does not " "assign under copy-on-write; use .loc")) if (isinstance(n, ast.Subscript) and isinstance(n.slice, ast.Constant) and type(n.slice.value) is int): avisos.append((n.lineno, "integer key: in pandas 3 it is " "a label; use .iloc")) return sorted(avisos) for linea, texto in escaner_migracion(ANTIGUO): print(f"line {linea}: {texto}") ``` ```output line 4: fillna(method=) removed: ffill() / bfill() line 5: alias 'M' removed: 'ME' line 10: chained assignment: does not assign under copy-on-write; use .loc line 11: integer key: in pandas 3 it is a label; use .iloc ``` Four warnings with their line: the `fillna(method=)` on line 4, the `M` alias on line 5, the chained assignment on line 10, which subtracts the cost in the months with a change of position, and the integer key `precios[0]` on line 11. Now the differential test, with five runs: the old script with each version, a migration that only fixes what raises an error in pandas 3, and the full migration, in both versions. ```python import re SOLO_ERRORES = (ANTIGUO .replace('precios.fillna(method="ffill")', "precios.ffill()") .replace('resample("M")', 'resample("ME")') .replace("precios[0]", "precios.iloc[0]")) MIGRADO = SOLO_ERRORES.replace('res["pnl"][cambios] -= 0.001', 'res.loc[cambios, "pnl"] -= 0.001') def correr(python, codigo, nombre="bt.py"): """Script output and categories of the warnings or the error.""" Path(nombre).write_text(codigo) r = subprocess.run([python, "-I", nombre], capture_output=True, text=True) avisos = sorted(set(re.findall(r": (\w+(?:Warning|Error)): ", r.stderr))) if r.returncode: return r.stderr.strip().splitlines()[-1].split(":")[0], [] return r.stdout.strip(), avisos for etiqueta, python, codigo in [ ("old, pandas 2.3", PY2, ANTIGUO), ("old, pandas 3.0", PY3, ANTIGUO), ("errors only, pandas 3.0", PY3, SOLO_ERRORES), ("migrated, pandas 3.0", PY3, MIGRADO), ("migrated, pandas 2.3", PY2, MIGRADO)]: salida, avisos = correr(python, codigo) print(f"{etiqueta:<25} {salida}") if avisos: print(" " * 26 + "warnings: " + ", ".join(avisos)) ``` ```output old, pandas 2.3 first close 252.38, months 48, capital 2.112132 warnings: FutureWarning old, pandas 3.0 TypeError errors only, pandas 3.0 first close 252.38, months 48, capital 2.152774 warnings: ChainedAssignmentError migrated, pandas 3.0 first close 252.38, months 48, capital 2.112132 migrated, pandas 2.3 first close 252.38, months 48, capital 2.112132 ``` The table tells the whole story: 1. The old script works in pandas 2.3 (with `FutureWarning`): final capital 2.112132. 2. In pandas 3 it stops with `TypeError`. 3. Fixing **only what raises an error**, it runs and gives 2.152774: **more** capital than the original. The chained assignment no longer subtracts the costs, the script does not fail and the only trace is a `ChainedAssignmentError` on the error output, which nobody reads in an unattended run. It is exactly the migration made by whoever (person or assistant) is chasing to make the code “work again”. 4. The full migration, with `res.loc[cambios, "pnl"] -= 0.001`, gives 2.112132 in pandas 3 and in pandas 2.3: the same result as the original, with the same code in both versions. That is the acceptance criterion for a migration: **the same result as the old version, with the old version and with the new one**, or each difference explained. For a real backtest, “the same result” means the same trades and the same equity curve up to rounding, not just the final number. ## backtrader to NautilusTrader The other frequent migration is between libraries: from backtrader, which has not published releases on PyPI since April 2023 (lesson 2.4.4), to NautilusTrader, an actively maintained event-driven engine (lesson 2.4.5). An assistant translates the syntax without trouble; what it does not translate on its own is the **semantics**: when an order is filled, at what price, with what default size. The table of equivalences, from both sets of documentation (consulted on October 23, 2026; NautilusTrader 1.231.0): | backtrader | NautilusTrader | Difference that matters | |---|---|---| | `Cerebro` | `BacktestEngine` (or `BacktestNode` with configuration and catalog) | The engine is assembled with venue, instrument, data and strategy | | `bt.feeds.PandasData` | `Bar` + `BarType` + `BarDataWrangler` | The bar's timestamp has to be that of its **close** | | `params = (("rapida", 10),)` | `StrategyConfig` subclass (`frozen=True`) | Immutable, typed configuration | | Indicators in `__init__` | Indicators in `__init__` + `register_indicator_for_bars` in `on_start` | The engine updates them before `on_bar` | | `next()` | `on_bar(bar)` | Same: one call per closed bar | | `self.data.close[0]` / `[-1]` | `bar.close` / `self.cache.bar(bar_type, 1)` | The cache indexes the other way round: 0 is the latest | | `self.buy()` | `self.order_factory.market(...)` + `self.submit_order(...)` | No default quantity: backtrader's sizer buys 1 unit | | `self.close()` | `self.close_all_positions(instrument_id)` | | | `self.position` | `self.portfolio.is_net_long(instrument_id)` | The position is queried from the `Portfolio` | | `notify_order` / `notify_trade` | `on_order_filled`, `on_order_event` / `on_position_closed` | Typed events | | `broker.setcash`, `setcommission` | `add_venue(starting_balances=…)`, `fee_model` | Commissions through a model (`FixedFeeModel`, `MakerTakerFeeModel`…) | | Market order: **open of the next bar** | Without latency: **close of the bar in which it is sent** | No native next-open mode | The last row is the one that changes the results. backtrader's documentation says that a market order is filled at the “opening price of the next bar”; NautilusTrader's, that without a latency model an order sent from `on_bar` is filled against the book left by that bar's close, and that the engine “does not offer a native mode for execution at the open of the next bar”. ```mermaid flowchart LR S[Close of t: signal] --> B[backtrader] S --> N[Nautilus] B --> A[Open of t+1] N --> C[Close of t] ``` The strategy is the `SmaCross` of lesson 2.4.4 (10- and 30-session averages, long only) on the daily bars of `KAPA`. There, with its own adapter, it gave 33 orders, and the first was created on March 10, 2022 and filled on the 11th at 250.74. The daily bars carry the 16:00 New York timestamp, their close. To be able to imitate the next open, a one-minute bar per session is also prepared, at 9:31, with the opening price in its four fields: ```python import warnings from pandas.errors import Pandas4Warning # The engine calls Timestamp.utcnow(), deprecated in pandas 3: # one warning per run, outside our control (lesson 2.4.5) warnings.filterwarnings("ignore", message="Timestamp.utcnow", category=Pandas4Warning) from nautilus_trader.model.currencies import USD from nautilus_trader.model.data import BarType from nautilus_trader.model.identifiers import ( InstrumentId, Symbol, Venue) from nautilus_trader.model.instruments import Equity from nautilus_trader.model.objects import Money, Price, Quantity from nautilus_trader.persistence.wranglers import BarDataWrangler diario = b5.groupby(sesion).agg( open=("apertura", "first"), high=("maximo", "max"), low=("minimo", "min"), close=("cierre", "last"), volume=("volumen", "sum")) def en_utc(fechas, hora, minuto): return ((fechas + pd.Timedelta(hours=hora, minutes=minuto)) .tz_localize("America/New_York").tz_convert("UTC")) VENUE = Venue("XNYS") kapa_n = Equity( instrument_id=InstrumentId(Symbol("KAPA"), VENUE), raw_symbol=Symbol("KAPA"), currency=USD, price_precision=2, price_increment=Price.from_str("0.01"), lot_size=Quantity.from_int(1), ts_event=0, ts_init=0) dia = diario.set_axis(en_utc(diario.index, 16, 0)) BT_DIA = BarType.from_str("KAPA.XNYS-1-DAY-LAST-EXTERNAL") barras_dia = BarDataWrangler(BT_DIA, kapa_n).process(dia) apertura = pd.DataFrame( {c: diario["open"] for c in ("open", "high", "low", "close")}) apertura["volume"] = 1_000.0 apertura.index = en_utc(diario.index, 9, 31) BT_AP = BarType.from_str("KAPA.XNYS-1-MINUTE-LAST-EXTERNAL") barras_ap = BarDataWrangler(BT_AP, kapa_n).process(apertura) print(len(barras_dia), "daily bars;", len(barras_ap), "opening bars") ``` ```output 1003 daily bars; 1003 opening bars ``` The translation of `SmaCross`. With `barra_apertura` empty it is the literal translation: it decides and sends in `on_bar` of the daily bar. With it, it decides on the daily bar and sends the order when it receives the opening bar of the next session: ```python from nautilus_trader.backtest.engine import BacktestEngine from nautilus_trader.config import ( BacktestEngineConfig, LoggingConfig, StrategyConfig) from nautilus_trader.indicators import SimpleMovingAverage from nautilus_trader.model.enums import ( AccountType, OmsType, OrderSide) from nautilus_trader.trading.strategy import Strategy class SmaCrossConfig(StrategyConfig, frozen=True): instrument_id: InstrumentId bar_type: BarType barra_apertura: BarType | None = None rapida: int = 10 lenta: int = 30 acciones: int = 100 # backtrader would buy 1 (SizerFix) class SmaCross(Strategy): def __init__(self, config: SmaCrossConfig) -> None: super().__init__(config) self.rapida = SimpleMovingAverage(config.rapida) self.lenta = SimpleMovingAverage(config.lenta) self.previo = None # sign of fast - slow self.pendiente = None def on_start(self) -> None: cfg = self.config self.register_indicator_for_bars(cfg.bar_type, self.rapida) self.register_indicator_for_bars(cfg.bar_type, self.lenta) self.subscribe_bars(cfg.bar_type) if cfg.barra_apertura is not None: self.subscribe_bars(cfg.barra_apertura) def cruce(self): """BUY, SELL or None, like bt.ind.CrossOver.""" if not self.indicators_initialized(): return None d = np.sign(self.rapida.value - self.lenta.value) previo, self.previo = self.previo, d largo = self.portfolio.is_net_long(self.config.instrument_id) if previo is None: return None if d > 0 and previo <= 0 and not largo: return OrderSide.BUY if d < 0 and previo >= 0 and largo: return OrderSide.SELL return None def on_bar(self, bar) -> None: if bar.bar_type == self.config.bar_type: lado = self.cruce() con_apertura = self.config.barra_apertura is not None if lado is not None and con_apertura: self.pendiente = lado # at the open elif lado is not None: self.enviar(lado) # at the close elif self.pendiente is not None: self.enviar(self.pendiente) self.pendiente = None def enviar(self, lado) -> None: self.submit_order(self.order_factory.market( self.config.instrument_id, lado, Quantity.from_int(self.config.acciones))) def ejecutar_nautilus(con_apertura): motor = BacktestEngine(BacktestEngineConfig( logging=LoggingConfig(log_level="ERROR"))) motor.add_venue(venue=VENUE, oms_type=OmsType.NETTING, account_type=AccountType.CASH, base_currency=USD, starting_balances=[Money(100_000, USD)]) motor.add_instrument(kapa_n) motor.add_data(barras_dia) if con_apertura: motor.add_data(barras_ap) motor.add_strategy(SmaCross(SmaCrossConfig( instrument_id=kapa_n.id, bar_type=BT_DIA, barra_apertura=BT_AP if con_apertura else None))) motor.run() f = motor.trader.generate_order_fills_report() sesion_ny = (pd.to_datetime(f["ts_last"]) .dt.tz_convert("America/New_York") .dt.tz_localize(None).dt.normalize()) return pd.DataFrame({"fecha": sesion_ny.to_numpy(), "lado": f["side"].astype(str).to_numpy(), "precio": f["avg_px"].astype(float) .to_numpy()}) literal = ejecutar_nautilus(False) imitada = ejecutar_nautilus(True) print("literal:", len(literal), "orders; imitated:", len(imitada)) imitada.head(3) ``` ```output literal: 33 orders; imitated: 33 fecha lado precio 0 2022-03-11 BUY 250.74 1 2022-03-31 SELL 232.00 2 2022-05-10 BUY 241.52 ``` The reference, with pandas and the same rule as the reconciliation in lesson 2.4.4: crossovers of the 10- and 30-session averages, buy on an upward cross if out, sell on a downward cross if in, and the fill price according to the convention: ```python c, o = diario["close"], diario["open"] x = np.sign(c.rolling(10).mean() - c.rolling(30).mean()) alza = (x > 0) & (x.shift(1) <= 0) baja = (x < 0) & (x.shift(1) >= 0) filas, dentro = [], False for i in range(len(c)): if (alza.iloc[i] and not dentro) or (baja.iloc[i] and dentro): dentro = not dentro filas.append((i, "BUY" if dentro else "SELL")) def referencia(al_cierre): out = [] for i, lado in filas: j = i if al_cierre else i + 1 if j < len(c): precio = c.iloc[j] if al_cierre else o.iloc[j] out.append((c.index[j], lado, round(float(precio), 2))) return pd.DataFrame(out, columns=["fecha", "lado", "precio"]) def coinciden(a, b): n = min(len(a), len(b)) a, b = a.iloc[:n].reset_index(drop=True), b.iloc[:n] b = b.reset_index(drop=True) fecha = (a["fecha"] == b["fecha"]) & (a["lado"] == b["lado"]) precio = fecha & np.isclose(a["precio"], b["precio"]) return int(fecha.sum()), int(precio.sum()), len(a), len(b) for nombre, nau, ref in [ ("literal vs close", literal, referencia(True)), ("literal vs next open", literal, referencia(False)), ("imitated vs next open", imitada, referencia(False))]: f, p, n1, n2 = coinciden(nau, ref) print(f"{nombre:<31} {n1} and {n2} orders; date and side {f}," f" price too {p}") ``` ```output literal vs close 33 and 33 orders; date and side 33, price too 33 literal vs next open 33 and 33 orders; date and side 0, price too 0 imitated vs next open 33 and 33 orders; date and side 33, price too 33 ``` The literal translation matches order by order, in date, side and price, the reference **at the close**: all 33. Against the reference at the next open, backtrader's convention, none match (all carry the previous date). The version with the opening bar reproduces backtrader's in all 33 orders, in date and in price; its first buy is the one on March 11, 2022 at 250.74, the same one the adapter in lesson 2.4.4 gave. Both translations are correct NautilusTrader code and run without a single warning. Only the reconciliation says which one reproduces what was there: it is the pandas differential test applied to a library migration. Which of the two conventions is the right one is not decided by the code, but by the market: filling at the close of the signal bar is only honest if the signal can be computed and the order sent before the close (lesson 1.1.4); otherwise, the next open is the prudent convention, and with large overnight gaps the difference can be the whole result. Two more details from the table of equivalences that an automatic translation skips: the size (backtrader buys 1 unit unless told otherwise; here, 100 shares set explicitly) and the commissions (none in either, so that they can be compared). ## What can be automated and what cannot | Step | Automated | For a person | |---|---|---| | Known look-ahead patterns | Static reviewer on every change | Decide whether each warning is a bug or a false positive | | Use of the future in any form | Truncation with several cutoffs | Choose cutoffs that cross the events that matter (delistings, rebalances) | | One-day leaks and execution on the bar | Placebo with signal-free prices | Set the threshold and repeat with other seeds if it is close | | Version migration | Pattern scanner and differential test | Explain every remaining difference | | Library migration | Order-by-order reconciliation with a reference | Decide which semantics is correct for the asset | Nothing in the table replaces thinking about the economic mechanism of the strategy (lesson 2.5.1), but everything in the “Automated” column can run in a repository's continuous integration, like the tests in lesson 6.5.1, and turns reviewing an assistant's proposal into reading a ten-line report. Lesson 6.5.3 covers the risks that none of these tests see: dependencies that do not exist, secrets in the code and code that looks correct and is not. ## Exercises 1. Add to the static reviewer a rule for the bug in B6: a `groupby(...).transform` with `"last"` or `"max"` on a period grouper. Does it now detect B6 without raising warnings on its fixed version or on the others? **Solution.** ```python def revision_ampliada(codigo): avisos = revision_estatica(codigo) for n in ast.walk(ast.parse(textwrap.dedent(codigo))): if (metodo(n) == "transform" and n.args and getattr(n.args[0], "value", None) in ("last", "max", "min")): avisos.append(f"l.{n.lineno - 1} transform('" f"{n.args[0].value}') spreads a later " "value") return avisos for nombre in BANCO: mal, bien = codigos(nombre) print(nombre, len(revision_ampliada(mal)), len(revision_ampliada(bien))) ``` ```output B1 1 0 B2 2 0 B3 1 0 B4 1 0 B5 1 0 B6 1 0 ``` With the new rule, B6 gives one warning with the bug and none fixed, and the rest of the bank does not change: each version with a bug keeps its warnings (B2 with two) and no fixed one raises a false alarm. The rule only looks at `"last"`, `"max"` and `"min"`, which are the aggregates that look at the end of the group; `"first"`, which B6 also uses, is harmless if the group starts before the decision. It is one more rule on a list that is never complete: the dynamic layer is still necessary. 2. The placebo threshold (a Sharpe of 1) was set with a single seed. Repeat the placebo with five seeds for the fixed versions and for B2 and B5 with the bug. What is the largest |Sharpe| of a fixed version? Does the placebo detect B2 or B5 with any seed? **Solution.** ```python for nombre in BANCO: mal, bien = (compilar(c) for c in codigos(nombre)) s_bien = [sharpe(bien(placebo(P, s))) for s in range(5)] linea = f"{nombre} fixed max |S| {max(map(abs, s_bien)):.2f}" if nombre in ("B2", "B5"): s_mal = [sharpe(mal(placebo(P, s))) for s in range(5)] linea += f" · with bug max |S| {max(map(abs, s_mal)):.2f}" print(linea) ``` ```output B1 fixed max |S| 0.21 B2 fixed max |S| 0.45 · with bug max |S| 0.65 B3 fixed max |S| 0.45 B4 fixed max |S| 0.25 B5 fixed max |S| 0.31 · with bug max |S| 0.33 B6 fixed max |S| 0.40 ``` With five seeds, the largest |Sharpe| of a fixed version is 0.45 (B2 and B3), far from the threshold of 1. And the placebo does not detect B2 or B5 with any seed (maximums of 0.65 and 0.33): as seen in the matrix, those two bugs do not depend on knowing future returns, and with signal-free returns they give no edge. That is why the placebo does not replace truncation. If a fixed version came close to the threshold, the right thing would be to repeat with more seeds and set the threshold from the distribution of those versions, not to raise it by eye. 3. Read the table of the test with nine constructs. Which of the changes give no warning in pandas 2.3? What test would you write for each one? **Solution.** Three changes give no warning in pandas 2.3 or in pandas 3: `dtype == object` (`True` becomes `False`), `astype(str)` with missing values (`'None'` becomes `nan`) and the conversion of dates to `int64` (from nanoseconds to microseconds). The chained assignment does warn in both, but in pandas 3 it no longer does anything. One test for each: - Text columns: check with `pd.api.types.is_string_dtype(columna)`, which the migration guide recommends because it works in both versions, and a test that verifies the column filter returns the expected ones. - Missing values: a test with a `None` in a text column that checks the result with `pd.isna`, never comparing with `None` or with the string `'None'`. - Dates to integers: do not convert with `astype("int64")`; if needed, set the unit (`.as_unit("ns")`) and a test that checks a known value, for example that midnight on January 2, 2024 is 1,704,153,600 seconds since 1970. - Chained assignment: a test that checks that the P&L with costs is lower than without costs in a case with changes of position; the lesson's differential test would have found it anyway. ## Self-assessment 1. A backtest returns the daily P&L and passes the truncation test: what it computes up to T does not change when the data after T are removed. What bug can it still have? - A. None: truncation detects any use of the future - B. A survivor universe - C. A full-sample normalization - D. Using the information of session t to trade in session t itself (execution on the signal bar, or a `shift(-1)` that moves a day forward): truncation keeps session T, so that leak is not visible *Answer D: Using the information of session t to trade in session t itself (execution on the signal bar, or a `shift(-1)` that moves a day forward): truncation keeps session T, so that leak is not visible.* When cutting at T, the data of T are still there. A bug that uses the close of t for the P&L of t passes truncation. In the lesson, B1 (`shift(-1)`) and B4 (weights not lagged) pass it and are caught by the placebo with signal-free prices, where their Sharpe ratios rise to 34 and 70. 2. Why does the static reviewer not detect the bug in B6 (the month's return with `groupby(...).transform("last")`)? - A. Because the code has none of the patterns on the list: grouping by month and spreading the last value is normal pandas usage; only running it with the future cut off reveals that it uses later data - B. Because B6 has no bug - C. Because the reviewer does not analyze functions with `groupby` - D. Because the bug is in the data, not in the code *Answer A: Because the code has none of the patterns on the list: grouping by month and spreading the last value is normal pandas usage; only running it with the future cut off reveals that it uses later data.* Static rules recognize specific forms (`shift(-k)`, `bfill`, `.mean()` without a window). There are infinitely many ways of looking at the future that have none of them. That is why the two layers complement each other: the static one is fast and says where the problem is; the dynamic one does not depend on the form. 3. In the differential test, the old script with only the errors fixed gives a higher final capital in pandas 3 than in pandas 2.3. What happened? - A. pandas 3 computes returns better - B. The chained assignment that subtracted the costs no longer does anything under copy-on-write: the script does not fail, it only emits a `ChainedAssignmentError`, and the backtest comes out without costs - C. Resampling with `ME` changes the dates - D. It is a rounding error *Answer B: The chained assignment that subtracted the costs no longer does anything under copy-on-write: the script does not fail, it only emits a `ChainedAssignmentError`, and the backtest comes out without costs.* `res["pnl"][cambios] -= coste` modified the original DataFrame in pandas 2. With copy-on-write it modifies a copy that is discarded. It is the most dangerous silent change of the migration, and pandas 2.3 already warned about it; the correct form is `res.loc[cambios, "pnl"] -= coste`. 4. You port backtrader's SmaCross to NautilusTrader by sending the market order in `on_bar`. The 33 orders match backtrader's in date, but not in price. Why? - A. Because NautilusTrader charges commissions by default - B. Because of the rounding of `Price` to two decimals - C. Because backtrader fills market orders at the open of the next bar and NautilusTrader, without a latency model, at the close of the bar in which they are sent; its documentation says it has no native mode for execution at the next open - D. Because the moving averages of the two libraries are different *Answer C: Because backtrader fills market orders at the open of the next bar and NautilusTrader, without a latency model, at the close of the bar in which they are sent; its documentation says it has no native mode for execution at the next open.* The literal translation reproduces the `coc` mode (fill at the close of the signal bar). To imitate the next open, the lesson adds an opening bar and sends the order when it arrives: that way the 33 orders match in date and price. 5. What is needed to accept a migration of a backtest? - A. That the new code runs without errors or warnings - B. That the assistant says it has reviewed the changes - C. That the unit tests pass with the new version - D. A differential test: the same backtest with the old version and with the new one gives the same trades and the same result, or the differences are explained one by one *Answer D: A differential test: the same backtest with the old version and with the new one gives the same trades and the same result, or the differences are explained one by one.* The most dangerous behavior changes raise no errors: the chained assignment that does not assign, the resolution of dates, the type of strings. They only show up when results are compared. Unit tests help, but they only cover what someone thought of testing. ## Key points - A backtest reviewer has two layers: the static one reads the syntax tree and flags known patterns with their line; the dynamic one runs the backtest with the future cut off and with signal-free prices, without depending on how it is written. - Against six backtests with planted bugs, the static layer caught five, truncation four and the placebo four; together, all six. None raised a warning on the six fixed versions. - Truncation does not see one-day leaks (B1 and B4): the placebo catches them, where those backtests give Sharpe ratios of 34 and 70 on prices that have nothing to predict. - pandas 3.0 brings copy-on-write, the `str` type by default, dates with inferred resolution and the removal of many deprecated APIs. What fails with an error is easy; the danger is in what is silent: chained assignment, `dtype == object`, `astype(str)` and the conversion of dates to integers. - The differential test runs the same script with pandas 2.3 and with pandas 3 and compares the outputs: it detects the migration that fixes the errors but leaves the backtest without costs. - backtrader fills at the next open and NautilusTrader at the close of the order's bar: a literal translation changes every fill price. A migration is accepted when it reconciles with the reference, order by order. ## Resources ### Research - [Your first system in Python: moving average crossover on SPY](https://pythonparatrading.com/en/first-system-moving-average-crossover/) · Research-0018 · 7 min ### Documentation - [pandas: what's new in version 3.0.0 (January 21, 2026)](https://pandas.pydata.org/docs/whatsnew/v3.0.0.html) · pandas.pydata.org - [pandas: migration guides (copy-on-write and the new string type)](https://pandas.pydata.org/docs/user_guide/migration.html) · pandas.pydata.org - [NautilusTrader: bar execution in backtests](https://nautilustrader.io/docs/latest/concepts/backtesting/bar-execution/) · nautilustrader.io - [backtrader: order creation and execution](https://www.backtrader.com/docu/order-creation-execution/order-creation-execution/) · backtrader.com - [Python: ast module (abstract syntax trees)](https://docs.python.org/3/library/ast.html) · docs.python.org ### Data - [Level 2 data: adjusted prices of the universe (Parquet)](https://pythonparatrading.com/curso/datos/nivel-2/universo2_ajustados.parquet) · Download · universo2_ajustados.parquet Previous lesson: [6.5.1 · Coding assistants with a test-first workflow](https://pythonparatrading.com/en/python-trading-course/level-6/test-first-ai-coding-assistants/) Next lesson: [6.5.3 · Risks of AI-generated code](https://pythonparatrading.com/en/python-trading-course/level-6/risks-of-ai-generated-code/)