Foundation models in finance: what the evidence says.

The hard question: tomorrow's return

Lessons 5.8.1 to 5.8.3 explained what a time series foundation model is, what families there are (Chronos, TimesFM, Moirai, Kronos…) and how to use them without retraining. This lesson asks something else: what is known about their performance in finance, and specifically on the task that matters most to anyone who trades, forecasting returns.

It is the toughest possible test for a pretrained model. Daily returns have an extremely low signal-to-noise ratio, fat tails, regime changes and almost no autocorrelation (lesson 1.6.2). A model that has learned patterns from thousands of traffic, energy or sales series finds little to recognize in them, and whatever little there is gets arbitraged away by other participants. The honest question is not “which model wins?” but “does any of them beat forecasting zero, with what statistical confidence and with what value after costs?”. A claim that a model “predicts” has to travel this whole chain:

flowchart TB
  A[Forecast] --> B[Loss and baseline]
  B --> C[Robust test]
  C --> D[Multiple testing]
  D --> E[Costs and capacity]
  E --> F[Conclusion]

Each link has its trap. The baseline must be the naive one (zero return or historical mean), not another complex model. The loss decides what gets rewarded: squared error punishes extreme days, absolute error targets the median. The test has to account for the fact that the losses of consecutive days or overlapping horizons are not independent. With many models, assets and variants, something comes out significant by chance (lesson 5.4.4). And a slightly better forecast may not pay the costs of trading it (lesson 5.1.3).

The 2026 benchmark: rankings versus the random walk

The syllabus cites a benchmark posted on arXiv on June 25, 2026: “Pretrained Time-Series Foundation Models for Financial Return Forecasting”, by Miquel Noguer i Alonso and Rodolfo Pereira Franklin (Artificial Intelligence Finance Institute), arXiv:2606.27100. What follows comes from the abstract page and the PDF of version 1, read on 2026-10-23. The design:

The results the syllabus repeats are in the paper as stated: the pretrained models win 8 of the 10 tasks, and Moirai-2.0 and TimesFM-2.5 have the best average ranks, 2.9 and 3.1 (table 4; they are followed by Chronos with 3.4, Chronos-2 with 3.9, iTransformer with 4.3 and KAN with 5.4). iTransformer, trained from scratch, wins the two META tasks. And the abstract puts it bluntly: “gains over the random-walk benchmark are small and sparse”.

The figure plots table 5 of the paper: the skill of the winner of each task. In six of the ten (AAPL, JPM and META, on both targets) the best model in the comparison has a larger absolute error than forecasting a zero return: skill from −2.9 to −5.2 %. The four improvements are in AMZN (8.6 and 8.2 %) and GOOG (22.9 and 22.8 %). The one-sided Diebold-Mariano test (table 7) only rejects the hypothesis of “equal to or worse than the random walk” in two cases, Chronos on AMZN and Moirai-2.0 on GOOG, both with p = 0.0421; for TimesFM-2.5 on AAPL and JPM, p = 0.9999. In table 4, each of the four best-ranked pretrained models has positive skill in 4 of the 10 tasks; the two trained from scratch, in none.

Three readings that the paper itself supports:

• Ranking is not predictability. Ordering eleven models tells you which is the least bad, not whether any of them is useful. The abstract phrases it like this: “strong rankings need not imply economically meaningful predictability in noisy markets”.
• Significance is exploratory. The tests cover several models, stocks and targets and the p-values are reported uncorrected; the authors ask readers to take the significant cases “as exploratory evidence rather than as family-level discoveries” and acknowledge that many differences among the top of the ranking are within the noise floor of the test window (about 200 sessions).
• It is not a trading test. The paper evaluates forecast error, not usefulness: calibration, turnover, costs and capacity are left out, and the authors state explicitly that their results should not be read as evidence of alpha.

The other study: strategies with thousands of stocks

The syllabus cites “another study” in which strategies built on these models perform worse than specific reference models. It is “Re(Visiting) Time Series Foundation Models in Finance”, by Eghbal Rahimikia (University of Manchester), Hao Ni (UCL) and Weiguan Wang (Shanghai University), arXiv:2511.18578, posted on November 23, 2025; the figures come from the PDF of version 1. Its design is very different from the previous one: daily excess returns of about 10,000 US stocks (more than 18 million observations), out-of-sample evaluation from 2001 to 2023 with models re-estimated every year, windows of 5, 21, 252 and 512 sessions, and decile long-short portfolios, equally weighted and rebalanced daily, built from each model's forecasts. The reference models are linear (OLS, Lasso, Ridge, Elastic Net, PCR), tree-based (XGBoost, CatBoost, LightGBM) and two small networks; the best is CatBoost.

The results of the no-retraining (zero-shot) mode use the public versions of Chronos (v1, from tiny to large) and of TimesFM (version 1 with 200 million parameters and version 2 with 500 million), not Chronos-Bolt or TimesFM-2.5. With a 512-session window and no costs, the best configuration, Chronos large, gives an annualized return of 20.17 % and a Sharpe of 2.92; the best TimesFM, −1.47 % and −0.18; CatBoost with the same window, 47.25 % and 6.46. In goodness of fit, with a 512 window, the out-of-sample R² of Chronos large is −1.37 % and that of the 500-million TimesFM, −2.80 %; CatBoost's, averaged over the four windows, −0.10 %. Fine-tuning on financial data does not close the gap. What does narrow it is pretraining from scratch on returns, year by year and without seeing the future: Chronos small pretrained this way reaches 36.84 % and a Sharpe of 5.42.

The figure adds what the study shows once costs are included: at 20 basis points or more, every long-short portfolio turns to a negative Sharpe, whatever the model. CatBoost's average Sharpe over the four windows falls from 6.44 without costs to −3.62 with 20 basis points and to −13.59 with 40. A Sharpe of 6 without costs on a portfolio rebuilt every day over thousands of stocks is a statistical result (the signal exists), not a strategy.

A methodological detail of this second study applies to any work with public models: the authors warn (note 17) that, depending on the data each model was pretrained on, there may be look-ahead bias: if the pretraining corpus contains the same financial series from the test period, the model has seen the future. That is why they pretrain their own models with no overlap. In this course's synthetic data the problem does not exist: none of these series was in the pretraining of Chronos-Bolt or TimesFM-2.5.

Volatility: where there is structure

The syllabus conclusion (“useful for volatility, covariates and series with structure”) has support, with nuances. Two recent papers, of which only what their arXiv abstracts say is cited here:

• Brini (2026), arXiv:2607.05291, compares nine foundation models without retraining with eight econometric specifications (the HAR family included) on 50 equity, currency and futures assets: “Foundation models do not deliver a uniform gain”. Averaging by asset against a well-specified Log-HAR, only one small model (Tiny Time Mixers) beats it at every horizon, and narrowly; a Mincer-Zarnowitz recalibration shows that much of the short-term advantage is a matter of scale, not dynamics; and the simple average of that model and the Log-HAR enters the Model Confidence Set for 98-100 % of the assets.
• Goel, Pasricha, Magris and Kanniainen (2025), arXiv:2505.1116311Note 1 · Goel, A., Pasricha, P., Magris, M. and Kanniainen, J. (2025): Foundation Time-Series AI Model for Realized Volatility Forecasting, arXiv:2505.11163, https://arxiv.org/abs/2505.11163., evaluate TimesFM on volatility: without retraining it is “a reasonable baseline”, and with incremental fine-tuning it outperforms traditional models under Diebold-Mariano and Giacomini-White tests.

Volatility has what returns lack: persistence, long memory and a recognizable shape (clustering, mean reversion) that resembles that of many series in the pretraining corpus. That is why a foundation model forecasts it reasonably well without tuning, and for the same reason a four-parameter HAR is a very tough rival. Lesson 5.8.5 measures this with a full lab.

The in-house test: returns unpredictable by construction

The studies use real data, in which nobody knows how much predictability there is. The course's synthetic data allow something no paper can do: test the models where the answer is known. In vol5_diario.parquet (README of the level 5 data, “Control” section) the conditional mean of the return is constant: $E[r_t \mid t-1] = \mu$ exactly, with fat tails and skewness. No model can beat the historical mean except by chance. It is the perfect control for a foundation model: if it “finds” something, it is measuring noise.

The protocol mimics the benchmark's, with a rolling origin and no leakage: at each origin $t$ the model receives the 512 previous sessions (from $t-512$ to $t-1$) and forecasts $r_t$. So that the lesson runs in a couple of minutes on CPU, one in every 40 sessions from 2018 to 2025 is taken on the six assets; real research would use every day and more series.

flowchart LR
  A["512 sessions"] --> B[Untuned model]
  B --> C["Forecast of r_t"]
  C --> D["Loss vs base"]

import numpy as np
import pandas as pd

d = pd.read_parquet("vol5_diario.parquet")
ACT = list(d["activo"].unique())
g = d.groupby("activo", sort=False)
r = {a: x["rendimiento"].to_numpy() for a, x in g}
rv = {a: x["rv_5min"].to_numpy() for a, x in g}
fechas = pd.DatetimeIndex(d.loc[d["activo"] == "indice", "fecha"])

CTX = 512                                   # context: 512 sessions
corte = fechas.searchsorted(pd.Timestamp("2018-01-02"))
orig = np.arange(corte, len(fechas), 40)    # one in every 40 sessions
act = np.repeat(ACT, len(orig))
y = np.concatenate([r[a][orig] for a in ACT])
ctx = [r[a][t - CTX:t].astype("float32") for a in ACT for t in orig]
media = np.concatenate([[r[a][:t].mean() for t in orig] for a in ACT])
print(f"{len(orig)} origins per asset, {len(y)} forecasts, "
      f"{fechas[orig[0]].date()} to {fechas[orig[-1]].date()}")

51 origins per asset, 306 forecasts, 2018-01-02 to 2025-12-16

There are two baselines. The benchmark's, the random walk on the price, which amounts to forecasting a zero return. And the one from the predictability literature (Campbell and Thompson, 2008; Welch and Goyal, 200822Note 2 · Welch, I. and Goyal, A. (2008): A Comprehensive Look at the Empirical Performance of Equity Premium Prediction. The Review of Financial Studies 21(4), 1455-1508, https://doi.org/10.1093/rfs/hhm014.), the historical mean of returns up to the previous day. Both models are used without retraining (how to load and configure them is in lesson 5.8.3), with the median as the point forecast.

import torch

torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
from chronos import BaseChronosPipeline
import timesfm

chronos = BaseChronosPipeline.from_pretrained(
    "amazon/chronos-bolt-small", device_map="cpu", dtype=torch.float32)
q, _ = chronos.predict_quantiles(
    inputs=[torch.tensor(c) for c in ctx], prediction_length=1,
    quantile_levels=[0.1, 0.5, 0.9])
f_ch = q[:, 0, 1].numpy().astype(float)        # forecast median

tfm = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
    "google/timesfm-2.5-200m-pytorch", torch_compile=False)
tfm.compile(timesfm.ForecastConfig(
    max_context=CTX, max_horizon=16, normalize_inputs=True,
    use_continuous_quantile_head=True, per_core_batch_size=len(ctx)))
punto, _ = tfm.forecast(horizon=1, inputs=ctx)
f_tf = punto[:, 0].astype(float)

prev = {"zero": np.zeros_like(y), "mean": media,
        "chronos": f_ch, "timesfm": f_tf}
pd.DataFrame({k: [np.std(v) / np.std(y), 100 * np.mean(v > 0)]
              for k, v in prev.items()},
             index=["std / std of r", "% forecasts > 0"]).round(3)

                 zero    mean  chronos  timesfm
std / std of r    0.0   0.028    0.072    0.112
% forecasts > 0   0.0  83.333   62.745   68.301

per_core_batch_size=len(ctx) makes TimesFM process the 306 series at once: the default value, 1, passes them one by one and on CPU multiplies the time. The two models' forecasts are small, as they should be: their standard deviation is 7.2 % (Chronos) and 11.2 % (TimesFM) of that of the return, more than that of the historical mean (2.8 %) but very far from what would be needed to explain anything. Both lean upward (63 % and 68 % of the forecasts are positive), like the mean, which is positive at 83 % of the origins because five of the six assets have μ > 0.

Skill against forecasting zero

The skill is the benchmark's: $1 - \text{MAE}_{\text{model}} / \text{MAE}_{\text{zero}}$, and its version with the mean squared error (MSE). With the historical mean as the base and the MSE, it is the out-of-sample $R^2$ of Campbell and Thompson:

def skill(f, base, m, perdida):
    """1 - mean loss of the model / mean loss of the base, in %."""
    L = (lambda p: (y[m] - p[m]) ** 2) if perdida == "mse" else (
        lambda p: np.abs(y[m] - p[m]))
    return 100 * (1 - L(f).mean() / L(base).mean())

filas = {}
for a in ACT:
    m = act == a
    filas[a] = {f"{k}_{p}": skill(prev[k], prev["zero"], m, p)
                for k in ["mean", "chronos", "timesfm"]
                for p in ["mse", "mae"]}
t = pd.DataFrame(filas).T
t.loc["asset mean"] = t.mean()
t.round(2)

            mean_mse  mean_mae  chronos_mse  chronos_mae  timesfm_mse  timesfm_mae
indice         -1.02      0.31        -6.55        -2.45        -1.72        -0.16
accion          0.12      1.26        -0.44         0.76         1.07        -0.76
bono            0.08      0.15        -3.63        -1.75        -6.33        -2.83
oro             0.94     -0.35        -0.38        -2.50         2.94        -0.43
divisa         -0.79     -0.53         3.88         2.03         2.38         0.89
cripto          1.01      0.43         2.28         1.86        -0.88        -1.12
asset mean      0.06      0.21        -0.81        -0.34        -0.42        -0.74

The table looks a lot like the benchmark's: skill of a few points, sometimes positive and sometimes negative. Chronos ranges from −6.55 % (index, MSE) to +3.88 % (currency); TimesFM, from −6.33 % (bond) to +2.94 % (gold). Averaged over the six assets, both lose against forecasting zero with both losses (from −0.34 to −0.81 %), and the historical mean comes out even (+0.06 and +0.21 %). If this table came from real data, someone could write that “TimesFM improves 2.9 % on gold” or “Chronos 3.9 % on currencies”. Here we know there is nothing to improve: those figures are the size of the noise with 51 forecasts.

Diebold-Mariano and the multiple-testing correction

To know whether a difference in losses is noise you need a test. The Diebold and Mariano (1995) test compares the losses of two forecasts of the same target, $d_t = L(e^{(1)}_t) - L(e^{(2)}_t)$, and tests whether their mean is zero with a variance that allows for autocorrelation (the long-run variance, $\hat\sigma^2_\infty$, with autocovariances up to $h-1$ for a horizon $h$):

$DM^*$ is the Harvey, Leybourne and Newbold (1997) small-sample correction, which is compared with a Student's t with $n-1$ degrees of freedom; it is the one the 2026 benchmark uses. Here $h = 1$ and the alternative hypothesis is one-sided: the model has a lower loss than the base ($DM^* < 0$).

from scipy import stats

def dm(l_mod, l_base, h=1):
    """Diebold-Mariano with the Harvey, Leybourne and Newbold
    correction. One-sided H1: the model has a lower loss."""
    dd = l_mod - l_base
    n = len(dd)
    u = dd - dd.mean()
    var = u @ u / n
    for k in range(1, h):                 # autocovariances up to h-1
        var += 2 * (u[k:] @ u[:-k]) / n
    t = dd.mean() / np.sqrt(var / n)
    t *= np.sqrt((n + 1 - 2 * h + h * (h - 1) / n) / n)
    return t, stats.t.cdf(t, df=n - 1)

pruebas = []
for a in ACT:
    m = act == a
    for k in ["chronos", "timesfm"]:
        for b in ["zero", "mean"]:
            for p, L in [("mse", np.square), ("mae", np.abs)]:
                t_, p_ = dm(L(y[m] - prev[k][m]), L(y[m] - prev[b][m]))
                pruebas.append((a, k, b, p, t_, p_))
pr = pd.DataFrame(pruebas, columns=["asset", "model", "base",
                                    "loss", "t", "p"])
print(len(pr), "tests; p < 0.05:", (pr["p"] < 0.05).sum())
pr.sort_values("p").head(4).round(3)

48 tests; p < 0.05: 1
     asset    model  base loss      t      p
34  divisa  chronos  mean  mse -1.704  0.047
32  divisa  chronos  zero  mse -1.551  0.064
41  cripto  chronos  zero  mae -1.512  0.068
43  cripto  chronos  mean  mae -1.486  0.072

Six assets, two models, two baselines and two losses make 48 tests, and one comes out “significant”: Chronos against the mean on the currency, with the squared error, t = −1.704 and p = 0.047. It is the asset with μ = 0 and no dependence at all between return and volatility (ρ = 0 in the README): the least predictable of the six. With 48 tests at 5 %, about 2.4 rejections are expected even if nothing works (somewhat fewer here, because the tests share data and are correlated). The figure shows all 48:

Most of the statistics are positive: the models lose more than the baselines, which is what you expect when a forecast adds variance without information. The standard way to control the family-wise error is to adjust the p-values. Holm (1979) sorts them and compares the smallest with $0.05/48$, the next with $0.05/47$, and so on; it is uniformly more powerful than Bonferroni and does not assume independence.

from statsmodels.stats.multitest import multipletests

rech, p_holm, _, _ = multipletests(pr["p"], alpha=0.05, method="holm")
print("rejections with Holm:", rech.sum(),
      "| minimum adjusted p:", round(p_holm.min(), 3))

rejections with Holm: 0 | minimum adjusted p: 1.0

No rejection survives: the minimum adjusted p is 1 (0.047 × 48 exceeds 1 and is truncated). It is the same logic as the Deflated Sharpe of lesson 5.4.4 applied to forecast tests. And it helps to read the benchmark: two rejections with p = 0.0421 within a family of models, stocks and targets would not survive Holm either, as its authors acknowledge when they ask for them to be read as exploratory.

The illusion of levels

A mistake that shows up in demos of foundation models on financial data: feeding the model the price and measuring accuracy on the price. The cell builds a price from each asset's returns, asks Chronos for tomorrow's price and compares it with two things: the sample mean of the price (the usual R²) and yesterday's price (the random walk). Then it translates the forecast into an implied return, $\hat P_t / P_{t-1} - 1$.

precio = {a: 100 * np.cumprod(1 + r[a]) for a in ACT}
ctx_p = [precio[a][t - CTX:t].astype("float32")
         for a in ACT for t in orig]
qp, _ = chronos.predict_quantiles(
    inputs=[torch.tensor(c) for c in ctx_p], prediction_length=1,
    quantile_levels=[0.5])
p_ch = qp[:, 0, 0].numpy().astype(float)
p_hoy = np.concatenate([precio[a][orig] for a in ACT])
p_ayer = np.concatenate([precio[a][orig - 1] for a in ACT])

def r2(obs, f, base):
    sse, sst = np.sum((obs - f) ** 2), np.sum((obs - base) ** 2)
    return 100 * (1 - sse / sst)

filas = {}
for a in ACT:
    m = act == a
    r_impl = p_ch[m] / p_ayer[m] - 1        # implied return
    filas[a] = {
        "R2_price": r2(p_hoy[m], p_ch[m], p_hoy[m].mean()),
        "MAPE_chronos": 100 * np.mean(np.abs(p_ch[m] / p_hoy[m] - 1)),
        "MAPE_yesterday": 100 * np.mean(np.abs(p_ayer[m] / p_hoy[m] - 1)),
        "R2_impl_ret": r2(y[m], r_impl, media[m])}
pd.DataFrame(filas).T.round(2)

        R2_price  MAPE_chronos  MAPE_yesterday  R2_impl_ret
indice     99.82          0.82            0.74       -22.38
accion     99.77          1.02            0.92        -0.48
bono       99.37          0.34            0.31       -42.84
oro        99.84          0.59            0.45       -40.46
divisa     98.94          0.38            0.36       -16.07
cripto     99.27          2.92            2.70       -17.85

The R² of the price ranges from 98.94 to 99.84 %, and a mean absolute percentage error (MAPE) of 0.3 to 2.9 % sounds precise. Both figures are an illusion: the price is so persistent that repeating yesterday's gives a lower MAPE than Chronos on all six assets (0.74 versus 0.82 % on the index, 2.70 versus 2.92 % on crypto). The return implied by the forecast loses against the historical mean on all of them, with an R² from −0.48 to −42.84 %: by forecasting the level, the model adds noise to yesterday's price. The header figure puts this row next to the others.

The same model where there is structure

The final contrast uses the same Chronos-Bolt, the same origins and the same context, but with the log of the realized variance instead of the return. In vol5 that series follows a HAR in logs with persistence 0.92: there is a lot of structure to learn from the past. The base is again the historical mean, now of the log RV.

ctx_v = [np.log(rv[a][t - CTX:t]).astype("float32")
         for a in ACT for t in orig]
qv, _ = chronos.predict_quantiles(
    inputs=[torch.tensor(c) for c in ctx_v], prediction_length=1,
    quantile_levels=[0.5])
v_ch = qv[:, 0, 0].numpy().astype(float)
lv = np.log(np.concatenate([rv[a][orig] for a in ACT]))
lv_media = np.concatenate([[np.log(rv[a][:t]).mean() for t in orig]
                           for a in ACT])
r2_v = {a: r2(lv[act == a], v_ch[act == a], lv_media[act == a])
        for a in ACT}
r2_r = {a: r2(y[act == a], f_ch[act == a], media[act == a])
        for a in ACT}
pd.DataFrame({"R2_log_RV": r2_v, "R2_return": r2_r}).round(1)

        R2_log_RV  R2_return
indice       67.7       -5.5
accion       24.1       -0.6
bono         48.6       -3.7
oro          56.0       -1.3
divisa       29.2        4.6
cripto       44.8        1.3

With volatility, the same model with no tuning at all explains between 24.1 % (stock) and 67.7 % (index) of the variance of the log RV out of sample; with the return, between −5.5 and +4.6 %. The header figure summarizes the four rows with the average of the six assets: 99.5 % for the price (illusion), 45.1 % for the log RV (real structure), −0.9 % for the return (nothing) and −23.3 % for the return implied by the price (worse than nothing). A high R² on volatility does not yet say whether the model is good: it has to be compared with what a HAR or a GARCH would do on the same dates and with the right loss. That is the lab in lesson 5.8.5.

How to read a study of foundation models in finance

Common mistakes

• Confusing ranking with predictability. The best of eleven models can be worse than forecasting zero; in the 2026 benchmark it is in six of ten tasks.
• Evaluating prices in levels. An R² of 99 % and a MAPE of 1 % say nothing if yesterday's price does as well or better.
• Keeping the only p < 0.05 out of 48. In the lesson's control, the only “significant” one is on the asset whose mean is exactly constant.
• Forgetting the right baseline. The random walk and the historical mean are hard to beat on returns; a model that is not compared with them has proven nothing.
• Reading the Sharpe of a daily portfolio without costs. With daily turnover over thousands of stocks, 20 basis points turn a Sharpe of 6 into a negative one.
• Ignoring possible overlap. A public model pretrained on financial series may have seen the test period of the real data.

Exercises

1. Repeat the benchmark protocol with Chronos-Bolt: a 20-session horizon, mean absolute error over the 20 sessions of each origin and skill against forecasting zero, with the same origins (except the last one, which does not have 20 sessions left). Does the conclusion change? Since the origins are 40 sessions apart, the windows do not overlap and $h = 1$ in Diebold-Mariano with the mean loss of each origin is enough.

H = 20
o20 = orig[:-1]
ctx20 = [r[a][t - CTX:t].astype("float32") for a in ACT for t in o20]
q20, _ = chronos.predict_quantiles(
    inputs=[torch.tensor(c) for c in ctx20], prediction_length=H,
    quantile_levels=[0.5])
f20 = q20[:, :, 0].numpy().astype(float)          # (origins, 20)
y20 = np.array([r[a][t:t + H] for a in ACT for t in o20])
a20 = np.repeat(ACT, len(o20))
eam_m = np.abs(y20 - f20).mean(axis=1)            # MAE per origin
eam_0 = np.abs(y20).mean(axis=1)
filas = {}
for a in ACT:
    m = a20 == a
    t_, p_ = dm(eam_m[m], eam_0[m])
    sk = 100 * (1 - eam_m[m].mean() / eam_0[m].mean())
    filas[a] = {"skill_%": sk, "t": t_, "p": p_}
pd.DataFrame(filas).T.round(3)

        skill_%      t      p
indice    0.703 -1.137  0.131
accion    0.281 -0.743  0.231
bono     -0.011  0.035  0.514
oro      -0.930  2.250  0.986
divisa   -0.408  2.396  0.990
cripto   -0.227  0.615  0.729

2. Simulate the multiple-testing problem: generate 2,000 pure-noise “models” for the index (normal forecasts with the standard deviation of Chronos's forecasts on that asset) and test each one against the historical mean with the squared error. What fraction gives p < 0.05? What is the p of the best one?

rng = np.random.default_rng(5)
m = act == "indice"
ruido = rng.normal(0, f_ch[m].std(), size=(2000, m.sum()))
l_base = (y[m] - media[m]) ** 2
ps = np.array([dm((y[m] - f) ** 2, l_base)[1] for f in ruido])
print(f"p < 0.05: {np.mean(ps < 0.05):.1%} | best p: {ps.min():.4f}"
      f" | with Holm: {multipletests(ps, method='holm')[0].sum()}")

p < 0.05: 4.2% | best p: 0.0004 | with Holm: 0

3. No code: a paper claims that “a foundation model without retraining achieves a Sharpe of 3 in a daily long-short strategy with the 500 stocks of the S&P 500, 2020-2024, versus 0.4 for momentum”. What questions would you ask before believing it?