Foundation model families: architecture, data and license.

A map of the families

Lesson 5.8.1 explained what they have in common: pretrain once on a huge corpus and forecast zero-shot from a context that is standardized and cut into patches. What distinguishes each family is the transformer architecture, what data it saw, what it returns, how big it is and under what license it is released. The table summarizes what their papers and cards say (the Hugging Face cards and the READMEs of their repositories, consulted in October 2026):

The header figure puts the sizes on a log scale. They range from 11 to 710 million parameters, tiny next to a language model, and the smallest in the figure, Moirai-2.0 small, is the one its authors champion: in their paper they train variants with 8 and 30 times more parameters and none of them beats it on the GIFT-Eval benchmark (“we do not observe scaling benefits”). With the pretraining data available, more parameters do not buy more accuracy.

The timeline tells another story, the speed of the field: almost every family releases a second version in under two years, and the latest TimesFM, version 3.0 from August 2026, changes the license of the weights.

flowchart TB
  C[Chronos] --> B[Chronos-Bolt]
  B --> C2[Chronos-2]
  T1[TimesFM] --> T25[TimesFM-2.5]
  T25 --> T3[TimesFM 3.0]
  M1[Moirai] --> M2[Moirai-2.0]

Chronos: from token to patch

The original Chronos (Ansari et al., 2024) is the one lesson 5.8.1 took apart: mean scaling, 4,093 bins, a T5 that predicts the next token with cross-entropy, and sampled trajectories. It was pretrained on 28 public datasets (about 84 billion observations), 10 million series augmented with TSMixup (mixtures of series) and 1 million synthetic series from Gaussian processes (KernelSynth). Its weights are not in the course cache; the code, with the same library, would be this:

from chronos import ChronosPipeline

chronos = ChronosPipeline.from_pretrained(
    "amazon/chronos-t5-small", device_map="cpu", dtype=torch.float32)
# 20 sampled trajectories: (series, samples, horizon)
muestras = chronos.predict(torch.tensor(series["indice"][-512:]),
                           prediction_length=5, num_samples=20)
q10, q50, q90 = np.quantile(muestras[0].numpy(), [0.1, 0.5, 0.9],
                            axis=0)

Chronos-Bolt (November 2024) swaps the token for the patch and sampling for direct quantiles. According to its card, it was trained on “nearly 100 billion time series observations” and is up to 250 times faster and 20 times more memory efficient than the original Chronos of the same size; the repository README adds 5 % less error. It is the model of lesson 5.8.1. To compare, first the data: the same as in the previous lesson, on two assets, with one origin every ten sessions so that TimesFM, which is slower, fits in the lesson's time.

import numpy as np
import pandas as pd
import torch

torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
vol = pd.read_parquet("vol5_diario.parquet")
series = {}
for a in ("indice", "cripto"):
    d = vol[vol["activo"] == a].set_index("fecha")
    series[a] = np.log(d["rv_5min"].to_numpy())
fechas = pd.DatetimeIndex(d.index)
C = 512
origenes = np.flatnonzero(fechas.year >= 2018)[::10]
print(f"{len(origenes)} origins per asset, from "
      f"{fechas[origenes[0]].date()} to {fechas[origenes[-1]].date()}")

202 origins per asset, from 2018-01-02 to 2025-12-31

from chronos import BaseChronosPipeline

bolt = BaseChronosPipeline.from_pretrained(
    "amazon/chronos-bolt-small", device_map="cpu",
    dtype=torch.float32)
n_bolt = sum(p.numel() for p in bolt.model.parameters())
print(f"Chronos-Bolt-small: {n_bolt / 1e6:.1f} M parameters, "
      f"T5 with {bolt.model.config.num_layers} + "
      f"{bolt.model.config.num_decoder_layers} layers, "
      f"d_model {bolt.model.config.d_model}")

def prev_bolt(y):
    v = np.stack([y[i - C:i] for i in origenes])
    q, _ = bolt.predict_quantiles(
        inputs=torch.tensor(v, dtype=torch.float32),
        prediction_length=1)
    return q[:, 0, :].numpy().astype(float)     # quantiles 0.1…0.9

q_bolt = {a: prev_bolt(y) for a, y in series.items()}

Chronos-Bolt-small: 47.7 M parameters, T5 with 6 + 6 layers, d_model 512

47.7 million parameters counted in the weights, the card's “48M”: a small T5 with six layers in the encoder and six in the decoder.

Chronos-2: covariates and group attention

Chronos-2 (October 2025) is the family's conceptual leap. According to its paper, it is an encoder-only transformer with 120 million parameters that alternates two kinds of attention: time attention, between patches of the same series, and group attention, between related series and covariates of the same example. With that it does, without retraining, univariate and multivariate forecasting and forecasting with past or known-future covariates (numerical or categorical); it reaches a context of 8,192 steps and a horizon of 1,024, and returns 21 quantiles (from 0.01 to 0.99). Its authors attribute those capabilities to pretraining on synthetic data that impose multivariate structures on univariate series. On their own benchmark (fev-bench) it comes first, ahead of TiRex and TimesFM-2.5.

The installed library (chronos-forecasting 2.3.2) ships Chronos2Pipeline, but the amazon/chronos-2 weights are not in the environment's cache, so its code is left unexecuted. The shape of the inputs is the one in its documentation (one dictionary per series); lesson 5.8.3 uses it with covariates:

from chronos import Chronos2Pipeline

chronos2 = Chronos2Pipeline.from_pretrained("amazon/chronos-2",
                                            device_map="cpu")
entradas = [{"target": series[a][-2048:]} for a in series]
cuantiles, media = chronos2.predict_quantiles(
    entradas, prediction_length=1, quantile_levels=[0.1, 0.5, 0.9])
# a list: one tensor per series (variables, horizon, quantiles)

TimesFM-2.5: decoder-only

TimesFM (Das et al., 2024, Google) is a decoder-only transformer: it reads the context in patches of 32 points with causal attention, like a language model, and each output token predicts a block of 128 steps. It was pretrained on around 100 billion points from Google Trends, Wikipedia page views and synthetic series. Version 2.5 (September 2025) goes down from 500 to 200 million parameters, raises the context from 2,048 to 16,384 steps, adds an optional continuous quantile head of about 30 million parameters (up to 1,024 steps) and drops the frequency indicator the previous version required, according to the project README. Its card adds the GIFT-Eval pretraining set to the corpus. The weights in the cache let you check the figures:

import timesfm

tfm = timesfm.TimesFM_2p5_200M_torch.from_pretrained(
    "google/timesfm-2.5-200m-pytorch", torch_compile=False)
for nombre, m in tfm.model.named_children():
    n = sum(p.numel() for p in m.parameters())
    print(f"{nombre:<28} {n / 1e6:6.1f} M")
n_tfm = sum(p.numel() for p in tfm.model.parameters())
cfg = tfm.model.config
print(f"{'total':<28} {n_tfm / 1e6:6.1f} M | max. context "
      f"{cfg.context_limit}, patch {cfg.input_patch_len}, output "
      f"{cfg.output_patch_len}, quantiles up to "
      f"{cfg.output_quantile_len}")

tokenizer                       1.8 M
stacked_xf                    196.7 M
output_projection_point         4.9 M
output_projection_quantiles    27.9 M
total                         231.3 M | max. context 16384, patch 32, output 128, quantiles up to 1024

The transformer (stacked_xf) and its input and point-output projections add up to about 203 million, the “200M” in the name; the continuous quantile head adds 27.9 million, the head of about 30 million in the README. In total, 231.3 million parameters.

Before forecasting you have to compile the configuration: the maximum context, the maximum horizon and several options. Two of them matter here. torch_compile=False avoids torch.compile, which on CPU needs a C compiler (README of the ml environment). And force_flip_invariance, on by default, runs the model on the series and on the sign-flipped series and combines the two; it guarantees a property that the equivariance section checks, at the cost of doubling the compute. The lesson turns it off for the comparison (in an earlier test, with it on the MSE changed in the third decimal place):

def compilar(flip):
    tfm.compile(timesfm.ForecastConfig(
        max_context=C, max_horizon=128, normalize_inputs=True,
        use_continuous_quantile_head=True, fix_quantile_crossing=True,
        force_flip_invariance=flip, per_core_batch_size=64))

compilar(flip=False)

def prev_tfm(y):
    punto, q = tfm.forecast(horizon=1,
                            inputs=[y[i - C:i] for i in origenes])
    return punto[:, 0], q[:, 0, :]     # q: mean and deciles 0.1…0.9

r_tfm = {a: prev_tfm(y) for a, y in series.items()}
punto, q = r_tfm["indice"]
print("quantile columns:", q.shape[1],
      "| point = quantile 0.5?", np.allclose(punto, q[:, 5]))

quantile columns: 10 | point = quantile 0.5? True

The quantile output has ten columns, the mean and the deciles from 0.1 to 0.9, and the point forecast returned by forecast is the 0.5 quantile, the median. The two families therefore compare the same thing: medians. Another detail: per_core_batch_size is the batch size; by default it is 1 and TimesFM forecasts series by series.

Head to head on volatility

Same origins, same 512-session context, same target (tomorrow's log RV). Besides the squared error and the R², the table gives the mean pinball loss over the nine deciles (the scoring rule from lesson 5.7.3: it rewards the whole distribution, not just the median) and the coverage of the 10 to 90 % interval.

tau = np.arange(1, 10) / 10

def pinball(y, q):
    u = y[:, None] - q
    return np.maximum(tau * u, (tau - 1) * u).mean()

filas = {}
for a, y in series.items():
    real = y[origenes]
    cand = {"naive": (y[origenes - 1], None),
            "Chronos-Bolt": (q_bolt[a][:, 4], q_bolt[a]),
            "TimesFM-2.5": (r_tfm[a][0], r_tfm[a][1][:, 1:])}
    for m, (f, qs) in cand.items():
        fila = {"MSE": np.mean((real - f) ** 2),
                "R²": np.corrcoef(real, f)[0, 1] ** 2}
        if qs is not None:
            fila["pinball"] = pinball(real, qs)
            fila["cov. 10-90"] = np.mean((real >= qs[:, 0])
                                         & (real <= qs[:, 8]))
        filas[(a, m)] = fila
tabla = pd.DataFrame(filas).T
tabla.round(3)

                       MSE     R²  pinball  cov. 10-90
indice naive         0.241  0.511      NaN         NaN
       Chronos-Bolt  0.205  0.543    0.140       0.762
       TimesFM-2.5   0.210  0.532    0.141       0.782
cripto naive         0.621  0.303      NaN         NaN
       Chronos-Bolt  0.370  0.514    0.167       0.807
       TimesFM-2.5   0.378  0.505    0.169       0.787

Both models beat the naive forecast and end up very close: on the index, MSE of 0.205 (Bolt) and 0.210 (TimesFM) versus 0.241; on crypto, 0.370 and 0.378 versus 0.621. The pinball loss is also almost the same (0.140 versus 0.141 on the index) and the coverage of the 10-90 % interval ranges from 76 to 81 %, around the nominal 80 %. The index figures do not match those of lesson 5.8.1 because here one origin in ten is used (202 days instead of 2,011). With so few days, the differences have to be measured:

def pareada(y, f1, f2):
    real = y[origenes]
    d = (real - f1) ** 2 - (real - f2) ** 2
    t = d.mean() / d.std(ddof=1) * np.sqrt(len(d))
    return f"{d.mean():+.4f} (t = {t:+.1f})"

for a, y in series.items():
    fb, ft = q_bolt[a][:, 4], r_tfm[a][0]
    print(f"{a:<7} Bolt − naive {pareada(y, fb, y[origenes - 1])}  "
          f"TimesFM − naive {pareada(y, ft, y[origenes - 1])}  "
          f"TimesFM − Bolt {pareada(y, ft, fb)}")

indice  Bolt − naive -0.0366 (t = -2.0)  TimesFM − naive -0.0316 (t = -1.7)  TimesFM − Bolt +0.0050 (t = +1.0)
cripto  Bolt − naive -0.2503 (t = -2.6)  TimesFM − naive -0.2423 (t = -2.6)  TimesFM − Bolt +0.0080 (t = +1.2)

Against the naive forecast, t between −1.7 and −2.6 with 202 days (in lesson 5.8.1, with every day of the index, Bolt gave t = −9.8: the sample rules). Between the two models, t = 1.0 and 1.2: they cannot be told apart. A model with 47.7 million parameters and another with 231.3 million, from two companies, with different data and architectures, give almost the same forecast here. That is to be expected on a series with a simple structure (a HAR in logs): any competent model extracts almost all the information in the context, and the rest is noise nobody can forecast.

Equivariance: what the architecture guarantees

A useful property of a forecasting model: if you multiply the series by $a$ and add $b$, the forecast should transform the same way, $f(ax + b) = a\,f(x) + b$. If it does not, the result depends on the units (percent or fraction, dollars or euros). Both models standardize the context before processing it, so they satisfy it by construction for $a > 0$. For $a < 0$ (flipping the series) there is no guarantee: the standardized series changes shape.

x = series["indice"][-C:]

def una_bolt(v):
    q, _ = bolt.predict_quantiles(
        inputs=torch.tensor(v, dtype=torch.float32),
        prediction_length=1)
    return q[0, 0, 4].item()

def una_tfm(v):
    return tfm.forecast(horizon=1, inputs=[v])[0][0, 0]

for nombre, f in (("Bolt", una_bolt), ("TimesFM, no flip", una_tfm)):
    base = f(x)
    e1 = abs(f(2 * x + 3) - (2 * base + 3))
    e2 = abs(f(-x) - (-base))
    print(f"{nombre:<18} |f(2x+3) − (2f(x)+3)| = {e1:.4f}   "
          f"|f(−x) − (−f(x))| = {e2:.4f}")
compilar(flip=True)
base = una_tfm(x)
print(f"{'TimesFM, with flip':<18} |f(−x) − (−f(x))| = "
      f"{abs(una_tfm(-x) + base):.4f}")

Bolt               |f(2x+3) − (2f(x)+3)| = 0.0000   |f(−x) − (−f(x))| = 0.0931
TimesFM, no flip   |f(2x+3) − (2f(x)+3)| = 0.0000   |f(−x) − (−f(x))| = 0.0982
TimesFM, with flip |f(−x) − (−f(x))| = 0.0000

Scaling by 2 and adding 3 changes nothing in either model. Flipping the series does: about 0.09 units of log RV in both, of the order of 10 % of the variance. Volatility has upward spikes and no downward ones; the inverted series has troughs in their place, and the model has no reason to treat it as the mirror image of the original. With force_flip_invariance=True, TimesFM averages f(x) and −f(−x), and the property holds exactly. For a signed series (returns, rate differences) that may be desirable; for volatility, which only spikes upward, it is not clear that it is, and it costs twice as much.

Moirai-2.0, TiRex, TimeGPT and Kronos

The other four families in the syllabus are not run in the course, each for its own reason (README of the ml environment). Their code is the one from their official documentation.

Moirai (Woo et al., 2024, Salesforce) was introduced as a masked encoder transformer with several patch sizes, “any-variate” attention (any number of variables in the same sequence) and a mixture-of-distributions output (Student's t, log-normal, negative binomial, low-variance normal), pretrained on LOTSA, more than 27 billion observations from nine domains. Moirai-2.0 (November 2025) simplifies everything: decoder-only, a single patch size, quantile loss and several output tokens per step, with a new corpus of 36 million series (about 295 billion observations). Its authors describe it as twice as fast and 30 times smaller than Moirai 1.0-Large, and better. Its library, uni2ts, requires numpy~=1.26 and torch<2.5, which do not exist for Python 3.14, and its weights are CC-BY-NC-4.0, for research only. The code from the README:

from uni2ts.model.moirai2 import Moirai2Forecast, Moirai2Module

modelo = Moirai2Forecast(
    module=Moirai2Module.from_pretrained(
        "Salesforce/moirai-2.0-R-small"),
    prediction_length=100,
    context_length=1680,
    target_dim=1,
    feat_dynamic_real_dim=0,
    past_feat_dynamic_real_dim=0,
)
predictor = modelo.create_predictor(batch_size=32)
# datos_gluonts: entries of a GluonTS PandasDataset
previsiones = predictor.predict(datos_gluonts)

TiRex (Auer et al., 2025, NX-AI; NeurIPS 2025) is the architectural exception: it is not a transformer but an xLSTM (specifically, with sLSTM blocks), a modern recurrent network that, according to its authors, keeps the ability to track the state of the series, useful at long horizons. It has 35 million parameters, was trained on 47.5 million series (from Chronos, augmented with TSMixup, synthetic and part of the GIFT-Eval pretraining set) and in its paper it beats larger models on GIFT-Eval and on the Chronos benchmark. Its license is the NXAI community license: a company with more than 100 million euros in annual revenue that builds it into a commercial product or service needs a commercial license. It is installed with pip install tirex-ts; it is not in the course environment. The example from its README:

from tirex import load_model, ForecastModel

modelo: ForecastModel = load_model("NX-AI/TiRex")
contexto = torch.tensor(np.stack([series[a][-512:] for a in series]))
cuantiles, media = modelo.forecast(context=contexto,
                                   prediction_length=1)

TimeGPT-1 (Garza, Challu and Mergenthaler-Canseco, 2023, Nixtla) was one of the first: an encoder-decoder transformer pretrained on “over 100 billion data points”. It is closed: there are no weights, it is used through a paid API with the nixtla SDK (Apache-2.0), and its intervals come from conformal prediction on the historical errors, not from the model. For a fund, the cost is not just the fee: the series travels to someone else's server. The signature is the one in the SDK reference (the key comes from an environment variable, never from the code):

import os
from nixtla import NixtlaClient

cliente = NixtlaClient(api_key=os.environ["NIXTLA_API_KEY"])
df = pd.DataFrame({"unique_id": "indice", "ds": fechas,
                   "y": series["indice"]})
prevision = cliente.forecast(df=df, h=5, freq="B", level=[80])

Kronos (Shi et al., 2025; accepted at AAAI 2026) is the only one on the list that is specific to markets. It is pretrained on more than 12 billion candles (OHLCV records) from 45 markets and seven frequencies, and its tokenizer turns each multivariate candle into two hierarchical subtokens (one coarse and one fine) with binary spherical quantization; a decoder-only transformer predicts the next candles by sampling. There are open versions of 4.1 (mini), 24.7 (small) and 102.3 million parameters (base), with a context of 512 candles in small and base, and an unreleased large version of 499.2 million; MIT license. It is not a PyPI package (the kronos package on PyPI is something else): you clone the repository and import from its model folder. The example from its README, with the columns it expects:

from model import Kronos, KronosTokenizer, KronosPredictor

tokenizer = KronosTokenizer.from_pretrained(
    "NeoQuasar/Kronos-Tokenizer-base")
modelo = Kronos.from_pretrained("NeoQuasar/Kronos-small")
predictor = KronosPredictor(modelo, tokenizer, max_context=512)

columnas = ["open", "high", "low", "close", "volume", "amount"]
x_df = velas.loc[:399, columnas]
pred_df = predictor.predict(
    df=x_df,
    x_timestamp=velas.loc[:399, "timestamps"],
    y_timestamp=velas.loc[400:519, "timestamps"],
    pred_len=120,
    T=1.0,          # sampling temperature
    top_p=0.9,      # nucleus sampling
    sample_count=1, # trajectories that are averaged
)

Without the device argument, the repository code picks CUDA, then MPS (Apple) and otherwise CPU.

What to look at when choosing

Exercises

1. With the forecasts already computed, measure the quantile-by-quantile calibration of the two models on crypto: the fraction of days on which the realized value falls below each forecast quantile. Which one is closer to the diagonal?

real = series["cripto"][origenes]
frec = pd.DataFrame({
    "nominal": tau,
    "Chronos-Bolt": (real[:, None] <= q_bolt["cripto"]).mean(0),
    "TimesFM-2.5": (real[:, None] <= r_tfm["cripto"][1][:, 1:]).mean(0),
})
print(frec.round(3).to_string(index=False))
print("mean absolute deviation:",
      (frec.iloc[:, 1:].sub(frec["nominal"], axis=0)).abs().mean()
      .round(3).to_dict())

 nominal  Chronos-Bolt  TimesFM-2.5
     0.1         0.104        0.094
     0.2         0.168        0.173
     0.3         0.277        0.287
     0.4         0.386        0.391
     0.5         0.475        0.505
     0.6         0.559        0.569
     0.7         0.644        0.644
     0.8         0.767        0.752
     0.9         0.911        0.881
mean absolute deviation: {'Chronos-Bolt': 0.026, 'TimesFM-2.5': 0.024}

2. TimesFM-2.5 accepts contexts of up to 16,384 steps. Repeat its index forecast with a context of 128 sessions (recompile with max_context=128). How much does the MSE worsen?

tfm.compile(timesfm.ForecastConfig(
    max_context=128, max_horizon=128, normalize_inputs=True,
    use_continuous_quantile_head=True, fix_quantile_crossing=True,
    force_flip_invariance=False, per_core_batch_size=64))
y = series["indice"]
p128, _ = tfm.forecast(horizon=1,
                       inputs=[y[i - 128:i] for i in origenes])
real = y[origenes]
print(f"MSE with 512: {np.mean((real - r_tfm['indice'][0]) ** 2):.3f}"
      f"   with 128: {np.mean((real - p128[:, 0]) ** 2):.3f}")
print("paired difference:",
      pareada(y, p128[:, 0], r_tfm["indice"][0]))

MSE with 512: 0.210   with 128: 0.213
paired difference: +0.0034 (t = +0.5)

3. A fund with 500 million euros in revenue wants to build a foundation model into a forecasting product it will sell to clients, without sending data outside. Which models from the lesson would you rule out on license or deployment grounds, and which would remain?