--- title: "5.8.1 What are time series foundation models" description: "Massive pretraining, tokens and patches, and a first zero-shot Chronos-Bolt forecast of realized volatility, compared with naive baselines." url: "https://pythonparatrading.com/en/python-trading-course/level-5/what-are-time-series-foundation-models/" lang: "en" image: "https://pythonparatrading.com/og/en/python-trading-course/level-5/what-are-time-series-foundation-models.png" --- # 5.8.1 · Time series foundation models: pretrain once, forecast without training > A time series foundation model is trained only once on tens of billions of observations from very different domains and is then used, without retraining, on a series it has never seen. This lesson explains what gets pretrained, how a series is turned into tokens or patches and what “zero-shot” means, and makes the first forecast with Chronos-Bolt on the level 5 realized volatility, with two naive baselines alongside. - Course: [Level 5: Machine learning and deep learning (Keras/PyTorch)](https://pythonparatrading.com/en/python-trading-course/level-5/) - Module: 5.8 · Time series foundation models - Duration: 60 min - Prerequisites: [5.7.2 · Time series architectures: TCN, N-BEATS, PatchTST and iTransformer](https://pythonparatrading.com/en/python-trading-course/level-5/tcn-n-beats-patchtst-itransformer/), [5.7.3 · Probabilistic models: quantile loss, DeepAR and intervals](https://pythonparatrading.com/en/python-trading-course/level-5/probabilistic-forecasting-quantile-loss-deepar/), [2.1.4 · Volatility models: ARCH, GARCH, EGARCH and GJR with the arch library](https://pythonparatrading.com/en/python-trading-course/level-2/volatility-models-arch-garch-egarch-gjr/) ## By the end of this lesson - Explain what a time series foundation model is: pretraining on a massive, heterogeneous corpus, and zero-shot use on new series. - Reproduce by hand the Chronos tokenization (mean scaling and uniform quantization) and the Chronos-Bolt patches, and check them against the library. - Make a zero-shot forecast with Chronos-Bolt-small without network access or retraining, and read its quantiles. - Evaluate it with a rolling origin against naive baselines, with paired differences and by year, and place it against the HAR and the oracle of the README. ## From one model per series to one model for all Up to here, every model in the course has been estimated with the data of its problem: a GARCH with the returns of one asset (lesson 2.1.4), a HAR or a network with the volatility of six assets (lesson 5.7.2). A **time series foundation model** reverses the order. Someone trains it once, on a huge corpus of series from very different domains (energy, traffic, sales, weather, web visits, finance, synthetic series), publishes the weights, and anyone can then use them on a series the model has never seen, **without retraining**. That is called **zero-shot** forecasting: the new series only enters as context, in the forward pass, and the weights do not change. ```mermaid flowchart TB A[Massive corpus] --> B[Pretraining] B --> C[Published weights] C --> D[Context] D --> E[Zero-shot] E --> F[Fine-tuning] ``` The idea comes from language models: a transformer that has seen enough texts completes a new one without anyone training it for that text. In time series, the bet is that there are regularities common to almost all of them (trends, seasonalities, persistence, level shifts, structured noise) and that a large model can learn them and recognize them in the context it receives. The corpus figures say it all: | Model | Pretraining corpus (according to its authors) | |---|---| | Chronos (Amazon, 2024) | 28 public datasets, about 890,000 series and about 84 billion observations, plus 10 million series augmented with TSMixup and 1 million synthetic Gaussian process series (KernelSynth) | | Chronos-Bolt (Amazon, 2024) | “nearly 100 billion time series observations” | | TimesFM (Google, 2024) | on the order of 100 billion points: Google Trends, Wikipedia page views and synthetic series | Lesson 5.8.2 covers the families with their architectures, sizes and licenses. This lesson sticks to what they have in common: how they turn a series into something a transformer can read and what you get when you use them without training. ## The series: the index's realized variance The test bench is the one from lessons 5.7.2 and 5.7.3: the `indice` asset of `vol5_diario.parquet`, 4,529 sessions from 2008 to 2025, with its realized variance for each day (`rv_5min`, in squared return). Its logarithm is forecast, which is almost Gaussian and is the scale on which the generator works. ```python import numpy as np import pandas as pd vol = pd.read_parquet("vol5_diario.parquet") ind = vol[vol["activo"] == "indice"].set_index("fecha") log_rv = np.log(ind["rv_5min"].to_numpy()) vol_anual = np.sqrt(252 * ind["rv_5min"].to_numpy()) fechas = pd.DatetimeIndex(ind.index) print(f"{len(log_rv)} sessions, {fechas[0].date()} to " f"{fechas[-1].date()}") print(f"log RV: mean {log_rv.mean():.2f}, std. {log_rv.std():.2f}") print(f"annual vol.: median {np.median(vol_anual):.1%}, " f"p99 {np.quantile(vol_anual, 0.99):.1%}") ``` ```output 4529 sessions, 2008-01-02 to 2025-12-31 log RV: mean -9.75, std. 0.64 annual vol.: median 12.1%, p99 25.6% ``` The log RV sits around −9.75 with a standard deviation of 0.64: a series far from zero and with little relative variation. That detail matters right away. ## From numbers to tokens: the Chronos recipe A language transformer works with a finite vocabulary. Chronos (Ansari et al., 2024) does the minimum to make a series look like one: it **scales** the context by dividing it by the mean of its absolute values and **quantizes** the result into uniform bins. With a context $x_1, \dots, x_C$: $$ s = \frac{1}{C}\sum_{i=1}^{C} |x_i|, \qquad \tilde x_i = \frac{x_i}{s}, \qquad \text{token}_i = \text{bin}(\tilde x_i) $$ According to the paper and the models' `config.json`, the bins are 4,093 equally spaced centers between −15 and +15 (a vocabulary of 4,096 with the special padding and end tokens), the context is 512 observations and the horizon, 64. The model (a T5 with 20 to 710 million parameters) is trained with cross-entropy to predict the next token, like a language model, and the probabilistic forecast comes from **sampling** token trajectories and undoing the scaling. Each bin measures $30/4{,}092 \approx 0.0073$ in scaled units. The `chronos-forecasting` library includes the tokenizer, which needs no weights: the configuration of `amazon/chronos-t5-small` is enough. The cell reproduces it by hand and applies it to two versions of the same information, the annual volatility and the logarithm of the variance: ```python import torch from chronos import ChronosConfig # The configuration of amazon/chronos-t5-small (the repo's config.json) cfg = ChronosConfig( tokenizer_class="MeanScaleUniformBins", tokenizer_kwargs={"low_limit": -15.0, "high_limit": 15.0}, n_tokens=4096, n_special_tokens=2, pad_token_id=0, eos_token_id=1, use_eos_token=True, model_type="seq2seq", context_length=512, prediction_length=64, num_samples=20, temperature=1.0, top_k=50, top_p=1.0) tok = cfg.create_tokenizer() centros = np.linspace(-15, 15, 4096 - 2 - 1) def tokens_a_mano(x): """Scale by the mean of |x| and round to the nearest center. Ids 0 and 1 are PAD and EOS; the bins start at 3.""" escala = np.abs(x).mean() ids = np.abs(x[:, None] / escala - centros).argmin(1) + 3 return ids, centros[ids - 3] * escala for nombre, x in (("annual vol", vol_anual[-512:]), ("log RV", log_rv[-512:])): ids, x_rec = tokens_a_mano(x) ids_chr, _, escala = tok.context_input_transform( torch.tensor(x[None, :], dtype=torch.float32)) igual = (ids == ids_chr[0, :512].numpy()).mean() print(f"{nombre:<10} scale {escala.item():8.4f} " f"distinct tokens {len(np.unique(ids)):4d} " f"max. error {np.abs(x - x_rec).max():.4f} " f"same as chronos {igual:.0%}") ``` ```output annual vol scale 0.1405 distinct tokens 168 max. error 0.0005 same as chronos 100% log RV scale 9.5590 distinct tokens 46 max. error 0.0350 same as chronos 100% ``` The library's tokenizer and the cell's agree. The comparison between the two series shows the weakness of mean scaling. The annual volatility (positive, with a mean of 0.14) uses 168 distinct bins and the rounding costs at most 0.0005, five hundredths of a volatility point. The logarithm of the variance, on the other hand, is always around −9.6: divided by its mean absolute value it is almost always close to −1, it only uses **46 bins** and the rounding reaches 0.035 in log, 3.6% of the variance. Mean scaling does not subtract the mean, so a series far from zero loses resolution. Exercise 2 measures it with the standardized series. ## Patches: what Chronos-Bolt and TimesFM do The second generation drops the one-token-per-value approach. **Chronos-Bolt** standardizes the context (subtracts the mean and divides by the standard deviation, what the code calls `InstanceNorm`), cuts it into **patches** of 16 observations and projects each patch to a vector with a small network; at the end it adds a regression token. The T5 encoder reads those vectors and the decoder produces at once the **0.1…0.9 quantiles of the next 64 steps** (direct multi-step forecasting), without sampling. **TimesFM** does something similar with patches of 32 and a decoder-only transformer (lesson 5.8.2). It is the same idea as PatchTST in lesson 5.7.2, at pretraining scale. ```mermaid flowchart LR S[Context] --> M[Mean scaling] M --> Q[One token per value] S --> N[Standardize] N --> P[Patches of 16] Q --> T[Transformer] P --> T ``` The consequence is about cost. Attention compares each token with all the others, so its cost grows with the square of the number of tokens: *Fig. — How many tokens the transformer processes for a 512-observation context: one per point in Chronos (plus the end token), a 16-point patch plus the regression token in Chronos-Bolt and a 32-point patch in TimesFM-2.5.* Three bars on a log scale. Chronos: 513 tokens. Chronos-Bolt, highlighted: 33. TimesFM-2.5: 16. Since attention compares every token with all others, its cost grows with the square: about 260,000 pairs in Chronos against about 1,100 in Bolt. The Chronos-Bolt-small weights are in the `ml` environment's cache, pinned by commit (environment README). The cell loads them without network access and checks the hand-made patches against the model's: ```python from chronos import BaseChronosPipeline torch.manual_seed(0) torch.use_deterministic_algorithms(True) bolt = BaseChronosPipeline.from_pretrained( "amazon/chronos-bolt-small", device_map="cpu", dtype=torch.float32) cc = bolt.model.chronos_config print(f"max. context {cc.context_length}, " f"patch {cc.input_patch_size}, " f"horizon {cc.prediction_length}, quantiles {cc.quantiles}") x = torch.tensor(log_rv[-512:], dtype=torch.float32)[None, :] z = (x - x.mean()) / x.std(correction=0) # standardize parches = z.unfold(-1, 16, 16) # (1, 32, 16) z_bolt, _ = bolt.model.instance_norm(x) print("patches:", tuple(parches.shape), "| equal to the model's:", torch.allclose(parches, bolt.model.patch(z_bolt), atol=1e-5)) print(f"standardized values: from {z.min():.2f} to {z.max():.2f}") ``` ```output max. context 2048, patch 16, horizon 64, quantiles [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] patches: (1, 32, 16) | equal to the model's: True standardized values: from -2.45 to 2.57 ``` Maximum context of 2,048 observations, patches of 16, a horizon of 64 steps and nine quantiles. Standardized, the log RV spans −2.45 to 2.57: the resolution problem of mean scaling disappears, because here there is no rounding (each patch enters with its real values) and the mean is subtracted first. ## A first forecast With the model loaded, forecasting is one call: it is given the context (the last 512 sessions) and the horizon. The chosen origin is Friday, March 6, 2020, the day before one of the shocks planted in the data (the README documents +1.8 in log-variance on March 9). The output is converted to annual volatility to make it easier to read: ```python i0 = fechas.get_loc(pd.Timestamp("2020-03-06")) + 1 q, _ = bolt.predict_quantiles( inputs=torch.tensor(log_rv[i0 - 512:i0], dtype=torch.float32), prediction_length=5, quantile_levels=[0.1, 0.5, 0.9]) a_vol = lambda l: np.sqrt(252 * np.exp(l)) # log RV to annual vol. tabla = pd.DataFrame( a_vol(q[0].numpy().astype(float)) * 100, index=fechas[i0:i0 + 5].date, columns=["q10", "q50", "q90"]) tabla["realized"] = vol_anual[i0:i0 + 5] * 100 tabla.round(1) ``` ```output q10 q50 q90 realized 2020-03-09 12.0 15.5 20.2 52.8 2020-03-10 12.0 16.0 21.4 28.6 2020-03-11 12.2 16.5 22.1 31.0 2020-03-12 12.4 16.9 22.6 24.9 2020-03-13 12.5 17.1 22.9 23.3 ``` The model forecasts a median volatility of 15.5% for Monday, with a 10 to 90% band from 12.0 to 20.2%, and a slight rise over the following days. Monday's realized volatility is 52.8%: the shock was not in the past of the series and nobody could have forecast it from it. From Tuesday on the realized value (28.6%, 31.0%…) stays above the band, because this forecast is five days ahead from Friday and is not updated. The figure at the top shows what happens when the forecast is redone each day with the new data point: the band jumps after the shock and chases the realized value. ## One-day zero-shot with a rolling origin A single forecast evaluates nothing. The honest evaluation is that of lesson 5.4.3 without the retraining: for each session from 2018 to 2025, the next day is forecast with the previous 512 sessions and compared with what happened. The model is never fitted, so there is no possible leak through training; the only rule is that the context ends on the day before. The baselines are the **naive** forecast (yesterday's log RV) and the **mean of the last 22 days**. ```python prueba = np.flatnonzero(fechas.year >= 2018) C = 512 def zero_shot(serie, origenes, contexto=C, lote=256): """Quantiles 0.1…0.9 for tomorrow at each origin (no retraining).""" salida = [] for k in range(0, len(origenes), lote): ventanas = np.stack([serie[i - contexto:i] for i in origenes[k:k + lote]]) q, _ = bolt.predict_quantiles( inputs=torch.tensor(ventanas, dtype=torch.float32), prediction_length=1) salida.append(q[:, 0, :].numpy().astype(float)) return np.concatenate(salida) q_bolt = zero_shot(log_rv, prueba) real = log_rv[prueba] previsiones = { "naive (yesterday)": log_rv[prueba - 1], "22-day mean": np.array([log_rv[i - 22:i].mean() for i in prueba]), "Chronos-Bolt (median)": q_bolt[:, 4], } def metricas(f): e2 = (real - f) ** 2 return {"MSE": e2.mean(), "R²": np.corrcoef(real, f)[0, 1] ** 2} tabla = pd.DataFrame({k: metricas(f) for k, f in previsiones.items()}) tabla.T.round(3) ``` ```output MSE R² naive (yesterday) 0.233 0.503 22-day mean 0.237 0.418 Chronos-Bolt (median) 0.181 0.551 ``` Chronos-Bolt, without ever having seen the series, gives a mean squared error (MSE) of 0.181 against 0.233 for the naive forecast and 0.237 for the 22-day mean, and an R² of 0.551 (the R² is that of the regression of the log RV on the forecast, the same measure as in the README). To put it in context: the README publishes, for the same period, an R² of **0.57 for a HAR** re-estimated every January and of **0.61 for the oracle**, the exact expectation of the variance with the information of the day before. The foundation model sits between the naive baseline and the HAR, closer to the HAR. Whether that is enough is decided by the lab of lesson 5.8.5, with GARCH and LightGBM in the same test. The forecast being compared is the **median** (the 0.5 quantile), not the mean. In log it makes no difference because the distribution is almost symmetric; in variance it would (the exponential of the median of the log understates the mean of the variance), and lesson 5.8.3 deals with it. ## Is the improvement larger than the noise? The error differences are read with the day-by-day paired difference, as in lesson 5.4.3: the hard days are hard for every model, and the difference removes that common noise. ```python def pareada(a, b): """Mean difference of squared errors (a − b) and its t.""" d = (real - previsiones[a]) ** 2 - (real - previsiones[b]) ** 2 return d.mean(), d.mean() / (d.std(ddof=1) / np.sqrt(len(d))) for ref in ("naive (yesterday)", "22-day mean"): dif, t = pareada("Chronos-Bolt (median)", ref) print(f"Bolt − {ref:<17} {dif:+.4f} t = {t:+.1f}") dentro = (real >= q_bolt[:, 0]) & (real <= q_bolt[:, 8]) print(f"coverage of the 10-90% interval: {dentro.mean():.1%} " f"({len(real)} days)") ``` ```output Bolt − naive (yesterday) -0.0516 t = -9.8 Bolt − 22-day mean -0.0557 t = -10.4 coverage of the 10-90% interval: 76.4% (2011 days) ``` Both improvements are far above the noise: t = −9.8 against the naive forecast and −10.4 against the 22-day mean. The test days overlap in the context but not in the error being measured (each day is forecast once), so the standard error of the difference is reasonable; if the errors had autocorrelation it would have to be corrected (Newey-West), but with a t of −10 a very strong autocorrelation would be needed to change the conclusion. The 10 to 90% interval covers **76.4%** of the days, not 80%: the quantiles are somewhat narrow. Lesson 5.8.3 measures the calibration quantile by quantile and corrects it. The result holds year by year: ```python anio = fechas.year[prueba] por_anio = pd.DataFrame({k: (real - f) ** 2 for k, f in previsiones.items()}).groupby(anio).mean() por_anio.round(3) ``` ```output naive (yesterday) 22-day mean Chronos-Bolt (median) fecha 2018 0.202 0.163 0.147 2019 0.229 0.194 0.166 2020 0.288 0.285 0.224 2021 0.214 0.301 0.181 2022 0.247 0.261 0.197 2023 0.201 0.216 0.159 2024 0.233 0.224 0.182 2025 0.249 0.251 0.194 ``` *Fig. — Mean squared error of the one-day forecast of the synthetic index's log realized variance, by year: naive, 22-day mean and zero-shot Chronos-Bolt.* Three lines from 2018 to 2025. Chronos-Bolt (highlighted) has the lowest error every year, between 0.147 and 0.224 (the peak, in 2020). The naive forecast ranges from 0.201 to 0.288 and the 22-day mean from 0.163 to 0.301, with its worst year in 2021, after the 2020 shock. Chronos-Bolt has the lowest error in all eight years. The worst is 2020 for all of them (the year of the March shocks). The 22-day mean suffers more in 2021: for weeks it drags along the days of extreme volatility of the previous year, which the foundation model discounts sooner. ## What zero-shot does not do A good zero-shot volatility forecast says nothing about other things, and it is worth being clear about that before going on: | The model does | The model does not | |---|---| | Extract persistence, level and dispersion from the recent history | Know the asset, the event calendar or the news | | Give quantiles without training anything | Guarantee that those quantiles are calibrated (here, 76.4% against 80%) | | Work on series with structure (volatility has plenty) | Predict what has no structure: the returns of `vol5` are unpredictable by construction | | Save the work of estimating a model per series | Beat the generating process: the oracle is the floor | Lesson 5.8.2 compares the families (Chronos, Chronos-Bolt, Chronos-2, TimesFM-2.5, Moirai-2.0, TiRex, TimeGPT and Kronos) and puts TimesFM-2.5 next to Chronos-Bolt; 5.8.3 goes into the practical decisions (context length, quantiles, covariates and fine-tuning). ## Exercises 1. Repeat the zero-shot evaluation with the `cripto` asset (use one origin out of every four to make it fast). Does Chronos-Bolt beat both baselines? Compare its R² with that of the README's HAR for that asset (0.40). **Solution.** ```python vc = vol[vol["activo"] == "cripto"].set_index("fecha") log_rv_c = np.log(vc["rv_5min"].to_numpy()) orig = prueba[::4] q_c = zero_shot(log_rv_c, orig) real_c = log_rv_c[orig] for nombre, f in (("naive", log_rv_c[orig - 1]), ("mean 22", np.array([log_rv_c[i - 22:i].mean() for i in orig])), ("Bolt", q_c[:, 4])): print(f"{nombre:<9} MSE {np.mean((real_c - f) ** 2):.3f} " f"R² {np.corrcoef(real_c, f)[0, 1] ** 2:.3f}") ``` ```output naive MSE 0.652 R² 0.252 mean 22 MSE 0.427 R² 0.328 Bolt MSE 0.395 R² 0.393 ``` In crypto the naive forecast is much worse (MSE 0.652): the daily RV of an asset with frequent jumps is very noisy and yesterday's value is a poor forecast. Chronos-Bolt drops to 0.395, better than the 22-day mean (0.427), and its R² of 0.393 is almost that of the README's HAR (0.40), with the oracle at 0.46. With 503 origins the R² has an error of a few hundredths: the comparison with the HAR reads as “similar”, not as a ranking. 2. Standardize the log RV context (subtract the mean and divide by the standard deviation) before tokenizing it with the Chronos recipe. How many bins does it use now? What does it tell you about the mean scaling of the original Chronos? **Solution.** ```python x = log_rv[-512:] z = (x - x.mean()) / x.std() for nombre, serie in (("raw log RV", x), ("z-scored log RV", z)): ids, rec = tokens_a_mano(serie) print(f"{nombre:<18} distinct tokens {len(np.unique(ids)):4d}") ``` ```output raw log RV distinct tokens 46 z-scored log RV distinct tokens 353 ``` Standardized, the same information uses 353 bins instead of 46: almost eight times more resolution. The mean scaling of the original Chronos is designed for positive series close to their typical level (sales, demand, visits), where dividing by the mean is natural. For series far from zero or with a sign, the second generation (Chronos-Bolt, Chronos-2) switched to standardizing, and on top of that it does not quantize. If you use a model that quantizes, the prior transformation of the series is part of the model. 3. A paper evaluates in 2025 a foundation model on daily returns of S&P 500 stocks from 2015 to 2023, and the model was pretrained on a corpus published in 2024 that includes public financial series. What problem does the evaluation have? Why do the course data not have it? **Solution.** The risk is **pretraining contamination**: if the test series (or series highly correlated with them, such as other stocks of the same index on the same dates) were in the corpus, the model may have memorized part of what it later “forecasts”, and the evaluation is no longer out of sample. It is the same leak as in lesson 5.2.2, but hidden in someone else's weights. Serious authors control for it: the Chronos-2 model card says it excludes the test portions of the datasets that overlap with the GIFT-Eval benchmark, and the Moirai-2.0 one talks about “non-leaking” subsets. For your own evaluation, the safe option is to test with data after the corpus cutoff date or with data that cannot be in it, like the synthetic data of this course, generated after the models were published. ## Self-assessment 1. What does it mean that a Chronos-Bolt forecast on `vol5` is zero-shot? - A. That the model has no parameters - B. That the forecast has no error on the first day - C. That the model has not been trained or tuned on that series: the weights are those of the pretraining and the series only enters as context - D. That the model is retrained from scratch for each series *Answer C: That the model has not been trained or tuned on that series: the weights are those of the pretraining and the series only enters as context.* The model learns once, with its corpus, and then receives the recent history of the series as context. Whatever it “learns” from the new series it does in the forward pass, without touching any weight. 2. Chronos tokenizes the logarithm of the index's realized variance (values around −9.8 with a standard deviation of 0.6) with mean scaling and 4,093 uniform bins in [−15, 15]. What happens? - A. The scaled series is almost always around −1, so it only uses about 46 bins and the rounding reaches 0.035 in log (3.6% of the variance) - B. Nothing: mean scaling is invariant to any transformation - C. It goes out of the [−15, 15] range and the model fails - D. It uses all the bins equally *Answer A: The scaled series is almost always around −1, so it only uses about 46 bins and the rounding reaches 0.035 in log (3.6% of the variance).* Mean scaling divides by the mean of |x| but does not subtract the mean. In a series far from zero and with little relative variation, all the scaled values fall within a narrow band. Standardized (subtracting the mean), the same series uses 353 bins. 3. Why is Chronos-Bolt much faster than the original Chronos of the same size? - A. Because it has fewer layers - B. Because it does not use attention - C. Because it only predicts the mean - D. Because it groups 16 observations into each token (512 data points are 33 tokens instead of 513) and predicts all the quantiles of the horizon at once, instead of sampling token by token *Answer D: Because it groups 16 observations into each token (512 data points are 33 tokens instead of 513) and predicts all the quantiles of the horizon at once, instead of sampling token by token.* Fewer tokens make attention cheaper, since its cost grows with the square of their number, and the direct quantile output avoids generating many autoregressive trajectories. Amazon puts the gain at up to 250 times in speed. 4. Zero-shot Chronos-Bolt gives an MSE of 0.181 in log RV against 0.233 for the naive forecast, with t = −9.8 in the paired difference. What can be concluded? - A. That Chronos-Bolt is the best possible volatility model - B. That it clearly beats the naive forecast on these data, but stays below the README's HAR (R² 0.551 against 0.57) and far from the oracle (0.61): it still has to be compared with models estimated on the series (lesson 5.8.5) - C. That it will also predict returns well - D. Nothing: 2,011 days are too few *Answer B: That it clearly beats the naive forecast on these data, but stays below the README's HAR (R² 0.551 against 0.57) and far from the oracle (0.61): it still has to be compared with models estimated on the series (lesson 5.8.5).* The naive baseline is easy to beat. The serious bar is a HAR, a GARCH or a LightGBM estimated on the series itself, and the floor is the oracle. And volatility has structure; the returns of `vol5`, by construction, do not. 5. On March 9, 2020 the realized volatility of the synthetic index jumps to 52.8% and the 10-90% band forecast the day before goes from 12.0 to 20.2%. What does it indicate? - A. That the model is broken - B. That the model needs more training - C. That the shock is an innovation with no warning in the data: no model that only sees the past of the series could forecast it, and the band reacts afterwards - D. That the context was too short *Answer C: That the shock is an innovation with no warning in the data: no model that only sees the past of the series could forecast it, and the band reacts afterwards.* The level README documents it as a +1.8 shock in log-variance on that day. A time series model, foundation or not, can only anticipate what the past contains; jumps are handled with risk management, not with forecasting. ## Key points - A foundation model is pretrained once on a huge, varied corpus (Chronos: about 84 billion observations from 28 datasets, plus augmented and synthetic data) and used zero-shot: the new series enters as context, without touching the weights. - Chronos turns each data point into a token: it divides it by the mean of |x| and rounds it to one of 4,093 bins in [−15, 15]. In a series far from zero, like the log of the variance, that leaves few useful bins. - Chronos-Bolt and TimesFM standardize the series and cut it into patches (16 and 32 data points): 512 observations are 33 or 16 tokens, and attention costs much less. - On the index's log RV, 2018-2025, zero-shot Chronos-Bolt-small gives an MSE of 0.181 against 0.233 for the naive forecast (t = −9.8) and an R² of 0.551, below the README's HAR (0.57) and the oracle (0.61). - Its quantiles are useful but not exact: the 10-90% interval covers 76.4% of the days. Shocks, like the one on March 9, 2020, are not forecast. ## Resources ### Research - [Volatility Estimators for Trading](https://pythonparatrading.com/en/volatility-estimators-for-trading/) · Research-0002 · 4 min ### Documentation - [Hugging Face: amazon/chronos-bolt-small model card](https://huggingface.co/amazon/chronos-bolt-small) · huggingface.co - [chronos-forecasting (Amazon Science GitHub)](https://github.com/amazon-science/chronos-forecasting) · github.com ### Books - [Ansari, A. F. et al. (2024): Chronos: Learning the Language of Time Series. arXiv:2403.07815](https://arxiv.org/abs/2403.07815) · arxiv.org - [Das, A., Kong, W., Sen, R. and Zhou, Y. (2024): A decoder-only foundation model for time-series forecasting. arXiv:2310.10688](https://arxiv.org/abs/2310.10688) · arxiv.org ### Data - [Level 5 data: returns and realized variance of six assets (Parquet)](https://pythonparatrading.com/curso/datos/nivel-5/vol5_diario.parquet) · Download · vol5_diario.parquet Previous lesson: [5.7.5 · End-to-end portfolio optimization: differentiable layers and Sharpe loss](https://pythonparatrading.com/en/python-trading-course/level-5/end-to-end-portfolio-optimization/) Next lesson: [5.8.2 · Foundation model families: architecture, data and license](https://pythonparatrading.com/en/python-trading-course/level-5/chronos-timesfm-moirai-kronos-families/)