---
title: "5.6.3 1D CNN, LSTM and GRU for financial sequences"
description: "Convolutional and recurrent networks to forecast realized volatility: windows, leak-free scaling and an honest comparison with the HAR."
url: "https://pythonparatrading.com/en/python-trading-course/level-5/cnn-1d-lstm-gru-sequences/"
lang: "en"
image: "https://pythonparatrading.com/og/en/python-trading-course/level-5/cnn-1d-lstm-gru-sequences.png"
---

# 5.6.3 · 1D CNN, LSTM and GRU: windows, scaling and leaks

> A recurrent network reads a window of days and decides what to remember; a convolutional one looks for local patterns in it. Both promise to learn from history what a linear model does not see. This lesson builds the windows and the scaling without looking at the future, trains a 1D CNN, an LSTM and a GRU to forecast the volatility of six assets and compares them with the HAR, which in these data is almost the true model.

- Course: [Level 5: Machine learning and deep learning (Keras/PyTorch)](https://pythonparatrading.com/en/python-trading-course/level-5/)
- Module: 5.6 · Deep learning with Keras 3
- Duration: 70 min
- Prerequisites: [5.6.2 · An MLP on a stock panel: regularize and compare](https://pythonparatrading.com/en/python-trading-course/level-5/mlp-for-tabular-financial-data/), [5.4.3 · Walk-forward with retraining: windows and frequency](https://pythonparatrading.com/en/python-trading-course/level-5/walk-forward-retraining-windows/)

## By the end of this lesson

- Build sequence windows with the target strictly after the input and check the alignment with a toy example.
- Scale each series only with its training segment and measure how much a model appears to improve when the scaling or a moving average uses the future.
- Define and train in Keras a causal 1D CNN, an LSTM and a GRU with early stopping on a chronological validation.
- Compare the networks with the HAR, the HAR with leverage and an oracle using a Diebold-Mariano test with Newey-West errors.

## Sequences, not rows

In the previous lessons each row was independent of the others: a feature vector with its label. A time series can be
treated that way if the features already summarize the history (the HAR summarizes 22 days in three means). Sequence
networks skip that step: they receive the **window** of the last days as is, a matrix of $L$ steps by $k$ variables,
and learn their own summary.

- **1D CNN**: filters a few steps long that slide along the window and detect local patterns (a jump, a change of
  slope), the same ones at any position.
- **LSTM** (Hochreiter and Schmidhuber, 1997) and **GRU** (Cho et al., 2014): they read the window step by step and keep
  an internal state; learned gates decide what to forget and what to add at each step. The GRU has two gates and the
  LSTM three, plus a separate cell state: the GRU has fewer parameters with the same state size.

The test bench is the level 5 volatility one: six synthetic assets (index, stock, bond, gold, currency and crypto)
from 2008 to 2025 with their daily realized variance. The true process is a HAR in the logarithm of the integrated
variance, with a leverage effect (the negative part of yesterday's return raises volatility), jumps in the stock and
the crypto, and **level shifts** on known dates (data README). A well-estimated HAR is almost optimal: the question is
whether a network gets there without being told, and whether it goes beyond it.

The split is chronological: training from 2008 to 2015, validation (for early stopping) 2016-2017 and test 2018-2025,
the same as the forecast table of the README. First, the mean level of the log RV in each segment:

```python
import os
os.environ["KERAS_BACKEND"] = "jax"      # before the first import of keras
import numpy as np
import pandas as pd
import statsmodels.api as sm
import keras
from keras import layers

v = pd.read_parquet("vol5_diario.parquet")
verdad = pd.read_parquet("vol5_verdad.parquet")
v = v.merge(verdad[["fecha", "activo", "iv_prevista"]],
            on=["fecha", "activo"]).sort_values(["activo", "fecha"])
v["lrv"] = np.log(v["rv_5min"])
F_VAL = np.datetime64("2016-01-01")
F_PRUEBA = np.datetime64("2018-01-01")
tramo = np.select([v["fecha"] < F_VAL, v["fecha"] < F_PRUEBA],
                  ["train", "validation"], "test")
niveles = v.groupby(["activo", tramo])["lrv"].mean().unstack()
niveles = niveles[["train", "validation", "test"]]
niveles["test - train"] = niveles["test"] - niveles["train"]
niveles.round(2)
```
```output
        train  validation   test  test - train
activo
accion  -8.99       -9.31  -9.24         -0.25
bono   -11.51      -11.61 -11.34          0.17
cripto  -6.95       -7.22  -7.32         -0.37
divisa -10.60      -10.13 -10.41          0.19
indice  -9.85       -9.54  -9.72          0.13
oro     -9.56       -9.92  -9.75         -0.18
```

The stock and the crypto drop in level between training and test (−0.25 and −0.37 in logs; in the stock the 2015 shift
is already inside the training period, which mixes the two regimes); the bond rises (+0.17, because of the 2022 shift)
and so does the currency (+0.19: its level shift of January 2015 leaves almost all of the training period in the
previous regime). A series that is non-stationary in level is the norm in finance, and it conditions how you scale.

## Windows that do not look at the future

The function that cuts the windows is the piece where it is easiest to be off by one step. The rule: the window that
ends on day $t$ contains days $t-L+1$ to $t$, and its target is day $t+1$. With NumPy's `sliding_window_view` each
window is a view, without copying data. Before using it with real data, it is tested with a series whose values are
their own position:

```python
def ventanas(z, L):
    """Windows of L rows ending at t (t = L-1 … n-2) and the
    value of column 0 at t + 1."""
    w = np.lib.stride_tricks.sliding_window_view(
        z, (L, z.shape[1]))[:, 0]
    return w[:-1], z[L:, 0]

juguete = np.arange(8, dtype="float32")[:, None]
Xj, yj = ventanas(juguete, 3)
for xv, yv in zip(Xj[:3], yj[:3]):
    print(xv.ravel().tolist(), "->", float(yv))
print("windows:", len(Xj), "| last target:", float(yj[-1]))
assert all(xv[-1, 0] + 1 == yv for xv, yv in zip(Xj, yj))
```
```output
[0.0, 1.0, 2.0] -> 3.0
[1.0, 2.0, 3.0] -> 4.0
[2.0, 3.0, 4.0] -> 5.0
windows: 5 | last target: 7.0
```

With 8 values and $L = 3$ you get 5 windows: the first, `[0, 1, 2]`, predicts the 3, and the last predicts the 7, the
last value of the series. The assertion checks in all of them that the target is the day after the last day of the
window. If someone changes the function, the assertion fails before anything is trained.

## Scaling can also look at the future

Networks train better with inputs on a similar scale, so each series is standardized: log RV minus its mean, divided
by its standard deviation. The question is **which mean**. The full-sample one includes 2018-2025, and if the series
shifts level, that mean tells the model where it is going to be. To measure how much that information is worth without
training any network, take the simplest possible forecast: predict that the log RV of each test day will be the mean
of the asset. With the training mean it is a legitimate forecast; with the full-sample one, it is not:

```python
def stats(s, hasta):
    """Mean and standard deviation of log RV and standard deviation
    of the return, with the rows before `hasta`."""
    m = s["fecha"] < hasta
    return (s.loc[m, "lrv"].mean(), s.loc[m, "lrv"].std(),
            s.loc[m, "rendimiento"].std())

filas = []
for a, s in v.groupby("activo"):
    prueba = s["fecha"] >= F_PRUEBA
    real = s.loc[prueba, "lrv"]
    m_ent = stats(s, F_VAL)[0]
    m_todo = s["lrv"].mean()                 # uses 2018-2025
    filas.append({"activo": a,
                  "training mean": np.mean((real - m_ent) ** 2),
                  "full-sample mean":
                      np.mean((real - m_todo) ** 2)})
fuga = pd.DataFrame(filas).set_index("activo")
fuga.loc["mean"] = fuga.mean()
fuga.round(3)
```
```output
        training mean  full-sample mean
activo
accion          0.544             0.491
bono            0.292             0.275
cripto          0.795             0.689
divisa          0.254             0.221
indice          0.418             0.402
oro             0.371             0.341
mean            0.446             0.403
```

*Fig. — How much the simplest forecast (each asset's mean) appears to improve over 2018-2025 when the mean uses the full sample instead of the training data only.* Horizontal bars by asset: accion 9.7%; bono 5.8%; cripto 13.3%; divisa 13.0%; indice 3.8%; oro 8.1%. The mean of the six (highlighted), 9.6%.

Computing the mean with the full sample lowers the test MSE of that forecast from 0.446 to 0.403 on average, 9.6% less,
without any model: more than 13% for the crypto and the currency. A network that receives inputs and targets
standardized this way starts with that advantage handed to it, because its initial bias (predicting 0) is already the
mean of the future. It is the same leak as the `StandardScaler` fitted before splitting the data (lesson 5.2.4), and in
series with level shifts it weighs more than in a cross-sectional panel.

The rule: mean and standard deviation of **each asset** from its training segment, and those same figures for
validation and test. Each window carries two variables per day: the standardized log RV and the return divided by its
training standard deviation. The standard deviation of each asset is also kept, to bring the errors back to log RV
units, along with the oracle (`iv_prevista`, the exact expectation of the integrated variance with the information up
to the previous day, which only exists because the data are synthetic):

```python
L = 22
ACTIVOS = sorted(v["activo"].unique())
partes = {k: [] for k in ("X", "y", "f", "a", "sd", "orac")}
for i, a in enumerate(ACTIVOS):
    s = v[v["activo"] == a]
    m, sd, sr = stats(s, F_VAL)              # training only
    z = np.c_[(s["lrv"] - m) / sd,
              s["rendimiento"] / sr].astype("float32")
    Xa, ya = ventanas(z, L)
    partes["X"].append(Xa)
    partes["y"].append(ya)
    partes["f"].append(s["fecha"].to_numpy()[L:])
    partes["a"].append(np.full(len(ya), i))
    partes["sd"].append(np.full(len(ya), sd))
    lor = (np.log(s["iv_prevista"].to_numpy()) - m) / sd
    partes["orac"].append(lor[L:])
X, y, fch, act, SD, orac = (np.concatenate(partes[k]) for k in
                            ("X", "y", "f", "a", "sd", "orac"))
ent = fch < F_VAL
val = (fch >= F_VAL) & (fch < F_PRUEBA)
te = fch >= F_PRUEBA
print("X:", X.shape, "| train, validation, test:",
      int(ent.sum()), int(val.sum()), int(te.sum()))

def mse(p, m):
    """MSE in log RV units (undoes each asset's scaling)."""
    return float(np.mean(((p[m] - y[m]) * SD[m]) ** 2))
```
```output
X: (27042, 22, 2) | train, validation, test: 11958 3018 12066
```

There are 27,042 windows of 22 days by 2 variables: 11,958 for training, 3,018 for validation and 12,066 for test. The
windows at the end of training and those at the start of validation do not overlap in the target (each one predicts a
single day), but they do share input days; with a single boundary and early stopping as the only use of validation,
no further purging is needed.

## The baselines

Four forecasts without a network, all per asset and estimated on the training data:

- **Naive**: tomorrow's log RV is today's.
- **HAR**: OLS on the 1-, 5- and 22-day means of the log RV, computed with the window itself.
- **HAR with leverage**: adds $\ell_t = \min(r_t, 0)$, the negative part of today's return (the $\gamma \ell$ term of
  the true process).
- **Calibrated oracle**: a straight line on the logarithm of `iv_prevista`. The line corrects the bias between the
  integrated variance and the measured realized variance (the RV does not see the overnight part and carries
  measurement error and jumps). It is the floor: no legitimate forecast can go below it except by luck.

```python
def har(X):
    l = X[:, :, 0]
    return np.c_[l[:, -1], l[:, -5:].mean(1), l.mean(1)]

def por_activo(Z):
    """OLS for each asset on its training data; predicts everything."""
    p = np.zeros_like(y)
    for i in range(len(ACTIVOS)):
        mi = act == i
        A_ = np.c_[np.ones(mi.sum()), Z[mi]]
        b = np.linalg.lstsq(A_[ent[mi]], y[mi & ent], rcond=None)[0]
        p[mi] = A_ @ b
    return p

H = har(X)
HL = np.c_[H, np.minimum(X[:, -1, 1], 0)]   # negative part of r_t
pred = {"naive (today's RV)": X[:, -1, 0],
        "HAR": por_activo(H),
        "HAR + leverage": por_activo(HL),
        "calibrated oracle": por_activo(orac[:, None])}
resumen = pd.Series({k: mse(p, te) for k, p in pred.items()},
                    name="test MSE")
resumen.round(4)
```
```output
naive (today's RV)    0.3184
HAR                   0.2137
HAR + leverage        0.2114
calibrated oracle     0.1962
Name: test MSE, dtype: float64
```

The naive forecast has an MSE of 0.318; the HAR goes down to 0.2137, leverage to 0.2114 and the oracle marks 0.1962.
Between the HAR with leverage and the oracle there is a 7% gap: that is the most a network can gain here. Any test
result below 0.1962 is, almost certainly, a leak.

A common way to commit one: smoothing with a centered moving average, which in pandas is an argument (`center=True`)
and in many indicator libraries is the default option of some filter. The cell replaces the HAR's weekly mean with the
one centered at $t$, which uses $t-2$ to $t+2$:

```python
c5 = []
for a in ACTIVOS:
    s = v[v["activo"] == a]
    m, sd, _ = stats(s, F_VAL)
    zc = ((s["lrv"] - m) / sd).rolling(5, center=True).mean()
    c5.append(zc.to_numpy()[L - 1:-1])    # aligned with the end t
c5 = np.nan_to_num(np.concatenate(c5))
H_mal = np.c_[H[:, 0], c5, H[:, 2]]   # t-2 … t+2 instead of t-4 … t
print(f"HAR with the centered weekly mean: MSE "
      f"{mse(por_activo(H_mal), te):.4f}")
```
```output
HAR with the centered weekly mean: MSE 0.1328
```

An MSE of 0.1328, 32% below the oracle. The model has learned nothing about volatility: the centered mean contains the
day to be forecast ($t+1$) and the next one. With a network the error is just as easy to make and harder to see,
because there are no coefficients to look at: a `Bidirectional` layer on a sequence that reaches the target, a filter
applied to the whole series before cutting windows or symmetric padding in a convolution that predicts at every step.
The comparison with the oracle, when there is one, and with an honest baseline, when there is not, is the defense.

## Three architectures in Keras

All three receive the same input, `(22, 2)`, and end in a `Dense(1)`. With the JAX backend, as in 5.6.2:

```mermaid
flowchart LR
  V[Window 22 x 2] --> C[Causal Conv1D]
  V --> LS[LSTM 16]
  V --> G[GRU 16]
  C --> F[Flatten]
  F --> D[Dense 1]
  LS --> D
  G --> D
```

- The **1D CNN** applies 16 filters of 5 days with `causal` padding: the output at position $t$ only depends on $t-4$
  to $t$. Then it flattens the 22 positions and combines them in the dense layer; that way it keeps where each pattern
  is (a `GlobalAveragePooling1D` would lose it, and in volatility what is recent matters more).
- The **LSTM** and the **GRU** read the 22 days in order and pass their final state, of 16 numbers, to the dense layer.

```python
def red(tipo, n=16):
    x_in = keras.Input((L, 2))
    if tipo == "CNN 1D":
        h = layers.Conv1D(n, 5, activation="relu",
                          padding="causal")(x_in)
        h = layers.Flatten()(h)
    elif tipo == "LSTM":
        h = layers.LSTM(n)(x_in)
    else:
        h = layers.GRU(n)(x_in)
    m = keras.Model(x_in, layers.Dense(1)(h),
                    name=tipo.replace(" ", ""))
    m.compile(keras.optimizers.Adam(1e-3), "mse")
    return m

for tipo in ("CNN 1D", "LSTM", "GRU"):
    print(f"{tipo:<7}{red(tipo).count_params():>5} parameters")
```
```output
CNN 1D   529 parameters
LSTM    1233 parameters
GRU      977 parameters
```

The three have between 500 and 1,250 parameters, against the 5 per asset of the HAR with leverage (30 in total). The
LSTM has more than the GRU with the same state: $4 \times (16 \times (2 + 16) + 16) = 1{,}216$ in the layer against 960.

Training uses Adam (0.001), batches of 512 and at most 30 epochs, with early stopping after 3 epochs without
improvement on validation and the weights of the best epoch. The six assets are trained **together** (a panel model
with the standardized series): networks need more data than a HAR, and standardizing by asset makes the series
comparable.

```python
historia = {}
for tipo in ("CNN 1D", "LSTM", "GRU"):
    keras.utils.set_random_seed(0)
    m = red(tipo)
    parada = keras.callbacks.EarlyStopping(
        patience=3, restore_best_weights=True)
    h = m.fit(X[ent], y[ent], epochs=30, batch_size=512, verbose=0,
              validation_data=(X[val], y[val]), callbacks=[parada])
    historia[tipo] = h.history["val_loss"]
    pred[tipo] = m.predict(X, batch_size=4096, verbose=0).ravel()
    mejor = int(np.argmin(h.history["val_loss"])) + 1
    print(f"{tipo:<7} epochs {len(h.history['loss']):>2}, best "
          f"validation in {mejor}")
tabla = pd.Series({k: mse(p, te) for k, p in pred.items()},
                  name="test MSE").sort_values()
tabla.round(4)
```
```output
CNN 1D  epochs 27, best validation in 24
LSTM    epochs 19, best validation in 16
GRU     epochs 19, best validation in 16
calibrated oracle     0.1962
GRU                   0.2106
LSTM                  0.2107
HAR + leverage        0.2114
HAR                   0.2137
CNN 1D                0.2137
naive (today's RV)    0.3184
Name: test MSE, dtype: float64
```

Early stopping kicked in for all three: the LSTM and the GRU stopped at epoch 19 with their best validation at 16, the
CNN at 27. In the test (the figure at the top puts the seven forecasts on one scale):

- The two recurrent networks end up **almost tied with the HAR with leverage**: 0.2106 the GRU, 0.2107 the LSTM, 0.2114
  the HAR with leverage.
- The 1D CNN, at 0.2137, matches the HAR without leverage: it does not exploit the information in the returns as well.
- All of them stay far from the oracle, 0.1962. The distance is the same for the network and for the linear model.

## Is the difference larger than the noise?

A difference of 0.0008 in the MSE says nothing without its standard error. The Diebold and Mariano (1995) test compares
two forecasts with the series of differences of their losses: if the mean of $d_t = e_{a,t}^2 - e_{b,t}^2$ cannot be
told apart from zero, the two forecasts are equally good. The daily losses are autocorrelated (volatility has memory
and so do the errors), so the standard error of the mean is computed with Newey-West. Here, $d_t$ is the mean over the
six assets of each day:

```python
def diebold_mariano(a, b, lags=10):
    """Mean of the daily loss difference (a - b), averaged across
    assets, with a Newey-West standard error."""
    e = pd.DataFrame({"f": fch[te],
                      "d": ((pred[a][te] - y[te]) * SD[te]) ** 2
                      - ((pred[b][te] - y[te]) * SD[te]) ** 2})
    d = e.groupby("f")["d"].mean().to_numpy()
    r = sm.OLS(d, np.ones_like(d)).fit(
        cov_type="HAC", cov_kwds={"maxlags": lags})
    return r.params[0], r.bse[0], r.tvalues[0]

for a, b in [("GRU", "HAR + leverage"),
             ("LSTM", "HAR + leverage"),
             ("HAR + leverage", "HAR"),
             ("calibrated oracle", "HAR + leverage")]:
    dif, et, t = diebold_mariano(a, b)
    print(f"{a:<21}- {b:<21}{dif:+.4f} ± {et:.4f}  t = {t:+.1f}")
```
```output
GRU                  - HAR + leverage       -0.0008 ± 0.0010  t = -0.8
LSTM                 - HAR + leverage       -0.0007 ± 0.0009  t = -0.8
HAR + leverage       - HAR                  -0.0023 ± 0.0005  t = -4.6
calibrated oracle    - HAR + leverage       -0.0152 ± 0.0013  t = -11.6
```

The GRU and the LSTM improve on the HAR with leverage by 0.0008 and 0.0007, with a standard error of 0.0010:
$t = -0.8$. **They cannot be told apart.** The test has more than enough power to see real differences: the leverage
term improves on the HAR with $t = -4.6$, and the oracle on the HAR with leverage with $t = -11.6$. What happens is
that the network and the linear model learn the same thing. In these data it was predictable: the true process is a
HAR with leverage, and a network can only win on what the linear model does not capture (the jumps, the nonlinearity
of leverage, the interactions between assets).

By asset, the average hides opposite behaviors:

```python
por = pd.DataFrame({k: [mse(pred[k], te & (act == i))
                        for i in range(len(ACTIVOS))]
                    for k in ("HAR", "HAR + leverage", "GRU",
                              "calibrated oracle")},
                   index=ACTIVOS)
por["GRU / HAR+lev."] = por["GRU"] / por["HAR + leverage"]
por.round(3)
```
```output
          HAR  HAR + leverage    GRU  calibrated oracle  GRU / HAR+lev.
accion  0.309           0.308  0.296              0.283           0.962
bono    0.134           0.133  0.136              0.130           1.018
cripto  0.398           0.397  0.385              0.354           0.969
divisa  0.121           0.121  0.121              0.115           1.000
indice  0.174           0.162  0.171              0.157           1.056
oro     0.147           0.147  0.155              0.139           1.052
```

*Fig. — Ratio of the GRU test MSE (2018-2025) to the HAR-with-leverage MSE, asset by asset: below 1 the network wins.* Bars from 1. Stock 0.962 and crypto 0.969 (the GRU wins); FX 1.000; bond 1.018, gold 1.052 and index 1.056 (highlighted; the HAR with leverage wins).

The GRU wins on the stock (ratio 0.962) and on the crypto (0.969), the two assets with **jumps** in the true process:
after a jump, the day's RV spikes without the background volatility changing as much, and a network that sees the
shape of the window can discount it better than a linear HAR, which treats a jump as persistent volatility. It loses on
the index (1.056) and gold (1.052): in the index leverage is the strongest of the six ($\gamma = 0.18$), and a panel
model with a single network for the six assets captures it worse than a HAR with its own coefficient.

This suggests the next reasonable step, which is not a bigger network: a HAR per asset with a term for the jumps (the
bipower variation, `bv_5min`, separates the continuous part from the jump part) or a small network that corrects the
HAR's residuals. And remember the rule of the module: the network is compared with the best linear baseline that can
be built, not with the weakest.

<aside class="callout" data-kind="nota">

In production you would train with more epochs, several averaged seeds (lesson 5.6.2) and longer windows, and retrain
every year (walk-forward, lesson 5.4.3). Here training is kept short so that the lesson runs in a few minutes on a CPU.
The order of magnitude of the conclusions does not change: with an almost linear process, the network matches the
well-specified linear model.

</aside>

## Sequences beyond volatility

The same pieces work for other sequences in the course, with the same traps:

| Application | Window | Typical trap |
|---|---|---|
| Daily volatility (this lesson) | 22 days of log RV and return | scaling with the full sample; centered means |
| 1-minute bars (`hf5`) | 30-120 minutes of return, volume and imbalance | windows that cross the open; normalizing with the whole day |
| Returns of a panel | 52 weeks of returns per stock | windows of stocks that did not exist yet (survivorship) |
| Order book | 100 updates of the levels | event timestamps versus receipt timestamps |

And for returns, not for volatility, remember the README's control: the conditional mean of `rendimiento` is constant
in these data, so no network can predict it. An LSTM that “gets right” tomorrow's direction in `vol5_diario` is
measuring noise or has a leak.

## Exercises

1. Train the GRU with only the first column of the window (the log RV, without the returns). How much worse is it in
   the test? Which part of the true model explains the difference?

**Solution.**

```python
keras.utils.set_random_seed(0)
x_in = keras.Input((L, 1))
m1 = keras.Model(x_in, layers.Dense(1)(layers.GRU(16)(x_in)))
m1.compile(keras.optimizers.Adam(1e-3), "mse")
m1.fit(X[ent][:, :, :1], y[ent], epochs=30, batch_size=512, verbose=0,
       validation_data=(X[val][:, :, :1], y[val]),
       callbacks=[keras.callbacks.EarlyStopping(
           patience=3, restore_best_weights=True)])
pred["GRU without returns"] = m1.predict(
    X[:, :, :1], batch_size=4096, verbose=0).ravel()
for k in ("HAR", "GRU without returns", "GRU"):
    print(f"{k:<21}{mse(pred[k], te):.4f}")
```
```output
HAR                  0.2137
GRU without returns  0.2117
GRU                  0.2106
```

Without the returns, the GRU ends up at 0.2117: better than the HAR (0.2137) and worse than the full GRU (0.2106). The
difference from the full GRU, 0.0011, is of the order of the one between the HAR and the HAR with leverage (0.0023):
the negative part of the return carries information that the log RV does not have. What the network without returns
gains over the HAR comes from handling the jumps better (stock and crypto).

2. Compute the R² of the regression of the test log RV on the forecast (the square of the correlation) for the HAR with
   leverage, the GRU and the oracle, asset by asset, and compare it with the table in the data README.

**Solution.**

```python
def r2(k, i):
    m = te & (act == i)
    return np.corrcoef(pred[k][m], y[m])[0, 1] ** 2

pd.DataFrame({k: [r2(k, i) for i in range(len(ACTIVOS))]
              for k in ("HAR + leverage", "GRU",
                        "calibrated oracle")},
             index=ACTIVOS).round(2)
```
```output
        HAR + leverage   GRU  calibrated oracle
accion            0.36  0.39               0.41
bono              0.50  0.49               0.51
cripto            0.40  0.42               0.46
divisa            0.45  0.45               0.48
indice            0.60  0.57               0.61
oro               0.57  0.56               0.60
```

The R² of the HAR with leverage and of the calibrated oracle are very close to those of the README (0.57 and 0.61 for
the index with the HAR re-estimated every year and the exact oracle; here 0.60 and 0.61, with a HAR estimated once and
by OLS on the window). The GRU has a higher R² than the HAR on the stock (0.39 against 0.36) and on the crypto (0.42
against 0.40) and a lower one on the index (0.57 against 0.60): the same story as the MSE by asset.

3. A colleague improves the GRU by changing `layers.GRU(16)` to `layers.Bidirectional(layers.GRU(16))` and presents a
   drop in the test MSE. With the windows of this lesson, can it be a leak? And if the model predicted the volatility
   of **each** day of the window (sequence to sequence)?

**Solution.**

With the windows of this lesson, no: each window ends at $t$ and the target is $t+1$, outside the window; reading it
backward as well as forward adds no future information, only another way of summarizing the past (and twice the
parameters). The improvement would have to be tested as always: Diebold-Mariano against the HAR with leverage and
several seeds. In a sequence-to-sequence model, on the other hand, the output of step $s$ is compared with the value of
$s+1$, and the backward pass of the bidirectional layer sees steps $s+1, s+2, \ldots$ of the same window: the target is
inside the input. It is the same leak as the centered moving average. Bidirectional layers are useful when the whole
sequence is known at the time of deciding (classifying a day that has already closed, for example), never for
predicting within it.

## Self-assessment

1. Each asset's log RV is standardized with its 2008-2025 mean and standard deviation before splitting into train, validation and test. What happens?

   - A. A leak: the mean includes the 2018-2025 level, and in these data that alone makes the simplest forecast appear to improve by 9.6% on average in the test
   - B. Nothing: standardizing is a linear transformation and does not change the information
   - C. The model trains more slowly
   - D. The windows stop overlapping

   *Answer A: A leak: the mean includes the 2018-2025 level, and in these data that alone makes the simplest forecast appear to improve by 9.6% on average in the test.* With level shifts (the bond rises in 2022, crypto falls in 2019), the full-sample mean tells the model where the series will be. The lesson measures it with the forecast “mean of each asset”: the test MSE drops from 0.446 to 0.403 just by computing the mean with everything.

2. A HAR with the weekly mean computed with a centered window (t−2 to t+2) gives a test MSE of 0.1328, better than the oracle (0.1962). What do you conclude?

   - A. That the centered mean is a better feature
   - B. That the oracle is badly calibrated
   - C. That the HAR must be retrained
   - D. That there is a leak: no model can go below the oracle, which uses the exact expectation with the available information; the centered window includes the value to be forecast and the next one

   *Answer D: That there is a leak: no model can go below the oracle, which uses the exact expectation with the available information; the centered window includes the value to be forecast and the next one.* A result below the theoretical floor is the clearest sign of future information. Centered moving averages, filters that smooth in both directions and bidirectional layers have the same problem.

3. The GRU has a test MSE of 0.2106 and the HAR with leverage 0.2114. The mean daily difference is −0.0008 ± 0.0010 (Newey-West). What does it say?

   - A. That the GRU is better
   - B. That they cannot be told apart: t = −0.8; the GRU matches, but does not beat, a linear model with five parameters per asset
   - C. That the HAR is overfitted
   - D. That more epochs are needed

   *Answer B: That they cannot be told apart: t = −0.8; the GRU matches, but does not beat, a linear model with five parameters per asset.* The same test does tell the HAR with leverage apart from the plain HAR (t = −4.6) and the oracle apart from the HAR with leverage (t = −11.6). The power is there; the difference between the network and the linear model is simply noise.

4. Why does the 1D CNN use `padding="causal"`?

   - A. Because it is faster
   - B. Because Keras does not support any other padding in Conv1D
   - C. So that the output has the same length and each position only sees earlier or equal positions; with `same`, the convolution at position t would use t+1 and t+2 of the window
   - D. To reduce the number of parameters

   *Answer C: So that the output has the same length and each position only sees earlier or equal positions; with `same`, the convolution at position t would use t+1 and t+2 of the window.* Inside a window that ends at t there is no future data, so here `same` would not leak anything after t. But the habit matters: in models that predict at every step (sequence to sequence), symmetric padding brings in the future of the sequence itself.

5. Without the returns in the window, the GRU goes from 0.2106 to 0.2117. Which part of the true process explains the difference?

   - A. The leverage effect: tomorrow's volatility depends on the negative part of today's return, which the log RV alone does not contain
   - B. The jumps
   - C. The level shifts
   - D. None: it is seed noise

   *Answer A: The leverage effect: tomorrow's volatility depends on the negative part of today's return, which the log RV alone does not contain.* The generator includes a leverage term (the negative part of yesterday's shock). The HAR improves with the same term (from 0.2137 to 0.2114). The network without returns sits between the two.

## Key points

- A window is built with the input up to t and the target at t+1; it is checked with a toy example and an assertion, not by eye.
- The scaling is fitted on each series' training segment. With level shifts, centering with the full sample makes the simplest forecast appear 9.6% better on average; a centered moving average takes the HAR below the oracle.
- A causal 1D CNN (529 parameters), an LSTM (1,233) and a GRU (977) learn from 22-day windows of log RV and return, with early stopping on a chronological validation.
- The recurrent networks match the HAR with leverage (0.2106 and 0.2107 against 0.2114; t = −0.8) and stay far from the oracle (0.1962): in these data the true process is almost linear and the HAR already captures it.
- The network wins on the assets with jumps (stock, crypto) and loses where leverage is strong (index): the average hides opposite behaviors.

## Resources

### Documentation

- [Keras: LSTM layer](https://keras.io/api/layers/recurrent_layers/lstm/) · keras.io
- [Keras: Conv1D layer (causal padding)](https://keras.io/api/layers/convolution_layers/convolution1d/) · keras.io

### Books

- [Hochreiter, S. and Schmidhuber, J. (1997): Long Short-Term Memory. Neural Computation 9(8), 1735-1780](https://doi.org/10.1162/neco.1997.9.8.1735) · doi.org
- [Cho, K. et al. (2014): Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation (GRU). arXiv:1406.1078](https://arxiv.org/abs/1406.1078) · arxiv.org
- [Corsi, F. (2009): A Simple Approximate Long-Memory Model of Realized Volatility. Journal of Financial Econometrics 7(2), 174-196](https://doi.org/10.1093/jjfinec/nbp001) · doi.org
- [Diebold, F. X. and Mariano, R. S. (1995): Comparing Predictive Accuracy. Journal of Business & Economic Statistics 13(3), 253-263](https://doi.org/10.1080/07350015.1995.10524599) · doi.org

### Data

- [Level 5 data: returns and realized variance of six assets (Parquet)](https://pythonparatrading.com/curso/datos/nivel-5/vol5_diario.parquet) · Download · vol5_diario.parquet

Previous lesson: [5.6.2 · An MLP on a stock panel: regularize and compare](https://pythonparatrading.com/en/python-trading-course/level-5/mlp-for-tabular-financial-data/)

Next lesson: [5.6.4 · Callbacks, a Sharpe loss and the model in production](https://pythonparatrading.com/en/python-trading-course/level-5/keras-callbacks-custom-loss-export/)