--- title: "5.3.5 Event-based sampling: the CUSUM filter" description: "Choosing when to observe: symmetric CUSUM filter with a volatility-scaled threshold, events by threshold and comparison with time bars." url: "https://pythonparatrading.com/en/python-trading-course/level-5/event-sampling-cusum-filter/" lang: "en" image: "https://pythonparatrading.com/og/en/python-trading-course/level-5/event-sampling-cusum-filter.png" --- # 5.3.5 · Event-based sampling: the symmetric CUSUM filter > A model that looks at the market every minute spends most of its time watching noise. The symmetric CUSUM filter samples only when the price has accumulated a large enough move since the last observation. This lesson builds it on FNX, studies the fixed threshold and the volatility-scaled one, measures how many events each threshold gives and in which regime they fall, and compares it with sampling at fixed time intervals and with a band rule. - Course: [Level 5: Machine learning and deep learning (Keras/PyTorch)](https://pythonparatrading.com/en/python-trading-course/level-5/) - Module: 5.3 · Labelling and sampling - Duration: 55 min - Prerequisites: [1.4.5 · Building bars: time, tick, volume, dollar and imbalance](https://pythonparatrading.com/en/python-trading-course/level-1/time-tick-volume-and-dollar-bars/), [4.2.5 · Drawdown control and kill switch](https://pythonparatrading.com/en/python-trading-course/level-4/drawdown-control-and-kill-switch/), [5.3.1 · Fixed-horizon labeling and its problems](https://pythonparatrading.com/en/python-trading-course/level-5/fixed-horizon-labeling/) ## By the end of this lesson - Implement the symmetric CUSUM filter with a fixed threshold or one scaled by an EWMA volatility, without later information. - Measure how the number of events, their distribution by hour and by regime change with the threshold, and why a fixed threshold depends on the volatility of each session. - Compare event-based sampling with time-based sampling at the same frequency and with a band rule, using the known answer in the data. - Apply the same filter to a trade tape and relate it to the information bars of lesson 1.4.5. ## When to observe Lessons 5.3.1 and 5.3.2 labeled observations; what remains is deciding **which observations**. The default option is one per bar: every minute, every day. It has two problems. The first is that most of those observations are noise: a midday minute in which the price does not move teaches nothing about the signal being sought. The second is that consecutive observations are nearly identical: features that barely change and labels that overlap, the problem that lesson 5.3.4 measured. Lesson 1.4.5 tackled a similar problem when building the bars: tick, volume or dollar bars, which close when there is activity and not when the clock ticks. **Event-based sampling** (López de Prado, 2018, section 2.5.2) goes one step further: given a series, whether of time bars or activity bars, it observes only when something relevant has happened. The most widely used filter is the **symmetric CUSUM**, a variant of Page's (1954) cumulative sums that lesson 4.2.5 used to detect that a strategy had stopped working. ## The symmetric CUSUM filter With $y_t$ the log return of each bar, the filter keeps two sums, one of rises and one of falls, clipped at zero: $$ S^+_t = \max\left(0,\; S^+_{t-1} + y_t\right), \qquad S^-_t = \min\left(0,\; S^-_{t-1} + y_t\right). $$ When $S^+_t > h_t$ there is an upward event; when $S^-_t < -h_t$, a downward one. After each event both sums go back to zero. Clipping at zero means a fall is not offset by an earlier rise: $S^+$ measures the cumulative rise since the last relative minimum, and the event arrives when that rise exceeds $h_t$. López de Prado's version subtracts from $y_t$ its expected mean, which for one-minute returns is zero for all practical purposes. ```mermaid stateDiagram-v2 [*] --> Accumulate Accumulate --> Accumulate: below h Accumulate --> Event: S+ > h or S− < −h Event --> Accumulate: reset to 0 ``` The data are FNX's (level 5 **synthetic** data): one year of 1-minute bars of a fictitious future with a latent regime of trend or reversion, which `hf5_verdad.parquet` gives minute by minute. As in the previous lessons, the regime is used only for measuring. ```python import numpy as np import pandas as pd barras = pd.read_parquet("hf5_barras_1min.parquet") verdad = pd.read_parquet("hf5_verdad.parquet") n_ses = len(barras) // 390 C = barras["cierre"].to_numpy().reshape(n_ses, 390) tend = (verdad["regimen"] == "tendencia").to_numpy() tend = tend.reshape(n_ses, 390) # the truth: only for measuring lr = np.diff(np.log(C), axis=1) # 389 per session sig = (pd.Series(lr.ravel()).ewm(span=100).std() .to_numpy().reshape(n_ses, 389)) print(f"minutes in trend: {tend.mean():.1%}") ``` ```output minutes in trend: 35.3% ``` 35.3% of the year's minutes are in trend. The threshold of the level's primary model is $h_t = 4\,\hat\sigma_t$, with $\hat\sigma_t$ the EWMA deviation (span of 100 minutes) of the 1-minute returns computed up to that very minute: it is known at the close of the minute that fires the event, which is when the decision is made. ```python def cusum(lr, umbral): """Symmetric CUSUM filter, session by session. lr: log returns (sessions x minutes); umbral: same shape (or a number). Accumulates rises in s_pos and falls in s_neg since the last event; when one exceeds the threshold, event, and both go back to 0. Returns session, minute (index of the close that fires), side (+1 upward break) and the threshold used.""" h = np.broadcast_to(umbral, lr.shape) filas = [] for d in range(lr.shape[0]): s_pos = s_neg = 0.0 for m in range(lr.shape[1]): if not np.isfinite(h[d, m]): continue s_pos = max(0.0, s_pos + lr[d, m]) s_neg = min(0.0, s_neg + lr[d, m]) if s_pos > h[d, m] or s_neg < -h[d, m]: lado = 1 if s_pos > h[d, m] else -1 filas.append((d, m + 1, lado, h[d, m])) s_pos = s_neg = 0.0 return pd.DataFrame( filas, columns=["sesion", "minuto", "lado", "umbral"]) ev = cusum(lr, 4.0 * sig) print(len(ev), "events,", round(len(ev) / n_ses, 1), "per session") print("minutes between events (median):", ev.groupby("sesion").minuto.diff().median()) ev[ev.sesion == 133].head(6).assign( umbral_pb=lambda x: (1e4 * x.umbral).round(1)).drop( columns="umbral") ``` ```output 6550 events, 26.5 per session minutes between events (median): 13.0 sesion minuto lado umbral_pb 3525 133 4 -1 14.2 3526 133 6 1 16.0 3527 133 11 1 16.6 3528 133 38 1 18.1 3529 133 45 1 18.8 3530 133 49 1 19.6 ``` The result is the README's 6,550 events, 26.5 per session, with a median of 13 minutes between one and the next. The header figure shows the session of January 15, 2025, the one used in the examples of lessons 5.3.2 and 5.3.4. In the morning uptrend, almost all events are upward: the price rises, the filter accumulates and fires again and again on the same side. In the afternoon reversion, upward and downward events alternate. The first three events arrive in the first 11 minutes with thresholds of 14 to 17 bp. The EWMA carries over the volatility of the previous afternoon, lower than that of the open, so the threshold is narrow precisely when the price moves most. It is the same lag that lesson 5.3.2 measured in the barriers. ## Fixed or scaled threshold The simplest alternative is a fixed threshold in basis points. With the same average level (4 times the median of $\hat\sigma$, 16.9 bp): ```python h_fijo = 4.0 * np.nanmedian(sig) ev_fijo = cusum(lr, h_fijo) print(f"fixed threshold: {1e4 * h_fijo:.1f} bp; {len(ev_fijo)} events") vol_ses = lr.std(axis=1) # volatility of each session tercil = pd.qcut(vol_ses, 3, labels=["low", "medium", "high"]) por_sesion = pd.DataFrame({ "tercil": tercil, "escalado": np.bincount(ev.sesion, minlength=n_ses), "fijo": np.bincount(ev_fijo.sesion, minlength=n_ses)}) por_sesion.groupby("tercil", observed=True).mean().round(1) ``` ```output fixed threshold: 16.9 bp; 6897 events escalado fijo tercil low 26.9 19.4 medium 26.6 27.6 high 26.0 36.9 ``` With a fixed threshold, calm sessions (low volatility tercile) give 19.4 events and turbulent ones 36.9: almost twice as many. A model trained on those events learns mostly from turbulent days, and the same 17 bp move means different things depending on the day. With the scaled threshold, each tercile gives between 26.0 and 26.9 events: the filter requires the same move **in units of volatility**. To know which events are better, they have to be labeled. With the triple barrier of lesson 5.3.2 and the side of the break (the momentum primary model): ```python def acierto(ev, h=30): """Triple barrier ±σ·√h (5.3.2); 1 if the side is right.""" out = [] for d, m, lado in ev[["sesion", "minuto", "lado"]].itertuples( index=False): b = sig[d, m - 1] * np.sqrt(h) ult = min(m + h, 389) tramo = np.log(C[d, m + 1:ult + 1] / C[d, m]) golpes = np.flatnonzero(np.abs(tramo) >= b) j = golpes[0] if len(golpes) else len(tramo) - 1 out.append(int(len(tramo) > 0 and lado * tramo[j] > 0)) return np.array(out) def resumen(ev): a = acierto(ev) t = tend[ev.sesion, ev.minuto] return {"por_sesion": len(ev) / n_ses, "en_tendencia": t.mean(), "acierto": a.mean(), "acierto_tend": a[t].mean(), "acierto_rev": a[~t].mean()} pd.DataFrame({"scaled k = 4": resumen(ev), "fixed": resumen(ev_fijo)}).T.round(3) ``` ```output por_sesion en_tendencia acierto acierto_tend acierto_rev scaled k = 4 26.518 0.388 0.507 0.604 0.446 fixed 27.923 0.388 0.504 0.588 0.450 ``` Both filters give a fraction of events in trend of 38.8% and a similar hit rate: 50.7% for the scaled one and 50.4% for the fixed one, a difference of half a standard error. The scaled one is somewhat more accurate in trend (60.4% against 58.8%). The main reason to scale is not the average hit rate but the composition of the sample. ## Threshold and number of events The threshold decides how many events there are. Sweeping it: ```python barrido = pd.DataFrame({k: resumen(cusum(lr, k * sig)) for k in (2, 3, 4, 5, 6, 8)}).T barrido.index.name = "k" barrido.round(3) ``` ```output por_sesion en_tendencia acierto acierto_tend acierto_rev k 2 68.575 0.364 0.502 0.565 0.465 3 40.308 0.373 0.503 0.575 0.460 4 26.518 0.388 0.507 0.604 0.446 5 18.785 0.396 0.506 0.612 0.436 6 13.976 0.413 0.507 0.615 0.431 8 8.636 0.441 0.507 0.637 0.404 ``` *Fig. — Raising the CUSUM threshold reduces the events and concentrates the remaining ones in the trend regime: from 36% to 44% of the events, against 35% of the minutes. The overall hit rate does not change because reversion worsens at the same time.* Two panels. Left, events per session by threshold k: 68.6 with k = 2, 40.3 with 3, 26.5 with 4 (highlighted), 18.8 with 5, 14.0 with 6 and 8.6 with 8. Right, three lines: the primary's hit rate in trend rises from 56.5% to 63.7%, in reversion falls from 46.5% to 40.4%, and the share of events in trend rises from 36.4% to 44.1%, above the 35% of minutes (dashed line). - **Events fall quickly with the threshold**: 68.6 per session with $k = 2$, 26.5 with $k = 4$ and 8.6 with $k = 8$. For a random walk, the expected time until a cumulative sum travels $k\sigma$ grows with $k^2$; on FNX, from $k = 2$ to $k = 4$ the events are divided by 2.6 and from 4 to 8 by 3.1, somewhat less than 4 because the session is cut at the close and the trend episodes speed up the path. - **Events concentrate in trend**: from 36.4% with $k = 2$ to 44.1% with $k = 8$, against 35.3% of the minutes. A large move is more likely during a trend, and the filter finds it. - **The total hit rate does not move** (from 50.2% to 50.7%), because while the hit rate in trend rises from 56.5% to 63.7%, the one in reversion falls from 46.5% to 40.4%: in a mean-reversion regime, the larger the deviation, the stronger the pullback, and the momentum side is wrong more often. The threshold is therefore a trade-off between quantity and composition: with $k = 8$ there are “purer” samples but a third of those with $k = 4$, and with them more variance in everything that is estimated. Like the width of the barriers (5.3.2), it is set by design and not by looking at the hit rate on the test sample. ## By events or by time Does the CUSUM gain anything over observing at fixed time intervals? With the momentum side computed over the previous interval and the same label: ```python def por_tiempo(n): """One sample every n minutes; side = sign of the previous n.""" filas = [] for d in range(n_ses): for m in range(n, 390, n): r = np.log(C[d, m] / C[d, m - n]) if r != 0: filas.append((d, m, int(np.sign(r)))) return pd.DataFrame(filas, columns=["sesion", "minuto", "lado"]) tiempo = pd.DataFrame({f"every {n} min": resumen(por_tiempo(n)) for n in (5, 15, 30)}).T tiempo.round(3) ``` ```output por_sesion en_tendencia acierto acierto_tend acierto_rev every 5 min 75.332 0.353 0.503 0.556 0.474 every 15 min 24.664 0.353 0.505 0.589 0.460 every 30 min 11.879 0.354 0.511 0.592 0.466 ``` Every 15 minutes gives 24.7 observations per session, almost the same as the CUSUM with $k = 4$. The differences: - Time-based sampling distributes the observations like the minutes: 35.3% in trend, no more and no less. The CUSUM tilts them toward trend (38.8%). - The total hit rate is almost the same: 50.5% every 15 minutes and 50.7% with the CUSUM. With about 6,000 events each, the standard error of each hit rate is around 0.6 points: there is no measurable difference. The CUSUM does not make good a primary model that is not; it changes which moments are observed. Where the difference does show is in **when** the observation happens: ```python def por_media_hora(ev): return np.bincount(ev.minuto // 30, minlength=13) / n_ses hora = pd.DataFrame({ "scaled CUSUM": por_media_hora(ev), "fixed CUSUM": por_media_hora(ev_fijo), "every 15 min": por_media_hora(por_tiempo(15))}, index=[f"{9 + (30 + 30 * i) // 60}:{(30 + 30 * i) % 60:02d}" for i in range(13)]) hora.round(1) ``` ```output scaled CUSUM fixed CUSUM every 15 min 9:30 3.0 4.1 1.0 10:00 1.8 2.7 2.0 10:30 1.6 2.2 2.0 11:00 1.6 1.9 2.0 11:30 1.7 1.8 2.0 12:00 1.9 1.8 2.0 12:30 1.9 1.8 2.0 13:00 2.0 1.8 2.0 13:30 2.1 1.8 2.0 14:00 2.1 1.8 2.0 14:30 2.1 1.9 2.0 15:00 2.2 2.1 2.0 15:30 2.4 2.3 2.0 ``` *Fig. — Events per half hour and session. Time sampling is flat; the CUSUM follows activity: more events at the open and the close. With a fixed threshold the U is sharper; scaled by an EWMA, the open still weighs more because the EWMA lags.* Three lines by half hour from 9:30 to 15:30. One sample every 15 minutes: 1 in the first half hour and 2 in the others. CUSUM with a fixed threshold: 4.1 events per session in the first half hour, falling to 1.8 at midday and rising to 2.3 in the last. Scaled CUSUM (highlighted): 3.0 in the first, 1.6 in mid-morning and a gradual rise to 2.4 in the last. Time-based sampling is flat (the first half hour has a single observation because the interval starts at 9:45). The CUSUM follows activity: with a fixed threshold, 4.1 events per session in the first half hour and 1.8 at midday, the same U that the volume bars of lesson 1.4.5 drew. Scaled, the U flattens but does not disappear: 3.0 at the open because of the EWMA lag and a gentle rise toward the close. If sampling uniform in units of volatility were wanted, the threshold would have to use the intraday profile, like the barrier of lesson 5.3.2. ## Bands against CUSUM: why the reset matters An intuitive rule for sampling is the **band** rule: an event every time the price leaves a band around its moving average (in the style of Bollinger bands). With a 30-minute exponential average and bands of $\pm 0.5\,\sigma\sqrt{30}$: ```python lc = np.log(C) ema = pd.DataFrame(lc.T).ewm(span=30).mean().to_numpy().T z = np.full_like(lc, np.nan) z[:, 1:] = (lc[:, 1:] - ema[:, 1:]) / (sig * np.sqrt(30)) fuera_arr, fuera_ab = z > 0.5, z < -0.5 cruce = np.zeros_like(lc, dtype=int) cruce[:, 1:] = (np.where(fuera_arr[:, 1:] & ~fuera_arr[:, :-1], 1, 0) - np.where(fuera_ab[:, 1:] & ~fuera_ab[:, :-1], 1, 0)) d_b, m_b = np.nonzero(cruce) bandas = pd.DataFrame({"sesion": d_b, "minuto": m_b, "lado": cruce[d_b, m_b]}) def racimos(ev): gap = ev.groupby("sesion").minuto.diff().dropna() return {"por_sesion": len(ev) / n_ses, "a_3_min_o_menos": (gap <= 3).mean(), "mediana_min": gap.median()} pd.DataFrame({"bands ±0.5": racimos(bandas), "CUSUM k = 4": racimos(ev)}).T.round(3) ``` ```output por_sesion a_3_min_o_menos mediana_min bands ±0.5 22.514 0.149 11.0 CUSUM k = 4 26.518 0.031 13.0 ``` The bands give 22.5 events per session, almost as many as the CUSUM, but 14.9% arrive 3 minutes or less after the previous one, against 3.1% for the CUSUM. When the price oscillates around the edge of the band, the rule fires every time it crosses it: clustered, nearly identical events that overlap labels and repeat features. The CUSUM, by resetting, requires a full move of $h$ since the last event, and that keeps them apart. ## The side of the event carries information A side effect of event-based sampling is that the sequence of sides already says something about the regime: ```python misma = (ev.sesion == ev.sesion.shift()).to_numpy() igual = (ev.lado == ev.lado.shift()).to_numpy()[misma] t_ev = tend[ev.sesion, ev.minuto][misma] print(f"same side as the previous event: {igual.mean():.3f}") print(f" in trend: {igual[t_ev].mean():.3f}") print(f" in reversion: {igual[~t_ev].mean():.3f}") ``` ```output same side as the previous event: 0.518 in trend: 0.624 in reversion: 0.451 ``` Overall, 51.8% of the events repeat the side of the previous one in the same session. In trend, 62.4%; in reversion, 45.1%. It is the signature visible in the header figure, and it is information known at the time of each event: a candidate feature for the meta-model (exercise 2 of lesson 5.3.3, where it turns out to be redundant with the flow imbalance). ## The same filter on the trade tape Nothing in the filter requires 1-minute bars: it works with any series of returns. On NOVA's regular trades of June 2, 2025 (level 1 synthetic data, lesson 1.1.1), trade by trade and with a fixed threshold of 10 bp: ```python op = pd.read_parquet("nova_operaciones.parquet") op = op[op["condicion"] == "regular"].reset_index(drop=True) r_op = np.diff(np.log(op["precio"].to_numpy()))[None, :] ev_op = cusum(r_op, 0.001) # 10 bp, fixed threshold hora_ny = op["ts"].iloc[ev_op.minuto].dt.tz_convert( "America/New_York") print(len(op), "trades,", len(ev_op), "events") franja = hora_ny.dt.floor("30min").dt.strftime("%H:%M") franja.value_counts().sort_index().to_frame("eventos").T ``` ```output 50938 trades, 165 events ts 09:30 10:00 10:30 11:00 11:30 12:00 12:30 13:00 13:30 14:00 14:30 15:00 15:30 eventos 23 18 10 10 6 13 6 8 16 9 9 13 24 ``` From 50,938 trades come 165 events, with 23 in the first half hour, 6 in the 11:30 one and 24 in the last. It is the activity U that the volume and dollar bars drew in lesson 1.4.5, reached by another route: there the bar closed on accumulating volume; here, on accumulating movement. With trade data there is a nuance: the bounce between bid and ask adds and subtracts returns without the price really moving. The CUSUM tolerates it better than a crossing rule because the bounce cancels out inside $S^+$ and $S^-$, but with thresholds on the order of the bid-ask spread it would fire on microstructure noise. ## Common mistakes - **Threshold computed with the future.** A volatility of the full session, or a centered EWMA, makes the filter fire “knowing” how much the day is going to move. - **Choosing the threshold by the hit rate on the test sample.** It is one more parameter of the search (2.5.4). - **Entering at the price that fires the event.** The event is known at the close of the minute; in live trading, the order is executed afterward (on FNX, the open of the next minute coincides with that close except for rounding to the tick, but in an asset with a bid-ask spread or with longer bars it does not). - **Forgetting the daily reset.** If the sums carry over from one session to the next, the overnight gap fires events at the open that cannot be traded at the closing price. ## Exercises 1. Compare the primary model's hit rate on the events of the first half hour with that of the rest of the session. How does it relate to the EWMA lag? **Solution.** ```python a = acierto(ev) apertura = (ev.minuto < 30).to_numpy() t_a = tend[ev.sesion, ev.minuto] for nombre, sel in (("opening", apertura), ("rest", ~apertura)): print(f"{nombre:>8}: {sel.sum()} events, hit rate" f" {a[sel].mean():.3f}, in trend {t_a[sel].mean():.3f}") ``` ```output opening: 743 events, hit rate 0.526, in trend 0.396 rest: 5807 events, hit rate 0.505, in trend 0.387 ``` The 743 events of the first half hour have a hit rate of 52.6% and the rest 50.5%, with an almost equal fraction in trend (39.6% against 38.7%). The difference is little more than one standard error (about 1.8 points with 743 events): it cannot be claimed that the open is better. What is certain is that there are many more events (743, 11% of the total in 8% of the time) because the EWMA lags and the threshold is narrow when volatility is highest. If those events are not wanted, the fix is the intraday profile (5.3.2), not raising $k$ for the whole day. 2. Repeat the CUSUM on the NOVA tape with thresholds of 5 and 20 bp. How many events come out and what happens to the U shape? **Solution.** ```python for h in (0.0005, 0.002): e_h = cusum(r_op, h) hny = op["ts"].iloc[e_h.minuto].dt.tz_convert("America/New_York") franja = hny.dt.floor("30min").dt.strftime("%H:%M") n = franja.value_counts().sort_index() print(f"{1e4 * h:.0f} bp: {len(e_h)} events;" f" 9:30 {n.get('09:30', 0)}, 11:30 {n.get('11:30', 0)}," f" 15:30 {n.get('15:30', 0)}") ``` ```output 5 bp: 417 events; 9:30 59, 11:30 19, 15:30 57 20 bp: 70 events; 9:30 9, 11:30 2, 15:30 10 ``` With 5 bp there are 417 events (59 in the first half hour, 19 in the 11:30 one and 57 in the last); with 20 bp, only 70 (9, 2 and 10). The U shape holds with all three thresholds, because it follows market activity and not the threshold. What changes is the resolution: with 20 bp there are slots with two events in the whole session, too few to measure anything, and with 5 bp the threshold gets close to NOVA's bid-ask spread (one cent on a 53 USD stock is about 2 bp) and the bounce between bid and ask starts to weigh. 3. Lesson 4.2.5 used a CUSUM to switch off a strategy when its returns deteriorated. What changes between that use and the one in this lesson, and why is a mean subtracted in that one and not in this one? **Solution.** The recursion is the same: cumulative sums of deviations, clipped at zero, compared with a threshold. The goal changes. In 4.2.5 the CUSUM is a sequential test for a change in mean: each standardized return is compared with a reference value $k$, halfway between the backtest mean and zero, so that the sum only grows steadily when returns fall below what is expected; when it exceeds the threshold, the alarm is raised and the strategy is stopped. The threshold is calibrated with the trade-off between false alarms and delay. Here the filter does not look for a permanent change but for cumulative moves of a certain size in either direction, and after each event it resets and goes on: it is a sampling rule. Only the expected mean of the returns is subtracted, which over one minute is practically zero, because what matters is the move, not the deviation from a target return. The threshold is chosen by the number and type of events wanted, not by a false alarm rate. ## Self-assessment 1. What does the symmetric CUSUM filter do after it fires an event? - A. It keeps accumulating from the value it had - B. It resets both sums, the upward and the downward one, to zero: the next event requires another full move from that point - C. It changes the threshold - D. It waits a fixed number of minutes *Answer B: It resets both sums, the upward and the downward one, to zero: the next event requires another full move from that point.* The reset is what prevents clusters: a band rule fires every time the price crosses the band, and if the price oscillates around it, it fires many times in a row. On FNX, 14.9% of the band events arrive 3 minutes or less after the previous one, against 3.1% for the CUSUM. 2. With a fixed threshold of 16.9 bp, the sessions in the lowest volatility tercile give 19.4 events and those in the highest 36.9. With the 4σ (EWMA) threshold, 26.9 and 26.0. What is the consequence for a model? - A. None - B. The fixed threshold is better because it gives more events - C. The scaled threshold looks into the future - D. With a fixed threshold, turbulent days dominate the training sample; with a scaled threshold, each session contributes a similar number of events and the required move is comparable *Answer D: With a fixed threshold, turbulent days dominate the training sample; with a scaled threshold, each session contributes a similar number of events and the required move is comparable.* A fixed threshold mixes two things: the signal being sought and the volatility level. Scaling with a volatility known at the time makes the sampling comparable between calm and turbulent days. 3. When the threshold goes up from k = 2 to k = 8, the fraction of events in trend rises from 36.4% to 44.1%, but the primary model's total hit rate stays at 50.2-50.7%. Why? - A. Because while the hit rate in trend goes up (from 56.5% to 63.7%), the one in reversion goes down (from 46.5% to 40.4%): a large move in reversion is more likely to be undone - B. Because the threshold has no influence on the hit rate - C. Because there are few events - D. Because of a bug in the triple barrier *Answer A: Because while the hit rate in trend goes up (from 56.5% to 63.7%), the one in reversion goes down (from 46.5% to 40.4%): a large move in reversion is more likely to be undone.* The CUSUM selects large moves. In trend, a large move tends to continue; in mean reversion, the larger the deviation, the stronger the pullback. The two effects offset each other in the aggregate. 4. Sampling every 15 minutes gives 24.7 observations per session, almost as many as the CUSUM (26.5). What difference is measured on FNX? - A. The CUSUM doubles the hit rate - B. Time-based sampling has no look-ahead bias and the CUSUM does - C. Time-based sampling distributes the observations like the minutes (35.3% in trend); the CUSUM tilts them toward trend (38.8%). The total hit rate is almost the same (50.5% against 50.7%) - D. None *Answer C: Time-based sampling distributes the observations like the minutes (35.3% in trend); the CUSUM tilts them toward trend (38.8%). The total hit rate is almost the same (50.5% against 50.7%).* The CUSUM does not make the primary model good: it changes which moments are observed. Its advantage is sampling when the market moves (more at the open, less at midday) and not repeating nearly identical observations. 5. How does the CUSUM of this lesson differ from Page's in lesson 4.2.5? - A. It is the same recursion of cumulative sums clipped at zero, but here it is used to sample (it resets after each event and goes on) and there to detect a change in mean and stop a strategy - B. They are unrelated algorithms - C. Page's has no threshold - D. The one in this lesson only works with prices *Answer A: It is the same recursion of cumulative sums clipped at zero, but here it is used to sample (it resets after each event and goes on) and there to detect a change in mean and stop a strategy.* Page (1954) proposed the CUSUM for quality control: it raises an alarm when the sum of deviations from the expected mean exceeds a threshold. López de Prado uses it on returns with zero expected mean to decide when to sample. ## Key points - Event-based sampling is deciding when to observe: only when the price has accumulated a move of size h since the last event. - The symmetric CUSUM filter accumulates rises and falls separately, clipped at zero, and resets both after each event: it does not fire in clusters when the price oscillates around a level (3.1% of events 3 minutes or less apart, against 14.9% for a band rule). - With a fixed threshold, the number of events follows each day's volatility (19.4 against 36.9 per session); scaled with an EWMA, it stays at about 26.5. - Raising the threshold reduces the events (from 68.6 to 8.6 per session) and concentrates them in trend (from 36% to 44%), without changing the total hit rate: reversion gets worse at the same time. - Compared with sampling every 15 minutes, the CUSUM follows activity (3.0 events in the first half hour, 1.6 in mid-morning) and tilts toward trend; the total hit rate is almost the same. - The filter works with any series: on the NOVA tape it draws the same intraday U as the activity bars of lesson 1.4.5. ## Resources ### Research - [Seasonality in Python. Searching for Intraday Patterns](https://pythonparatrading.com/en/seasonality-intraday-preprocessing/) · Research-0024 · 9 min ### Documentation - [pandas: DataFrame.ewm](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.ewm.html) · pandas.pydata.org ### Books - [Marcos López de Prado: Advances in Financial Machine Learning (Wiley, 2018), chapter 2 (2.5.2, “Event-Based Sampling”)](https://www.wiley.com/en-us/Advances+in+Financial+Machine+Learning-p-9781119482086) · wiley.com - [E. S. Page, “Continuous inspection schemes”, Biometrika 41 (1954), 100-115 (DOI)](https://doi.org/10.1093/biomet/41.1-2.100) · doi.org ### Data - [Level 5 data: FNX 1-minute bars (Parquet)](https://pythonparatrading.com/curso/datos/nivel-5/hf5_barras_1min.parquet) · Download · hf5_barras_1min.parquet - [Level 1 data: NOVA trades of June 2, 2025 (Parquet)](https://pythonparatrading.com/curso/datos/nivel-1/nova_operaciones.parquet) · Download · nova_operaciones.parquet Previous lesson: [5.3.4 · Label uniqueness, overlap weights and the sequential bootstrap](https://pythonparatrading.com/en/python-trading-course/level-5/label-uniqueness-weights-sequential-bootstrap/) Next lesson: [5.4.1 · Why random k-fold fails on financial data](https://pythonparatrading.com/en/python-trading-course/level-5/why-random-k-fold-fails-finance/)