Hypothesis tests, p-values, bootstrap and regression with robust standard errors.

A backtest is a sample

A four-year backtest with a mean return of 15% a year does not say that the strategy earns 15% a year. It says that, in those four particular years, it earned that. Had the market rolled other dice, the mean would be different. The question of this lesson is how different it could have turned out and, in particular, whether a result like that is compatible with the true edge being zero.

Lesson 1.6.3 already used the t-statistic and the bootstrap for calendar effects. Here the method is formalized, its power is measured and the two problems that produce the most false t's in practice are dealt with: observations that overlap and assets that move together.

flowchart TB
  H[Null hypothesis] --> E[Statistic]
  E --> D[Distrib. under H0]
  D --> P[p-value]
  P --> R{p < α?}
  R -- yes --> RE[Reject H0]
  R -- no --> NR[Do not reject]

• Null hypothesis (H0): the expected mean return is zero. It is what you try to refute.
• Statistic: a number computed from the sample that would be small if H0 were true. For the mean, Student's t:

with $\overline{r}$ the mean of the $n$ returns and $s$ their standard deviation.

• Distribution under H0: how the statistic would be distributed if H0 were true. With independent observations and large $n$, a standard normal; with the bootstrap, the one that comes out of resampling the data.
• p-value: the probability, if H0 were true, of obtaining a statistic at least as extreme as the one observed. It is not the probability that H0 is true.

When deciding with a threshold α (5% by convention) you can make two errors:

In trading, the type I error is the expensive one: you put money into a strategy with no edge and pay costs for nothing. The type II error is quieter: you discard a good strategy because the sample was short.

A useful identity connects the t with the Sharpe. If $SR$ is the annual Sharpe and the sample lasts $T$ years,

A Sharpe of 1 over 4 years gives a t of 2; a Sharpe of 0.5 needs 16 years to reach 2.

Four strategies with a known answer

ruido2_estrategias.parquet holds the daily returns of 1,000 fictional strategies over the 1,003 sessions from 2022 to 2025. They are synthetic data: 995 have no edge at all (true Sharpe 0) and five have a planted one, documented in ruido2_verdad.csv. That one of them wins here says nothing about any real market; it serves to know whether a method gets it right. Four are chosen:

• e0518, true Sharpe 1.5: the clearest edge in the set.
• e0258, true Sharpe 0.7: a real but modest edge.
• e0001, no edge: any one of them.
• the best of the 995 with no edge, by its measured Sharpe.

import numpy as np
import pandas as pd
from scipy import stats

r = pd.read_parquet("ruido2_estrategias.parquet").pivot(
    index="fecha", columns="estrategia", values="rendimiento")
verdad = pd.read_csv("ruido2_verdad.csv").set_index("estrategia")
sr = r.mean() / r.std() * np.sqrt(252)
nulas = verdad.index[verdad["sharpe_real"] == 0]
mejor_nula = sr[nulas].idxmax()
elegidas = ["e0518", "e0258", "e0001", mejor_nula]

filas = {}
for e in elegidas:
    t, p = stats.ttest_1samp(r[e], 0)
    filas[e] = {"sharpe_real": verdad.loc[e, "sharpe_real"],
                "sharpe_medido": sr[e],
                "media_anual_%": 100 * 252 * r[e].mean(),
                "t": t, "p": p,
                "sr_x_raiz_anos": sr[e] * np.sqrt(len(r) / 252)}
print(len(r), "sessions;", len(nulas), "strategies with no edge")
pd.DataFrame(filas).T.round(3)

1003 sessions; 995 strategies with no edge
       sharpe_real  sharpe_medido  media_anual_%      t      p  sr_x_raiz_anos
e0518          1.5          1.579         33.165  3.150  0.002           3.150
e0258          0.7          0.622          8.492  1.240  0.215           1.240
e0001          0.0          0.003          0.063  0.007  0.995           0.007
e0598          0.0          1.503         26.607  2.999  0.003           2.999

The last column checks the identity: the t from scipy.stats.ttest_1samp matches the measured Sharpe times the square root of the years (3.98 years). The table already shows the two errors:

• e0518 (true Sharpe 1.5) comes out with t = 3.15 and p = 0.002: the test gets it right.
• e0258 has a real edge, but its measured Sharpe is 0.62 and its p, 0.215: with four years it cannot be told apart from noise. It is a type II error, and it is not exceptional bad luck: with a true Sharpe of 0.7 and four years, the expected t is 1.4.
• The best of the nulls, e0598, has a measured Sharpe of 1.50 and p = 0.003: more “significant” than one of the good ones. It is the maximum of 995 tests; the p-value of a test is only valid for a hypothesis chosen before looking. Lesson 2.1.8 deals with that selection; here it is enough to remember that no method in this lesson corrects it.

The bootstrap: the distribution without formulas

The t-test assumes that the mean is normally distributed. With fat-tailed returns (lesson 1.6.2) and a thousand observations, the approximation is usually good, but you do not always have a formula: for the Sharpe, the drawdown or the median you have to work harder. The bootstrap avoids the formula: it resamples the data with replacement thousands of times, recomputes the statistic on each resample and uses the dispersion of those values as its error.

flowchart LR
  M[Sample of n days] --> R[Resample B times]
  R --> E[Statistic ×B]
  E --> I[CI percentiles]

The i.i.d. bootstrap draws individual days, so it destroys any dependence between consecutive days. The block bootstrap draws stretches of consecutive days and keeps the dependence within each stretch. The arch library has both: IIDBootstrap and StationaryBootstrap (Politis and Romano, 1994), whose blocks have a random length with a given mean.

from arch.bootstrap import IIDBootstrap, StationaryBootstrap

def media_anual(x):
    return 252 * x.mean()

def intervalos(x, bloque=20, reps=2000, semilla=7):
    x = np.asarray(x)
    m, se = media_anual(x), 252 * x.std(ddof=1) / np.sqrt(len(x))
    q = stats.t.ppf(0.975, len(x) - 1)
    out = {"t": (m - q * se, m + q * se)}
    for nombre, bs in [("iid", IIDBootstrap(x, seed=semilla)),
                       ("bloques", StationaryBootstrap(
                           bloque, x, seed=semilla))]:
        ic = bs.conf_int(media_anual, reps=reps, method="percentile")
        out[nombre] = tuple(ic.ravel())
    return out

ic = {(e, k): 100 * np.array(v) for e in elegidas
      for k, v in intervalos(r[e]).items()}
pd.DataFrame(ic, index=["inf_%", "sup_%"]).T.round(1)

               inf_%  sup_%
e0518 t         12.5   53.8
      iid       13.5   55.0
      bloques    9.0   55.5
e0258 t         -4.9   21.9
      iid       -4.7   22.1
      bloques   -5.0   22.2
e0001 t        -18.8   19.0
      iid      -18.7   20.2
      bloques  -19.1   19.3
e0598 t          9.2   44.0
      iid        9.0   43.7
      bloques    6.8   46.0

The three methods give almost the same interval for the four strategies; the block one comes out somewhat wider at the bottom for e0518 and e0598 (9.0 versus 12.5 for the first), because of resampling randomness and because of volatility clustering, which the blocks preserve. It is no coincidence that they agree: these daily returns have no autocorrelation (only volatility clustering), so resampling individual days or blocks of 20 gives the same thing, and with 1,003 days the mean is already distributed almost like a normal. It is the usual situation with daily returns of a strategy that does not hold positions for many days: the t-test is enough.

The practical lesson is in the width. The interval for e0518 runs from about 12% to 55% a year: the edge is real, but four years do not say whether it is large or small. The one for e0258, from −6% to +22%, includes zero even though the edge exists.

The bootstrap also gives a p-value. To test H0, the sample is centered at zero (its mean is subtracted, so the null is true in the resamples) and you count what fraction of the resampled means is as extreme as the observed one:

x = r["e0258"].to_numpy()
bs = StationaryBootstrap(20, x - x.mean(), seed=7)
medias = np.array([media_anual(d[0][0])
                   for d in bs.bootstrap(4000)])
p_bs = np.mean(np.abs(medias) >= abs(media_anual(x)))
print(f"observed annual mean: {100 * media_anual(x):.2f}%")
print(f"bootstrap p: {p_bs:.3f}  (t-test p: "
      f"{stats.ttest_1samp(x, 0).pvalue:.3f})")

observed annual mean: 8.49%
bootstrap p: 0.232  (t-test p: 0.215)

The two p-values are similar (0.232 versus 0.215) and lead to the same conclusion. On daily returns without autocorrelation, the bootstrap confirms the t-test; its value shows up when the formula does not exist or the dependence does, as in the sections that follow.

Power: how much history you need

The figure at the top answers the inverse question: if the edge exists, with what probability does a test detect it? 4,000 strategies of 1,003 days are simulated with Student's t innovations (fat tails) and four true Sharpes. It is a seeded simulation, not the course data: that way the answer is known for each one.

rng = np.random.default_rng(211)
n, sims = 1003, 4000
filas = {}
for sr_real in [0, 0.5, 1.0, 1.5]:
    z = rng.standard_t(5, (sims, n)) * np.sqrt(3 / 5)  # variance 1
    x = sr_real / np.sqrt(252) + z
    t = x.mean(1) / (x.std(1, ddof=1) / np.sqrt(n))
    p = 2 * stats.t.sf(np.abs(t), n - 1)
    filas[sr_real] = {
        "rechaza_5%": np.mean(p < 0.05),
        "teorica": stats.norm.cdf(sr_real * np.sqrt(n / 252) - 1.96),
        "p_mediano": np.median(p)}
pd.DataFrame(filas).T.round(3)

     rechaza_5%  teorica  p_mediano
0.0       0.056    0.025      0.498
0.5       0.166    0.168      0.294
1.0       0.528    0.514      0.042
1.5       0.850    0.849      0.003

With Sharpe 0 (the null), 5.6% of the simulations reject at 5%: the nominal size, with the noise of 4,000 replications. The p-values are uniform: each 0.05 bin receives 5% of them, which is what you see in the left panel of the figure. With Sharpe 1, the power is 52.8%: half of the time a Sharpe 1 strategy tested over four years does not pass the threshold. With Sharpe 0.5, only 16.6%. The teorica column (the normal with mean $SR\sqrt{T}$, ignoring rejection on the negative side) is almost exactly right.

Solving for $T$ in the condition “expected t = 1.96 + 0.84” (the 0.84 gives 80% power) yields the rule of thumb:

for s in [0.25, 0.5, 1.0, 1.5, 2.0]:
    anos = ((stats.norm.ppf(0.975) + stats.norm.ppf(0.8)) / s) ** 2
    print(f"Sharpe {s:4.2f}: {anos:6.1f} years")

Sharpe 0.25:  125.6 years
Sharpe 0.50:   31.4 years
Sharpe 1.00:    7.8 years
Sharpe 1.50:    3.5 years
Sharpe 2.00:    2.0 years

With a true Sharpe of 0.5, already a good figure for a real strategy after costs, you would need 31 years of history to detect it with 80% probability. That is why short backtests with a high Sharpe should be read with suspicion (the measured Sharpe of the lucky ones is inflated, lesson 2.5.4) and why low-Sharpe strategies are validated with many assets at once rather than with a long history of only one.

Overlapping returns: the t that multiplies

Many signals predict a long horizon: “the return over the next 21 sessions”. If that target is computed every day, two consecutive observations share 20 of their 21 sessions. The mean is still correct; the standard error of the classic formula is not. With overlap $h$, the autocovariances are positive up to lag $h-1$ and the true variance of the mean is about $h$ times what $s^2/n$ assumes: the naive t comes out inflated by about $\sqrt{h}$.

The Newey-West estimator (1987) corrects the variance by adding the autocovariances $\hat\gamma_j$ up to a lag $L$, with decreasing weights that guarantee the result is positive:

In statsmodels it is a regression on a constant with cov_type="HAC". First, by hand and with the library, on a strategy with no edge:

import statsmodels.api as sm

def nw_var_media(y, L):
    y = np.asarray(y) - np.mean(y)
    n = len(y)
    g = [y[j:] @ y[:n - j] / n for j in range(L + 1)]
    lrv = g[0] + 2 * sum((1 - j / (L + 1)) * g[j]
                         for j in range(1, L + 1))
    return lrv / n

h = 21
lr = np.log1p(r["e0001"]).cumsum()
y = (lr.shift(-h) - lr).dropna()       # 21 sessions, every day
ols = sm.OLS(y.to_numpy(), np.ones(len(y))).fit()
hac = sm.OLS(y.to_numpy(), np.ones(len(y))).fit(
    cov_type="HAC", cov_kwds={"maxlags": h})
print(f"autocorrelation at lag 1: {y.autocorr(1):.3f}")
print(f"naive t: {ols.tvalues[0]:.2f}; "
      f"Newey-West t: {hac.tvalues[0]:.2f}")
print(f"standard error by hand: {np.sqrt(nw_var_media(y, h)):.6f}; "
      f"statsmodels: {hac.bse[0]:.6f}")

autocorrelation at lag 1: 0.959
naive t: -0.97; Newey-West t: -0.25
standard error by hand: 0.007131; statsmodels: 0.007131

The autocorrelation at lag 1 is 0.96, close to the theoretical 20/21. The naive t is about four times the Newey-West one, and the standard error computed by hand matches the statsmodels one (which uses the same Bartlett weights by default). In this strategy both t's are far from 2, but the important question is what happens across all of them.

The 995 strategies with no edge make it possible to measure the false positive rate of each method on data whose truth is known:

lr = np.log1p(r[nulas]).cumsum()
Y = (lr.shift(-h) - lr).dropna()

def p_media(y, L=None):
    m = sm.OLS(y, np.ones(len(y)))
    f = m.fit() if L is None else m.fit(
        cov_type="HAC", cov_kwds={"maxlags": L})
    return f.pvalues[0]

tasas = {
    "daily, t": np.mean([stats.ttest_1samp(r[e], 0).pvalue < 0.05
                         for e in nulas]),
    "overlapping, OLS": np.mean([p_media(Y[e].to_numpy()) < 0.05
                                 for e in nulas])}
for L in [10, 21, 42]:
    tasas[f"overlapping, NW L={L}"] = np.mean(
        [p_media(Y[e].to_numpy(), L) < 0.05 for e in nulas])
tasas["no overlap, t"] = np.mean(
    [stats.ttest_1samp(Y[e].iloc[::h], 0).pvalue < 0.05
     for e in nulas])
pd.Series(tasas, name="falsos_positivos").round(3)

daily, t                0.044
overlapping, OLS        0.653
overlapping, NW L=10    0.183
overlapping, NW L=21    0.094
overlapping, NW L=42    0.068
no overlap, t           0.039
Name: falsos_positivos, dtype: float64

With daily returns, the t-test gives 4.4% false positives: the nominal level. With the same strategies and a target overlapping over 21 sessions, the regression with classic errors declares 65.3% of them significant: two out of three strategies with no edge “predict” their own return. Newey-West corrects it almost completely, and the choice of $L$ matters: with 10 lags (fewer than the overlap) 18.3% remains; with 21, 9.4%; with 42, 6.8%. Sampling one observation every 21 sessions (no overlap) brings it back to 3.9%, at the cost of being left with 47 observations.

The bootstrap also fails with overlap if the i.i.d. one is used, and it works with blocks longer than the overlap:

y = Y["e0001"].to_numpy()
ee = {"classic formula": y.std(ddof=1) / np.sqrt(len(y)),
      "Newey-West L=42": np.sqrt(nw_var_media(y, 42))}
for nombre, bs in [
        ("iid bootstrap", IIDBootstrap(y, seed=3)),
        ("blocks mean 63", StationaryBootstrap(63, y, seed=3))]:
    m = [d[0][0].mean() for d in bs.bootstrap(2000)]
    ee[nombre] = np.std(m)
pd.Series(ee, name="error_tipico_media").round(5)

classic formula    0.00186
Newey-West L=42    0.00810
iid bootstrap      0.00191
blocks mean 63     0.00792
Name: error_tipico_media, dtype: float64

The i.i.d. bootstrap reproduces the error of the classic formula (0.0019) because, by drawing individual days, it breaks the overlap; the block one, with blocks of 63 sessions on average, gives an error on the same scale as Newey-West, about four times larger. The rule is the same for both methods: the window (lags or block) has to cover the dependence.

Multiple regression with robust errors: carry and momentum in futures

The most frequent case is a predictive regression: the future return of an asset on one or more of today's signals. futuros2_continuos.parquet and futuros2_carry.parquet hold 24 synthetic futures markets from 2000 to 2025 with two planted effects (level 2 README): the expected annual return is $0.5\,c_{\text{exogeno}} + 0.1\,c_{\text{reversion}} + 0.15\,\sigma\,\operatorname{sign}(r_{252})$, with $\sigma$ the long-run volatility of each market and the last term a time-series momentum. The observed carry, carry_anual, is the sum of the two parts: an exogenous one, with a premium, and another that only discounts the return of the price to its anchor, with almost no premium. The natural regression is

with $y$ the annualized return of the next 21 sessions and $c$ the observed carry. The known answer is $b_2 \approx 0.15$; $b_1$ is a mix of 0.5 and 0.1 depending on which part of the carry dominates (the README measures 0.15 with month-end data). $\sigma$ is estimated with all returns up to each date (expanding window): it is the long-run volatility without looking ahead. The observations are daily, so there is an overlap of 21; in addition, the 24 markets share days and factors. The Driscoll-Kraay estimator (1998) handles both: for each date it sums the regressor × residual products of all markets and applies Newey-West to that series. In statsmodels it is cov_type="hac-groupsum" with the date index.

cont = pd.read_parquet("futuros2_continuos.parquet")
carry = pd.read_parquet("futuros2_carry.parquet")
lp = np.log(cont.pivot(index="fecha", columns="mercado",
                       values="cierre_ajustado"))
c = carry.pivot(index="fecha", columns="mercado",
                values="carry_anual")
fwd = (lp.shift(-h) - lp) * 252 / h            # annualized
# long-run σ of each market, without looking ahead
vol = lp.diff().expanding(252).std() * np.sqrt(252)
mom = np.sign(lp - lp.shift(252)) * vol
panel = pd.concat({"y": fwd, "carry": c, "mom": mom}, axis=1)
panel = panel.stack(future_stack=True).dropna()
X = sm.add_constant(panel[["carry", "mom"]])
fecha = panel.index.get_level_values("fecha").factorize()[0]

ols = sm.OLS(panel["y"], X).fit()
dk = sm.OLS(panel["y"], X).fit(
    cov_type="hac-groupsum", cov_kwds={"time": fecha, "maxlags": h})
print(len(panel), "rows,", panel.index.get_level_values(
    "mercado").nunique(), "markets")
pd.DataFrame({"coef": ols.params, "t_OLS": ols.tvalues,
              "t_DK": dk.tvalues}).round(3)

150384 rows, 24 markets
        coef   t_OLS   t_DK
const  0.013   5.844  0.808
carry  0.171   8.683  1.596
mom    0.107  10.511  1.834

The coefficients recover the answer with error: 0.107 for momentum (truth 0.15) and 0.171 for carry, within the expected mix and close to the README's 0.15. What changes completely is the confidence. With classic errors, the t's are 8.68 and 10.51: apparent certainty. With Driscoll-Kraay, 1.60 and 1.83: neither passes the threshold, even though both effects exist. There are 150,384 rows, but the information is much smaller: about 300 independent monthly returns per market, and markets that move together in crises. That the honest test does not reject does not say there is no effect (here you know there is one): it says that 26 years of 24 correlated markets are not enough to prove it with a t of 2. It is the power of the previous section, now in a panel.

The check without sophisticated formulas is to remove the overlap: one date out of every 21 and errors clustered by date (cov_type="cluster"), which correct the correlation between markets on the same day.

fechas = np.sort(panel.index.get_level_values("fecha").unique())
mensual = panel[panel.index.get_level_values("fecha").isin(
    fechas[::h])]
Xm = sm.add_constant(mensual[["carry", "mom"]])
grupos = mensual.index.get_level_values("fecha").factorize()[0]
om = sm.OLS(mensual["y"], Xm).fit()
cl = sm.OLS(mensual["y"], Xm).fit(
    cov_type="cluster", cov_kwds={"groups": grupos})
print(len(mensual), "rows on", len(fechas[::h]), "dates")
pd.DataFrame({"coef": om.params, "t_OLS": om.tvalues,
              "t_cluster": cl.tvalues}).round(3)

7176 rows on 299 dates
        coef  t_OLS  t_cluster
const  0.011  1.100      0.588
carry  0.209  2.288      1.536
mom    0.053  1.126      0.668

Without overlap, the classic t of carry drops from 8.68 to 2.29 and that of momentum, from 10.51 to 1.13; clustering by date, to 1.54 and 0.67. The coefficients move more than with Driscoll-Kraay (0.209 and 0.053: keeping one date out of every 21 throws away information and leaves a noisier estimate), but the conclusion is the same: the honest uncertainty is several times the naive one.

What if the signal were useless? A persistent placebo regressor, with no relation at all to prices, is the acid test. 20 AR(1) placebos are simulated with the same persistence as the generator's carry (0.997 daily), each one independent per market, and you count how many times it comes out “significant”:

rng = np.random.default_rng(2111)
rech = {"OLS": 0, "Driscoll-Kraay": 0, "DK L=63": 0}
for _ in range(20):
    e = rng.standard_normal(lp.shape) * np.sqrt(1 - 0.997**2)
    pl = np.empty(lp.shape)
    pl[0] = rng.standard_normal(lp.shape[1])
    for i in range(1, len(pl)):
        pl[i] = 0.997 * pl[i - 1] + e[i]
    pl = pd.DataFrame(pl, index=lp.index, columns=lp.columns)
    d = pd.concat({"y": fwd, "carry": c, "mom": mom, "pl": pl},
                  axis=1).stack(future_stack=True).dropna()
    Xp = sm.add_constant(d[["carry", "mom", "pl"]])
    tt = d.index.get_level_values("fecha").factorize()[0]
    m = sm.OLS(d["y"], Xp)
    rech["OLS"] += m.fit().pvalues["pl"] < 0.05
    for k, L in [("Driscoll-Kraay", h), ("DK L=63", 63)]:
        f = m.fit(cov_type="hac-groupsum",
                  cov_kwds={"time": tt, "maxlags": L})
        rech[k] += f.pvalues["pl"] < 0.05
print({k: f"{v} of 20" for k, v in rech.items()})

{'OLS': '14 of 20', 'Driscoll-Kraay': '1 of 20', 'DK L=63': '1 of 20'}

With classic errors, the placebo is “significant” in 14 of 20 simulations. With Driscoll-Kraay, in 1 with 21 lags and in 1 with 63. Twenty replications are few to measure a rate precisely, but the order of magnitude is clear: with persistent signals and overlapping targets, classic errors are not an optimistic approximation, they are a discovery generator.

Which standard error to use

flowchart TB
  Q1{Dependence?} -- no --> Q2{Several assets?}
  Q1 -- yes --> Q3{Several assets?}
  Q2 -- no --> A[Classic t or iid]
  Q2 -- yes --> B[Cluster by date]
  Q3 -- no --> C[Newey-West L=2h]
  Q3 -- yes --> D[Driscoll-Kraay]

Three warnings that no row of the table resolves:

• Selection. If a hundred variants were tried and the best one is reported, all these p-values are too low (lessons 2.1.8 and 2.5.4).
• Costs. A significant coefficient on gross returns may not survive the bid-ask spread and commissions (lesson 2.4.7). Statistical significance is not profitability.
• Stability. A t of 3 over 20 years may come from a single crisis. Look at the coefficient by subperiods before trusting it (lesson 2.5.6).

Exercises

1. Repeat the false positive table for the 995 null strategies with a target overlapping over 5 sessions and over 63 sessions, with classic errors and with Newey-West with $L = h$. How does the problem grow with the overlap?

for hh in [5, 63]:
    lr = np.log1p(r[nulas]).cumsum()
    Yh = (lr.shift(-hh) - lr).dropna()
    ols_fp = np.mean([p_media(Yh[e].to_numpy()) < 0.05
                      for e in nulas])
    nw_fp = np.mean([p_media(Yh[e].to_numpy(), hh) < 0.05
                     for e in nulas])
    print(f"h = {hh:2d}: OLS {ols_fp:.3f}; NW L=h {nw_fp:.3f}")

h =  5: OLS 0.364; NW L=h 0.076
h = 63: OLS 0.805; NW L=h 0.138

2. Compute the 95% interval of the annual Sharpe of e0518 and e0258 with a stationary block bootstrap with a mean block length of 20. Does it include the true Sharpe? Does it include zero?

def sharpe(x):
    return np.sqrt(252) * x.mean() / x.std()

for e in ["e0518", "e0258"]:
    bs = StationaryBootstrap(20, r[e].to_numpy(), seed=11)
    inf, sup = bs.conf_int(sharpe, reps=2000,
                           method="percentile").ravel()
    print(f"{e}: measured Sharpe {sharpe(r[e].to_numpy()):.2f}, "
          f"95% CI [{inf:.2f}, {sup:.2f}], "
          f"true {verdad.loc[e, 'sharpe_real']}")

e0518: measured Sharpe 1.58, 95% CI [0.43, 2.77], true 1.5
e0258: measured Sharpe 0.62, 95% CI [-0.43, 1.69], true 0.7

3. A colleague presents a regression of the 12-month return of 50 stocks on a value signal, with monthly data over 15 years (9,000 rows), and a t of 6 with classic errors. What two problems does the t have and how would you recompute it?