Optimization with CVaR, drawdown, turnover and costs.

What variance does not see

Variance measures dispersion around the mean: a 3% gain increases it as much as a 3% loss, and it does not care whether losses arrive spread out or in a row. For someone managing money both things matter: what hurts is the tail of losses and the cumulative drawdown from the last peak. For a normal distribution it makes no difference, because variance summarizes everything; for a strategy that sells insurance, it does.

The data in this lesson are the daily returns in excess of the risk-free rate of nine synthetic strategies (estrategias4_diarias.parquet), similar to those of levels 2 and 3, with crises, volatility clustering and dated events that imitate real episodes: the quant crash of August 2007, the carry collapse of October 2008, the momentum crash of 2009, the Volmageddon of February 2018 and March 2020. They are synthetic: the numbers serve to compare methods, not as evidence that a strategy works. The tenth column, degradada, is left out (it is for lesson 4.2.5).

import numpy as np
import pandas as pd

est = pd.read_parquet("estrategias4_diarias.parquet")
est = est.set_index("fecha").drop(columns="degradada")
R = est.values                       # simple excess returns
T, n = R.shape

def cvar_hist(x, beta):
    """Average loss on the worst 1 − beta fraction of days."""
    perdida = -np.asarray(x)
    var = np.quantile(perdida, beta)
    return perdida[perdida >= var].mean()

tabla = pd.DataFrame({
    "vol_%": 100 * np.sqrt(252) * est.std(),
    "asimetria": est.skew(),
    "peor_dia_%": 100 * est.min(),
    "cvar95_%": [100 * cvar_hist(est[c], 0.95) for c in est],
    "cvar99_%": [100 * cvar_hist(est[c], 0.99) for c in est],
})
print(T, "sessions,", n, "strategies")
tabla.round(2)

5031 sessions, 9 strategies
                   vol_%  asimetria  peor_dia_%  cvar95_%  cvar99_%
tendencia_futuros  10.73       0.08       -4.94      1.52      2.53
carry_futuros       7.54      -0.18       -3.19      1.09      1.77
carry_divisas       9.09      -0.20       -4.59      1.31      2.18
venta_puts         11.90      -0.35       -6.56      1.73      2.93
pares               5.96      -0.46       -3.90      0.87      1.47
momentum_acciones  12.19      -0.56       -8.04      1.77      3.10
valor_acciones      9.04       0.08       -3.18      1.28      1.95
venta_volatilidad  16.37      -2.76      -19.29      2.45      4.71
reversion_corto     7.07      -0.03       -4.01      1.02      1.75

Volatility selling has the highest volatility (16.4%), a skewness of −2.76 and a worst day of −19.3%: it is the tail strategy. Its 99% CVaR (4.71%) is 50% higher than the next one, that of equity momentum (3.10%). Put selling and equity momentum have more moderate tails, but long ones too.

VaR and CVaR

Given a confidence level β (0.95 or 0.99) and the daily loss $L = -r$:

• The VaR at level β is the β quantile of the losses: the loss that is exceeded only on $(1-\beta)$ of the days.
• The CVaR (conditional value at risk, also expected shortfall) is the average loss when the VaR is exceeded: the mean of the worst $(1-\beta)$ of the days.

VaR says where the tail starts and nothing about what is inside it; CVaR averages it. CVaR is also a coherent measure in the sense of Artzner, Delbaen, Eber and Heath (1999): among other properties, it is subadditive (the risk of a portfolio does not exceed the sum of the risks) and convex in the weights. VaR is not subadditive in general, and minimizing it is a non-convex problem with many local minima. How to estimate VaR and CVaR, and how to check whether a VaR model is right, are lesson 4.2.3; here they matter as optimization objectives.

The practical problem with CVaR is that it depends on a quantile, and quantiles are not convenient functions for an optimizer. Rockafellar and Uryasev (2000) solved the problem with an auxiliary function:

For fixed weights, $F_\beta$ is convex in α, its minimum is reached at the VaR and its minimum value is the CVaR. Best of all: minimizing $F_\beta$ at the same time over $w$ and α gives the minimum CVaR portfolio, without computing any quantile. Before optimizing, the check with the 1/N portfolio:

from scipy.optimize import minimize_scalar

w_1n = np.full(n, 1 / n)
p = R @ w_1n                          # return of the 1/N portfolio

def F(alfa, beta=0.95):
    """Rockafellar-Uryasev function: its minimum over alfa is the CVaR."""
    return alfa + np.maximum(-p - alfa, 0).mean() / (1 - beta)

res = minimize_scalar(F, bounds=(0, 0.05), method="bounded",
                      options={"xatol": 1e-10})
print(f"VaR 95% (quantile):     {np.quantile(-p, 0.95):.5f}")
print(f"optimal alpha:          {res.x:.5f}")
print(f"direct CVaR 95%:        {cvar_hist(p, 0.95):.5f}")
print(f"minimum of F:           {res.fun:.5f}")

VaR 95% (quantile):     0.00429
optimal alpha:          0.00430
direct CVaR 95%:        0.00622
minimum of F:           0.00622

The α that minimizes $F$ (0.00430) matches the 95% quantile of the losses (0.00429; the difference is the discretization of the quantile between two observations) and the minimum of $F$, 0.00622, matches the CVaR computed by hand. The figure shows why: to the left of the VaR, raising α subtracts more than it adds; to the right, the function grows with slope 1.

Minimum CVaR as a linear program

The $\max(\cdot, 0)$ term is linearized with one auxiliary variable per scenario, $u_t \ge 0$ and $u_t \ge -r_t'w - \alpha$. The result is a linear program with $T + n + 1$ variables:

flowchart TB
  E[Scenarios r_t] --> U[Excess u_t]
  A[Threshold alpha] --> U
  W[Weights w] --> U
  U --> O[alpha + mean u_t]
  O --> S[LP solver]

Each day of the sample is a scenario. With 5,031 days that means 5,041 variables and about 10,000 constraints: for a linear solver such as HiGHS, a small problem. The function cvar_cp returns the expression and its constraints so it can be used as an objective or as a constraint:

import cvxpy as cp
import riskfolio as rp
from skfolio import RiskMeasure
from skfolio.optimization import MeanRisk, ObjectiveFunction

def cvar_cp(w, R, beta):
    """CVaR as an LP: returns the expression and its constraints."""
    alfa = cp.Variable()
    u = cp.Variable(R.shape[0])
    expr = alfa + cp.sum(u) / ((1 - beta) * R.shape[0])
    return expr, [u >= -R @ w - alfa, u >= 0]

def min_cvar(R, beta=0.95):
    w = cp.Variable(R.shape[1])
    expr, extra = cvar_cp(w, R, beta)
    cp.Problem(cp.Minimize(expr),
               [cp.sum(w) == 1, w >= 0] + extra).solve(solver="HIGHS")
    return np.clip(w.value, 0, None)

propia = min_cvar(R)

skf = MeanRisk(objective_function=ObjectiveFunction.MINIMIZE_RISK,
               risk_measure=RiskMeasure.CVAR, cvar_beta=0.95).fit(est)
port = rp.Portfolio(returns=est, alpha=0.05)
port.assets_stats(method_mu="hist", method_cov="hist")
rkf = port.optimization(model="Classic", rm="CVaR", obj="MinRisk",
                        hist=True)

print(f"CVaR 95%:  {100 * cvar_hist(R @ propia, 0.95):.4f} %")
print(f"skfolio:   {np.abs(skf.weights_ - propia).max():.1e}")
print(f"riskfolio: {np.abs(rkf['weights'] - propia).max():.1e}")
pd.Series(propia, est.columns).round(3)

CVaR 95%:  0.4627 %
skfolio:   7.0e-10
riskfolio: 3.7e-10
tendencia_futuros    0.116
carry_futuros        0.151
carry_divisas        0.082
venta_puts           0.022
pares                0.256
momentum_acciones    0.071
valor_acciones       0.145
venta_volatilidad    0.000
reversion_corto      0.157
dtype: float64

The three agree to $10^{-9}$: skfolio (cvar_beta=0.95) and Riskfolio-Lib (alpha=0.05, hist=True) solve the same formulation. The portfolio spreads the weight among pairs, short-term reversal, value, futures carry and trend, and leaves volatility selling at zero. Its 95% CVaR is 0.46% a day.

Does anything change compared with minimum variance?

def min_var(R):
    w = cp.Variable(R.shape[1])
    L = np.linalg.cholesky(np.cov(R.T))
    cp.Problem(cp.Minimize(cp.sum_squares(L.T @ w)),
               [cp.sum(w) == 1, w >= 0]).solve(solver="CLARABEL")
    return np.clip(w.value, 0, None)

def metricas(w, R):
    p = R @ w
    curva = np.cumprod(1 + p)
    return {
        "vol_%": 100 * np.sqrt(252) * p.std(),
        "cvar95_%": 100 * cvar_hist(p, 0.95),
        "cvar99_%": 100 * cvar_hist(p, 0.99),
        "max_caida_%": 100 * (curva / np.maximum.accumulate(curva)
                              - 1).min(),
        "sharpe": np.sqrt(252) * p.mean() / p.std(),
    }

carteras = {"1/N": w_1n, "min_var": min_var(R),
            "min_cvar95": min_cvar(R, 0.95),
            "min_cvar99": min_cvar(R, 0.99)}
pesos = pd.DataFrame(carteras, index=est.columns)
print(pd.DataFrame({k: metricas(w, R) for k, w in carteras.items()})
      .T.round(2))
pesos.round(3)

            vol_%  cvar95_%  cvar99_%  max_caida_%  sharpe
1/N          4.49      0.62      0.96       -14.59    0.84
min_var      3.45      0.47      0.67        -8.94    0.98
min_cvar95   3.46      0.46      0.67        -9.38    0.98
min_cvar99   3.51      0.47      0.65       -10.25    1.01
                     1/N  min_var  min_cvar95  min_cvar99
tendencia_futuros  0.111    0.116       0.116       0.123
carry_futuros      0.111    0.137       0.151       0.115
carry_divisas      0.111    0.068       0.082       0.118
venta_puts         0.111    0.024       0.022       0.039
pares              0.111    0.253       0.256       0.195
momentum_acciones  0.111    0.090       0.071       0.086
valor_acciones     0.111    0.150       0.145       0.142
venta_volatilidad  0.111    0.001       0.000       0.000
reversion_corto    0.111    0.162       0.157       0.183

Very little. Minimum variance and 95% minimum CVaR have almost the same weights and the same metrics; 99% minimum CVaR moves away a little (more reversal, fewer pairs) and lowers the 99% CVaR from 0.67% to 0.65%. The reason is that minimum variance already discards the fat-tailed strategy: volatility selling has so much variance that it weighs zero, or nearly so, under any measure, and the portfolio that remains, of eight weakly correlated strategies, has tails with little asymmetry. When the asymmetry is in assets that variance does not penalize enough, the two measures diverge more; with these data, the practical difference lies elsewhere.

What historical CVaR does not see

The difference appears when return is sought. The experiment: with data from June 2009 to January 2018 (a period without large crises in the data), choose the portfolio with the maximum mean return at 5% volatility (mean-variance) and the one with the maximum return at the same 99% CVaR as that portfolio (mean-CVaR). Both have the same tail risk in the sample. Then measure them from February 2018 to 2025, with the Volmageddon and March 2020 inside.

dentro = est.loc["2009-06":"2018-01"].values
fuera = est.loc["2018-02":].values
mu, Sig = dentro.mean(0), np.cov(dentro.T)

# mean-variance: maximum return at 5% volatility
w = cp.Variable(n)
cp.Problem(cp.Maximize(mu @ w),
           [cp.quad_form(w, Sig) * 252 <= 0.05**2,
            cp.sum(w) == 1, w >= 0]).solve(solver="CLARABEL")
w_mv = np.clip(w.value, 0, None)

# mean-CVaR: maximum return with the same in-sample 99% CVaR
tope = cvar_hist(dentro @ w_mv, 0.99)
w = cp.Variable(n)
expr, extra = cvar_cp(w, dentro, 0.99)
cp.Problem(cp.Maximize(mu @ w),
           [expr <= tope, cp.sum(w) == 1, w >= 0] + extra
           ).solve(solver="HIGHS")
w_cv = np.clip(w.value, 0, None)

print(pd.DataFrame({"media_var": w_mv, "media_cvar": w_cv},
                   index=est.columns).round(2))
filas = {}
for nombre, wt in [("media_var", w_mv), ("media_cvar", w_cv)]:
    for tramo, X in [("dentro", dentro), ("fuera", fuera)]:
        m = metricas(wt, X)
        m["peor_dia_%"] = 100 * (X @ wt).min()
        filas[(nombre, tramo)] = m
pd.DataFrame(filas).T.round(2)

                   media_var  media_cvar
tendencia_futuros       0.05        0.00
carry_futuros           0.00        0.00
carry_divisas           0.00        0.00
venta_puts              0.22        0.32
pares                   0.07        0.00
momentum_acciones       0.24        0.16
valor_acciones          0.14        0.20
venta_volatilidad       0.10        0.07
reversion_corto         0.19        0.26
                   vol_%  cvar95_%  cvar99_%  max_caida_%  sharpe  peor_dia_%
media_var  dentro   5.00      0.68      1.05        -7.04    1.31       -1.68
           fuera    5.15      0.72      1.26       -11.22    0.90       -2.37
media_cvar dentro   5.56      0.74      1.04        -8.47    1.23       -1.57
           fuera    5.54      0.78      1.30       -13.51    0.79       -2.61

In sample, mean-CVaR delivers what it promises: the same 99% CVaR (1.04% versus 1.05%), a less bad worst day (−1.57% versus −1.68%) and somewhat more volatility, which it does not care about. To achieve it, it raises put selling from 22% to 32%. Out of sample, mean-CVaR loses on everything: a 99% CVaR of 1.30%, a maximum drawdown of 13.5% versus 11.2% and a worst day of −2.61%. The worst days tell the story:

fechas_f = est.loc["2018-02":].index
pf = pd.DataFrame({"media_var": fuera @ w_mv,
                   "media_cvar": fuera @ w_cv,
                   "venta_puts": est.loc["2018-02":, "venta_puts"]},
                  index=fechas_f)
sesgo = est.loc["2009-06":"2018-01", "venta_puts"].skew()
print(f"skewness of venta_puts in 2009-2018: {sesgo:.2f}")
(100 * pf.nsmallest(4, "media_cvar")).round(2)

skewness of venta_puts in 2009-2018: 0.32
            media_var  media_cvar  venta_puts
fecha
2020-03-16      -2.37       -2.61       -6.55
2020-03-12      -1.15       -1.95       -5.81
2018-10-03      -1.42       -1.89       -4.90
2018-02-05      -2.15       -1.81       -2.66

March 2020: put selling loses 6.55% on the 16th and 5.81% on the 12th. Mean-CVaR, with more put selling, suffers it more. In the 2009-2018 sample, put selling had had a moderate tail (slightly positive skewness in that stretch), and historical CVaR can only see the losses that happened. The optimizer did what it was asked: it looked for the asset whose measured tail risk was low for its return, which is precisely the one carrying the hidden risk. It is the same “error maximization” phenomenon of lesson 4.1.1, now in the tails.

Drawdown: CDaR and maximum drawdown

The drawdown at $t$ is the distance between the running maximum up to $t$ and the value at $t$. The maximum drawdown is the largest of them; the CDaR (conditional drawdown at risk, Chekhlov, Uryasev and Zabarankin, 2005) is the mean of the worst $(1-\beta)$ of the drawdowns, the equivalent of CVaR over the path.

To make it linear, the uncompounded cumulative sum of the returns is used, $c_t = \sum_{s \le t} r_s'w$, which is linear in $w$. The running maximum is modeled with a ceiling $h_t$: $h_t \ge c_t$, $h_t \ge h_{t-1}$, $h_0 \ge 0$. The drawdown is $h_t - c_t$ (the lowest ceiling compatible with the constraints is exactly the running maximum, and the optimizer has no incentive to raise it). With that, CDaR is linearized with the same Rockafellar-Uryasev trick and the maximum drawdown is the maximum of $h_t - c_t$. With small daily returns, the uncompounded sum and the logarithm of the compounded value are similar; the metrics in the table are computed on the compounded value.

The experiment: optimize with 2006-2015 and measure in 2016-2025.

def caidas_cp(w, R):
    """Drawdowns of the (uncompounded) cumulative sum as linear
    constraints: ceiling ≥ sum and non-decreasing ceiling."""
    c = cp.cumsum(R @ w)
    techo = cp.Variable(len(R))
    restr = [techo >= c, techo[1:] >= techo[:-1], techo[0] >= 0]
    return techo - c, restr

def min_cdar(R, beta=0.95):
    w, alfa, z = cp.Variable(R.shape[1]), cp.Variable(), \
        cp.Variable(len(R))
    dd, restr = caidas_cp(w, R)
    obj = alfa + cp.sum(z) / ((1 - beta) * len(R))
    cp.Problem(cp.Minimize(obj),
               restr + [z >= dd - alfa, z >= 0, cp.sum(w) == 1,
                        w >= 0]).solve(solver="HIGHS")
    return np.clip(w.value, 0, None)

def min_maxdd(R):
    w = cp.Variable(R.shape[1])
    dd, restr = caidas_cp(w, R)
    cp.Problem(cp.Minimize(cp.max(dd)),
               restr + [cp.sum(w) == 1, w >= 0]).solve(solver="HIGHS")
    return np.clip(w.value, 0, None)

ant, post = est.loc[:"2015"].values, est.loc["2016":].values
cart = {"1/N": w_1n, "min_var": min_var(ant),
        "min_cvar99": min_cvar(ant, 0.99), "min_cdar95": min_cdar(ant),
        "min_max_caida": min_maxdd(ant)}
filas = {}
for k, w in cart.items():
    a, b = metricas(w, ant), metricas(w, post)
    filas[k] = {"caida_dentro_%": a["max_caida_%"],
                "caida_fuera_%": b["max_caida_%"],
                "sharpe_dentro": a["sharpe"],
                "sharpe_fuera": b["sharpe"],
                "n_ef": 1 / (w ** 2).sum()}
pd.DataFrame(filas).T.round(2)

               caida_dentro_%  caida_fuera_%  sharpe_dentro  sharpe_fuera  n_ef
1/N                    -14.59          -9.67           0.50          1.18  9.00
min_var                 -8.52          -5.03           0.68          1.30  6.57
min_cvar99              -9.85          -5.19           0.63          1.34  6.29
min_cdar95              -5.70          -9.43           0.89          0.99  5.49
min_max_caida           -5.33          -8.54           0.88          0.97  5.23

The header figure puts these drawdowns side by side. The two portfolios that optimize the drawdown achieve the lowest one in sample (5.70% and 5.33%, versus 8.52% for minimum variance) and a higher Sharpe (0.89 and 0.88). Out of sample, their drawdowns almost double (9.43% and 8.54%), well above minimum variance (5.03%) and 99% minimum CVaR (5.19%), and their Sharpes (0.99 and 0.97) are the two lowest of the five. The 1/N portfolio falls more than any other in sample (14.59%) and 9.67% out of sample, barely more than minimum CDaR.

The explanation is statistical. The maximum drawdown over ten years is decided by two or three episodes. Minimizing it means choosing the combination that best dodged those episodes, and with nine strategies there are combinations that dodge them partly by chance. Variance and CVaR use every day or the whole worst 5% of days: thousands or hundreds of observations against a few drawdown paths. It is the multiple testing problem (lesson 2.1.8) inside an optimizer. Drawdown works as a control constraint (lesson 4.2.5) or as an evaluation metric, with care; as an optimization objective with few years of data, it overfits.

Costs and turnover

Everything above assumes that rebalancing is free. It is not, and in a large portfolio execution costs can eat the return of a low-risk strategy. A reasonable cost model, as a fraction of the portfolio, for moving $\Delta w_i$ of asset $i$ with a portfolio of value $V$:

The first term is the half spread $h_i/2$. The second is square-root impact: the cost per unit traded grows with the square root of the fraction of daily volume traded, $Y\sigma\sqrt{Q/ADV}$, and multiplied by the amount traded it gives an exponent of 3/2 (lessons 2.4.7 and 4.3.1). Both terms are convex, so they fit in cvxpy: cp.abs and cp.power(·, 1.5). riesgo4_liquidez.csv gives, for each asset of the universe as a fictitious ETF, the average daily volume, the bid-ask spread, the daily volatility and the coefficient $Y$.

precios = pd.read_parquet("universo4_precios.parquet")
precios = precios.set_index("fecha")
fact = pd.read_parquet("universo4_factores_diarios.parquet")
fact = fact.set_index("fecha")
r = precios.pct_change()
r.iloc[0] = precios.iloc[0] / 100 - 1
cols = [c for c in precios if c not in ("MONET", "RF_T13")]
r = r[cols]
exceso = r.sub(fact["rf"], axis=0)

liq = pd.read_csv("riesgo4_liquidez.csv", index_col="activo").loc[cols]
CARTERA = 5e9                                   # USD
a = liq["horquilla_pb"].values / 2 / 1e4        # half spread
b = (liq["impacto_y"] * liq["vol_diaria"]
     * np.sqrt(CARTERA / liq["adv_usd"])).values

def coste(dw):
    """Cost as a fraction of the portfolio: spread + square root."""
    dw = np.abs(dw)
    return a @ dw + b @ dw**1.5

ej = ["RV_EEUU", "RF_T37", "MP_ENE", "INFRA"]
pb = [1e4 * coste(np.where(np.array(cols) == c, 0.01, 0)) / 0.01
      for c in ej]
pd.DataFrame({"adv_musd": liq.loc[ej, "adv_usd"] / 1e6,
              "dias_adv": 0.01 * CARTERA / liq.loc[ej, "adv_usd"],
              "coste_pb_de_lo_movido": pb}, index=ej).round(2)

         adv_musd  dias_adv  coste_pb_de_lo_movido
RV_EEUU   7302.01      0.01                   5.94
RF_T37     798.84      0.06                   4.66
MP_ENE      99.76      0.50                  88.17
INFRA       23.48      2.13                  99.61

With USD 5 billion, moving 1% of the portfolio in the US equity ETF is one hundredth of its daily volume and costs 5.94 bp of the amount moved; in the 3-7 year bond, 4.66 bp. In energy it is half a day of volume and 88 bp; in infrastructure, more than two days of volume and almost 100 bp. An optimizer without costs treats a point of each one the same.

There are two ways to put costs into the problem:

• Penalize the cost in the objective: minimize $\text{CVaR}(w) + \kappa \cdot \text{cost}(w - w_0)$, with $w_0$ the current portfolio. κ translates costs into units of risk: with κ = 1, one basis point of cost weighs the same as one basis point of daily CVaR.
• Limit turnover: add $\sum_i |w_i - w_{0,i}| \le \tau$ and leave the objective alone.

flowchart TB
  P[Current portfolio w0] --> M{"Improvement > cost?"}
  M -- yes --> T[Trade]
  M -- no --> N[Do not trade]
  T --> F[New portfolio]
  N --> F

With the cost in the objective, a no-trade zone appears: the $|\Delta w|$ term has a nonzero slope at $\Delta w = 0$, so moving an asset only pays if the marginal risk improvement exceeds its marginal cost, and the zone is wider the more expensive the asset. It is what threshold rebalancing in lesson 4.1.7 does explicitly.

The experiment: 95% minimum CVaR of the 36 risky assets of the universe (without the money market fund or the 1-3 year bill, as in lesson 4.1.4), reoptimized on the first day of each month with the previous year, with 5 billion and four values of κ, plus a turnover cap of 5% a month with no cost in the objective. Costs are charged on the day of the change. The problem is built once per variant with cvxpy parameters (cp.Parameter) for the scenarios and the current portfolio; each month only their values change and cvxpy reuses the compilation. The no_optimos column counts the times the solver did not finish with status optimal.

V = 252
fechas = exceso.index
primeras = pd.Series(fechas, fechas).groupby(
    fechas.to_period("M")).first()
rebal = [d for d in primeras if fechas.get_loc(d) >= V]
m = len(cols)

def problema(kappa, tope=None):
    """Minimum 95% CVaR + kappa · cost of going from w0 to w."""
    Rp, w0 = cp.Parameter((V, m)), cp.Parameter(m)
    w = cp.Variable(m)
    expr, extra = cvar_cp(w, Rp, 0.95)
    dw = w - w0
    c = a @ cp.abs(dw) + b @ cp.power(cp.abs(dw), 1.5)
    restr = [cp.sum(w) == 1, w >= 0] + extra
    if tope is not None:
        restr.append(cp.norm1(dw) <= tope)
    return cp.Problem(cp.Minimize(expr + kappa * c), restr), Rp, w0, w

def resolver(prob, Rp, w0, w, X, previo):
    Rp.value, w0.value = X, previo
    prob.solve(solver="CLARABEL")
    wn = np.clip(w.value, 0, None)
    return wn / wn.sum(), prob.status != "optimal"

inicial = problema(0)                   # the first portfolio, without costs

def rebalanceo(kappa, tope=None):
    prob = problema(kappa, tope)
    previo, trozos, rot, gasto, malos = None, [], [], [], 0
    for j, d in enumerate(rebal):
        i = fechas.get_loc(d)
        X = exceso.iloc[i - V : i].values
        if previo is None:
            previo, _ = resolver(*inicial, X, np.full(m, 1 / m))
        wn, malo = resolver(*prob, X, previo)
        malos += malo
        dw = wn - previo
        rot.append(np.abs(dw).sum())
        gasto.append(coste(dw))
        fin = rebal[j + 1] if j + 1 < len(rebal) else None
        bl = r.loc[d:fin]
        bl = bl.iloc[:-1] if fin is not None else bl
        g = (1 + bl).cumprod().values
        valor = g @ wn
        rr = valor / np.r_[1, valor[:-1]] - 1
        rr[0] -= gasto[-1]               # the cost, on the day of the change
        trozos.append(pd.Series(rr, bl.index))
        previo = g[-1] * wn / valor[-1]
    p = pd.concat(trozos)
    p = p - fact["rf"].loc[p.index]
    return {"exceso_%": 25200 * p.mean(),
            "vol_%": 100 * np.sqrt(252) * p.std(),
            "cvar95_%": 100 * cvar_hist(p, 0.95),
            "rotacion_%": 1200 * np.mean(rot),
            "coste_pb": 12e4 * np.mean(gasto), "no_optimos": malos}

resultado = {f"kappa={k}": rebalanceo(k) for k in (0, 0.3, 1, 2)}
resultado["cap 5 %"] = rebalanceo(0, tope=0.05)
print(rebal[0].date(), "to", fechas[-1].date())
pd.DataFrame(resultado).T.round(2)

2007-02-01 to 2025-12-31
           exceso_%  vol_%  cvar95_%  rotacion_%  coste_pb  no_optimos
kappa=0        0.49   3.41      0.50      164.07     55.16         0.0
kappa=0.3      0.56   3.35      0.49       56.79     11.85         0.0
kappa=1        0.48   3.28      0.49       29.12      3.63         0.0
kappa=2        0.55   3.26      0.48       16.40      1.41         0.0
cap 5 %        0.47   3.35      0.49       58.40     15.48         0.0

Without costs in the objective (κ = 0), the monthly minimum CVaR turns over 164% a year and pays 55 bp a year. The solution of a linear program sits at a vertex: a small change in the scenarios makes it jump to another vertex, even though the CVaR barely changes. With κ = 0.3 turnover drops to 57% and the cost to 11.9 bp; with κ = 1, to 29% and 3.6 bp; with κ = 2, to 16% and 1.4 bp. The out-of-sample CVaR does not get worse: from 0.50% to 0.48%. Nor does volatility (from 3.41% to 3.26%). The solver finishes at optimal in every month of every variant. The net excess return moves between 0.47% and 0.56% with no clear order: the cost difference, of up to 54 bp a year, is the same size as the noise in the return of these portfolios over 19 years, and cannot be distinguished in a backtest. Not seeing it does not mean not paying it: the cost is certain and the return difference is not.

The 5% monthly cap gives a turnover similar to κ = 0.3 (58%) but costs more (15.5 bp): it limits how much moves, not what moves, and spends its budget on the expensive assets if they improve the CVaR. The penalty knows what each asset costs.

Which κ to choose? There is no correct one. The honest approach is to set it before looking at the out-of-sample result, or to choose it with time-series cross-validation (lesson 2.5.2), and not to search for the one that maximizes the backtest Sharpe. Boyd et al. (2017) develop this approach for multiple periods: costs, impact and constraints in a single convex problem that is solved at each rebalance.

Common mistakes

• Optimizing the measure that is going to be shown. The in-sample maximum drawdown and CVaR look good because they were optimized on it; out of sample, the overfitting shows up.
• CVaR with a window without crises. It only sees the tails that occurred. Check which episodes the window contains and complete it with stress.
• Ignoring the solver status. An optimal_inaccurate from cvxpy with a power cone is frequent when the solution is in the no-trade zone ($\Delta w = 0$ is the vertex of the cone). Check prob.status at each rebalance and do not use weights from a problem that did not converge.
• Costs as a turnover cap. A cap that is the same for all assets does not distinguish the liquid ETF from the illiquid one.
• Log returns in the portfolio. Summing weights times returns is only exact with simple returns; with log returns, the portfolio CVaR and Sharpe come out biased.

Exercises

1. Repeat the 99% minimum CVaR of the nine strategies with min_cvar and with skfolio (cvar_beta=0.99), compare the weights and say which strategy gains the most weight when going from 95% to 99%.

w99 = min_cvar(R, 0.99)
skf99 = MeanRisk(objective_function=ObjectiveFunction.MINIMIZE_RISK,
                 risk_measure=RiskMeasure.CVAR,
                 cvar_beta=0.99).fit(est)
dif = np.abs(skf99.weights_ - w99).max()
print(f"difference with skfolio: {dif:.1e}")
(pd.Series(w99 - min_cvar(R, 0.95), est.columns)
 .sort_values().round(3))

difference with skfolio: 4.8e-09
pares               -0.061
carry_futuros       -0.036
valor_acciones      -0.003
venta_volatilidad    0.000
tendencia_futuros    0.006
momentum_acciones    0.015
venta_puts           0.017
reversion_corto      0.025
carry_divisas        0.036
dtype: float64

2. With a portfolio of 500 million instead of 5 billion, how much does it cost to move 1% in each of the four assets in the example? Which term of the cost changes and why does it not change in proportion?

b_500 = (liq["impacto_y"] * liq["vol_diaria"]
         * np.sqrt(5e8 / liq["adv_usd"])).values
def coste_500(dw):
    dw = np.abs(dw)
    return a @ dw + b_500 @ dw**1.5

pd.Series([1e4 * coste_500(np.where(np.array(cols) == c, 0.01, 0))
           / 0.01 for c in ej], index=ej).round(2)

RV_EEUU     2.18
RF_T37      2.15
MP_ENE     30.63
INFRA      39.20
dtype: float64

3. No code: a colleague proposes choosing each month the minimum maximum drawdown portfolio of the last three years, “because it is what the client cares about most”. Explain with the results of the lesson what you would expect and what you would propose instead.