Cost modeling: commissions, bid-ask spread, impact, financing and securities borrowing.

Three families of costs

Lesson 1.1.5 measured the costs of a specific order on NOVA's book. Here the problem is different: putting them into a backtest spanning years, with thousands of orders, so that the net result resembles what a real account would produce. For that, costs have to be separated by the variable they depend on, because each one is modeled differently.

flowchart TB
  C[Total cost] --> E[Explicit]
  C --> I[Implicit]
  C --> M[Holding]
  E --> CO[Commission and fees]
  I --> HS[Half spread]
  I --> IM[Impact]
  M --> FI[Financing and borrow]

The data is from level 2 of the course: synthetic, with made-up fees and rates but with the shape of real ones. The broker names (Alcor, Brio, Cenit) are fictitious.

Commissions: four schemes, four functions

import numpy as np
import pandas as pd

tarifas = pd.read_csv("costes2_brokers.csv")
tarifas.drop(columns=["tipo", "unidad"])

  broker        plan  tramo_desde  tramo_hasta  tarifa  minimo_orden  maximo_pct_efectivo
0  Alcor  por_accion            0          NaN  0.0050          1.00                  1.0
1  Alcor  escalonado            0     300000.0  0.0035          0.35                  1.0
2  Alcor  escalonado       300000    3000000.0  0.0020          0.35                  1.0
3  Alcor  escalonado      3000000   20000000.0  0.0015          0.35                  1.0
4  Alcor  escalonado     20000000          NaN  0.0010          0.35                  1.0
5   Brio        fijo            0          NaN  4.9500           NaN                  NaN
6  Cenit  porcentaje            0          NaN  0.0800          2.00                  NaN

Four schemes found at real brokers:

• Per share with minimum and maximum (Alcor, plan por_accion): 0.005 USD per share, at least 1 USD per order and at most 1% of the traded value.
• Tiered by monthly volume (Alcor, plan escalonado): the per-share rate falls as the volume traded in the month grows; the minimum drops to 0.35 USD.
• Fixed per order (Brio): 4.95 USD per order, whether for 10 shares or 10,000.
• Percentage of traded value (Cenit): 0.08% of the amount, with a minimum of 2 USD.

A single function receives the rows of the plan, the number of shares, the price and the shares already traded in the month (they only count for the tiered plan):

def comision(plan, acciones, precio, volumen_mes=0):
    """Commission in USD of an order according to the rows of a plan."""
    acciones = abs(acciones)
    efectivo = acciones * precio
    fila = plan.iloc[0]
    if fila["tipo"] == "escalonado":
        # tier according to the volume ALREADY traded in the month
        hasta = plan["tramo_hasta"].fillna(np.inf)
        fila = plan[(plan["tramo_desde"] <= volumen_mes)
                    & (volumen_mes < hasta)].iloc[0]
    if fila["tipo"] in ("por_accion", "escalonado"):
        c = fila["tarifa"] * acciones
    elif fila["tipo"] == "fijo_por_orden":
        c = fila["tarifa"]
    elif fila["tipo"] == "porcentaje":
        c = fila["tarifa"] / 100 * efectivo
    else:
        raise ValueError(f"unknown type: {fila['tipo']}")
    if pd.notna(fila["minimo_orden"]):
        c = max(c, fila["minimo_orden"])
    if pd.notna(fila["maximo_pct_efectivo"]):
        c = min(c, fila["maximo_pct_efectivo"] / 100 * efectivo)
    return round(c, 4)


planes = {f"{b}/{p}": g.reset_index(drop=True)
          for (b, p), g in tarifas.groupby(["broker", "plan"])}
list(planes)

['Alcor/escalonado', 'Alcor/por_accion', 'Brio/fijo', 'Cenit/porcentaje']

A cost function is production code: if it charges wrong, every backtest that uses it is wrong. It is tested with cases computed by hand, one per rule: the minimum, the maximum, each tier and the boundaries between tiers.

def test_por_accion_normal():
    # 1,000 shares at 50 USD: 5 USD, between minimum and maximum
    assert comision(planes["Alcor/por_accion"], 1000, 50) == 5.0

def test_por_accion_minimo():
    assert comision(planes["Alcor/por_accion"], 20, 50) == 1.0

def test_por_accion_maximo():
    # 1,000 shares at 0.20 USD: 5 USD > 1% of 200 USD
    assert comision(planes["Alcor/por_accion"], 1000, 0.2) == 2.0

def test_escalonado_tramos():
    p = planes["Alcor/escalonado"]
    assert comision(p, 1000, 50, volumen_mes=0) == 3.5
    assert comision(p, 1000, 50, volumen_mes=299_999) == 3.5
    assert comision(p, 1000, 50, volumen_mes=300_000) == 2.0
    assert comision(p, 1000, 50, volumen_mes=25_000_000) == 1.0

def test_escalonado_minimo():
    assert comision(planes["Alcor/escalonado"], 10, 50) == 0.35

def test_fijo_y_porcentaje():
    assert comision(planes["Brio/fijo"], 10, 50) == 4.95
    assert comision(planes["Brio/fijo"], 10_000, 50) == 4.95
    assert comision(planes["Cenit/porcentaje"], 1000, 50) == 40.0
    assert comision(planes["Cenit/porcentaje"], 10, 50) == 2.0

def test_venta_igual_que_compra():
    p = planes["Alcor/por_accion"]
    assert comision(p, -1000, 50) == comision(p, 1000, 50)

tests = [f for n, f in list(globals().items())
         if n.startswith("test_") and callable(f)]
for t in tests:
    t()
print(f"{len(tests)} tests passed")

7 tests passed

In a project, these functions go in a module and the tests in a file run by pytest (lesson 0.4.5); here they are called by hand so that the cell is self-contained. The sell test looks trivial, but it catches the most frequent error when reusing a commission function with signed quantities.

Cost by order size

With the functions tested, the cost of each plan in basis points of the traded value for orders from 500 USD to 1 million USD, with a 50 USD stock:

precio = 50.0
importes = [500, 1_000, 5_000, 20_000, 50_000,
            100_000, 500_000, 1_000_000]
curva = pd.DataFrame({
    nombre: [1e4 * comision(p, int(i / precio), precio) / i
             for i in importes]
    for nombre, p in planes.items()}, index=importes)
curva.index.name = "importe_usd"
curva.round(2)

             Alcor/escalonado  Alcor/por_accion  Brio/fijo  Cenit/porcentaje
importe_usd
500                       7.0              20.0      99.00              40.0
1000                      3.5              10.0      49.50              20.0
5000                      0.7               2.0       9.90               8.0
20000                     0.7               1.0       2.48               8.0
50000                     0.7               1.0       0.99               8.0
100000                    0.7               1.0       0.50               8.0
500000                    0.7               1.0       0.10               8.0
1000000                   0.7               1.0       0.05               8.0

The header figure draws these curves on a log scale. Three readings:

• Minimums dominate small orders. At 500 USD, Alcor's per-share plan costs 20 bp and Brio's fixed plan, 99 bp: almost 1% per side. A system that turns over a small account in small orders pays more in commissions than it usually earns.
• Fixed per order wins on large orders. Brio charges the same for 10,000 shares as for 10; above about 50,000 USD (at 50 USD per share) it is cheaper than the per-share plan.
• The percentage does not scale. Cenit costs 8 bp whenever its minimum is exceeded: for large orders it is ten times more expensive than the others.

The crossover point between Brio and Alcor's per-share plan depends on the stock price, because a per-share rate weighs more, in proportion, on cheap stocks:

importe = 20_000
filas = []
for px in [5, 20, 50, 200]:
    n = int(importe / px)
    filas.append({
        "precio": px, "acciones": n,
        "alcor_pb": 1e4 * comision(planes["Alcor/por_accion"],
                                   n, px) / importe,
        "brio_pb": 1e4 * comision(planes["Brio/fijo"],
                                  n, px) / importe})
pd.DataFrame(filas).set_index("precio").round(2)

        acciones  alcor_pb  brio_pb
precio
5           4000      10.0     2.48
20          1000       2.5     2.48
50           400       1.0     2.48
200          100       0.5     2.48

An order of 20,000 USD costs 10 bp on a 5 USD stock and 0.5 bp on a 200 USD one (where the 1 USD minimum already applies). A backtest that models commissions as a fixed percentage of the traded value understates the cost of strategies that buy cheap stocks, which tend to be the small ones, the fallen ones and the ones close to bankruptcy: exactly the ones that populate the short leg of a momentum strategy or the long leg of a value strategy.

Spread and impact: the square-root law

A market order pays the half spread: it buys at the ask or sells at the bid, half the spread away from the midpoint. In the level 2 universe, universo2_caracteristicas.parquet provides for each stock and month the average quoted spread, horquilla_pb, which the generator makes depend on market capitalization and volatility, and which is multiplied by 1.6 in the stress periods:

car = pd.read_parquet("universo2_caracteristicas.parquet")
car["tamano"] = car.groupby("fecha")["capitalizacion"].transform(
    lambda c: pd.qcut(c, 3, labels=["small", "mid", "large"]))
calma, estres = pd.to_datetime(["2007-06-29", "2008-10-31"])
tabla = (car[car["fecha"].isin([calma, estres])]
         .pivot_table(index="tamano", columns="fecha",
                      values="horquilla_pb", aggfunc="median",
                      observed=True))
tabla.columns = ["June 2007", "October 2008"]
tabla.round(1)

        June 2007  October 2008
tamano
small         5.1          12.8
mid           3.5           8.0
large         2.4           5.0

The median spread of the small stocks is twice that of the large ones, and in October 2008 all of them widen between two and three times. The same happens in real data: costs rise precisely when strategies trade the most (trend strategies react to the falls, reversion strategies buy into them), and an average cost over the whole sample does not capture it.

The spread covers small orders. An order that is a sizable fraction of the traded volume moves the price while it is being executed: that is market impact. The most robust empirical regularity about its size is the square-root law: the average impact of a large order (a “metaorder” spread over the day) grows as the square root of its size relative to volume, scaled by daily volatility11Note 1 · Tóth, Lempérière, Deremble, de Lataillade, Kockelkoren and Bouchaud (2011), “Anomalous price impact and the critical nature of liquidity in financial markets”, Physical Review X 1, 021006, propose an explanation of the law and provide empirical evidence; the book by Bouchaud, Bonart, Donier and Gould (2018) develops it. Both are in the resources.:

with $Q$ the shares in the order, $V$ the average daily volume and $\sigma_d$ the daily volatility. The coefficient $Y$ is not a universal constant: it is estimated with your own execution data and changes with the market and with the way of executing. In this lesson $Y = 0.7$ is an assumption, and volatility is the standard deviation of daily returns over the last 63 days. For three stocks in the universe as of June 30, 2015, the largest, the median and the smallest by market capitalization:

aj = pd.read_parquet("universo2_ajustados.parquet")
px = aj.pivot(index="fecha", columns="ticker",
              values="cierre_ajustado")
sigma = px.pct_change(fill_method=None).rolling(
    63, min_periods=40).std()

fecha = pd.Timestamp("2015-06-30")
dia = car[car["fecha"] == fecha].sort_values("capitalizacion")
tres = dia.iloc[[-1, len(dia) // 2, 0]][
    ["ticker", "cierre", "capitalizacion", "volumen_medio_21d",
     "horquilla_pb"]].set_index("ticker")
tres["sigma_d_pct"] = 100 * sigma.loc[fecha, tres.index]
Y = 0.7

def coste_total_pb(t, importe):
    fila = tres.loc[t]
    acciones = int(importe / fila["cierre"])
    com = comision(planes["Alcor/por_accion"], acciones,
                   fila["cierre"])
    media_horq = fila["horquilla_pb"] / 2
    part = acciones / fila["volumen_medio_21d"]
    impacto = 1e4 * Y * fila["sigma_d_pct"] / 100 * np.sqrt(part)
    return 1e4 * com / importe + media_horq + impacto

for imp in [10_000, 1_000_000, 10_000_000]:
    tres[f"{imp:,}"] = [coste_total_pb(t, imp) for t in tres.index]
tres.drop(columns="capitalizacion").round(2)

        cierre  volumen_medio_21d  horquilla_pb  sigma_d_pct  10,000  1,000,000  10,000,000
ticker
QXEH    236.94           16348165           1.8         1.88    2.11       3.22        7.79
QXNR     79.11             590137           3.7         1.43    4.31      17.10       48.71
QXWM     12.56             155718          23.9         2.31   27.50     131.64      381.83

The table and the figure show the total one-side cost (commission, half spread and impact) for three sizes. At 10,000 USD almost all of it is spread, and the small stock already costs more than ten times what the large one does (27.5 bp versus 2.1). At 10 million, impact dominates: in the large stock, the order is a tiny fraction of what trades each day and costs 7.8 bp; in the small one it is more than five times its daily volume, and the estimated cost, 382 bp, no longer makes sense as the cost of a single order: it would have to be split over many days. That is why the capacity of a strategy (lesson 2.2.3) is almost always set by its least liquid leg, and why a backtest with a fixed cost per trade overstates strategies that trade small stocks with a lot of capital.

Financing and securities borrowing in a long-short

A long-short portfolio buys some stocks and sells others short. Selling short requires borrowing the shares: the lender charges an annual fee on the borrowed value, and the cash from the sale stays as collateral, which the broker may pay interest on (the rebate). If the portfolio uses leverage, the long part not covered by own capital is financed with a margin loan at a rate above the risk-free rate. costes2_prestamo.csv provides, per stock and date range, the borrow rate and the availability:

prestamo = pd.read_csv("costes2_prestamo.csv",
                       parse_dates=["desde", "hasta"])
print(prestamo.groupby("disponibilidad")["tasa_prestamo_anual"]
      .describe()[["count", "min", "50%", "max"]])
prestamo[prestamo["disponibilidad"] == "nula"]

                count    min    50%   max
disponibilidad
alta            129.0   0.25   0.38   0.5
baja             10.0   6.30  20.60  24.4
nula              4.0  26.40  38.45  52.8
    ticker      desde      hasta  tasa_prestamo_anual disponibilidad
51    QXKV 2008-04-27 2009-05-29                 52.8           nula
90    QXQT 2007-09-16 2008-10-17                 26.4           nula
132   QXYX 2019-06-02 2020-07-02                 31.9           nula
142   QXZT 2015-04-12 2016-05-13                 45.0           nula

The general rate is between 0.25% and 0.50% a year. Hard-to-borrow stocks cost from 6% to 24%, and the four with nula (none) availability cannot be sold short at any price during their last year. They are the four bankruptcies in the universe (universo2_eventos.csv). The same happens in the real market: the stocks that fall the most are the most in demand for short selling, and borrowing becomes more expensive or disappears.

The test strategy is the one from lesson 2.3.5: 12-1 momentum across the whole universe trading each month, buying the winner quintile and selling the loser one, with equal weights, 100% long and 100% short, monthly rebalancing. In this data momentum has a planted premium (level 2 README: spread between quintiles of ≈ 8% a year) with two crashes in 2009 and 2020. First, the monthly returns with the delisting return included:

eventos = pd.read_csv("universo2_eventos.csv",
                      parse_dates=["fecha"])
mensual = px.resample("ME").last()
r = mensual.pct_change(fill_method=None)
for _, e in eventos.dropna(
        subset=["rentabilidad_deslistado"]).iterrows():
    mes = px[e["ticker"]].last_valid_index() + pd.offsets.MonthEnd(0)
    r.loc[mes, e["ticker"]] = ((1 + r.loc[mes, e["ticker"]])
                               * (1 + e["rentabilidad_deslistado"]) - 1)

# 12-1 momentum at each month-end close: months m-11 to m-1
mom = (1 + r).rolling(11).apply(np.prod, raw=True).shift(1) - 1
car["mes"] = car["fecha"] + pd.offsets.MonthEnd(0)
def mensual_de(col):
    return car.pivot(index="mes", columns="ticker",
                     values=col).reindex(r.index)
precio_m = mensual_de("cierre")
adv_m = mensual_de("volumen_medio_21d")
horq_m = mensual_de("horquilla_pb")
sigma_m = sigma.resample("ME").last()
tipo = pd.read_csv("tipo_libre2.csv", parse_dates=["fecha"])
rf_m = (tipo.set_index("fecha")["tipo_3m"].resample("ME").mean()
        .reindex(r.index) / 100)
print(r.loc["2008-10-31", ["QXQT"]].round(3).to_dict())

{'QXQT': -1.0}

Without that correction, the bankruptcy of QXQT in October 2008 would vanish from the data with its last quote of 0.85 USD, and nobody would book the −100%. With it, the month of the bankruptcy counts in full.

The backtest applies, at each rebalance, all the costs in this lesson. The assumptions go at the top of the function, in plain view:

K, REBATE, CREDITO = 20e6, -0.005, 0.01


def condiciones_prestamo(fecha):
    vigente = prestamo[(prestamo["desde"] <= fecha)
                       & (prestamo["hasta"] >= fecha)]
    v = vigente.set_index("ticker")
    return v["tasa_prestamo_anual"] / 100, v["disponibilidad"]


def long_short(excluir_nula=True, apal=1.0, capital=K):
    pesos, filas = pd.Series(dtype=float), []
    fwd = r.shift(-1)
    for f in r.index[12:-1]:
        m = mom.loc[f].dropna()
        m = m[precio_m.loc[f, m.index].notna()]
        q = pd.qcut(m.rank(method="first"), 5, labels=False)
        tasa, disp = condiciones_prestamo(f)
        largos = list(q.index[q == 4])
        cortos = [t for t in q.index[q == 0]
                  if not (excluir_nula and disp.get(t) == "nula")]
        obj = pd.concat([pd.Series(apal / len(largos), largos),
                         pd.Series(-apal / len(cortos), cortos)])
        giro = obj.sub(pesos, fill_value=0).abs()
        giro = giro[giro > 0]  # delisted stocks already weigh 0
        efectivo = capital * giro
        acc = efectivo / precio_m.loc[f, giro.index]
        part = acc / adv_m.loc[f, giro.index]
        com = sum(comision(planes["Alcor/por_accion"], a, p)
                  for a, p in zip(acc, precio_m.loc[f, giro.index]))
        horq = (giro * horq_m.loc[f, giro.index] / 2e4).sum()
        imp = (giro * Y * sigma_m.loc[f, giro.index]
               * np.sqrt(part)).sum()
        bruto = apal * (fwd.loc[f, largos].mean()
                        - fwd.loc[f, cortos].mean())
        filas.append({
            "fecha": f + pd.offsets.MonthEnd(1),
            "bruto": bruto,
            "comision": com / capital,
            "horquilla": horq,
            "impacto": imp,
            "prestamo": apal * tasa.reindex(cortos)
                        .fillna(0.0025).mean() / 12,
            "rebate": apal * (rf_m.loc[f] + REBATE) / 12,
            "credito": max(apal - 1, 0) * (rf_m.loc[f] + CREDITO) / 12,
            "rotacion": giro.sum() / 2})
        # weights drift with the month's returns
        rr = fwd.loc[f, obj.index].fillna(-1.0)
        pesos = obj * (1 + rr) / (1 + bruto)
    d = pd.DataFrame(filas).set_index("fecha")
    d["neto"] = (d["bruto"] - d["comision"] - d["horquilla"]
                 - d["impacto"] - d["prestamo"] + d["rebate"]
                 - d["credito"])
    return d


ingenuo = long_short(excluir_nula=False)
real = long_short()
print(len(real), "months:", real.index[0].date(), "to",
      real.index[-1].date())

227 months: 2007-02-28 to 2025-12-31

The weight of each stock drifts during the month with its return, so each rebalance pays only the difference between the current weight and the target. A stock that stops trading reaches the rebalance with zero weight (its fillna(-1) means “no longer there”), and the portfolio replaces it. The annual breakdown of the two versions:

def anual(d):
    a = 12 * d.drop(columns="rotacion").mean()
    a["rotacion_anual"] = 12 * d["rotacion"].mean()
    a["sharpe_bruto"] = (d["bruto"].mean() / d["bruto"].std()
                         * np.sqrt(12))
    a["sharpe_neto"] = (d["neto"].mean() / d["neto"].std()
                        * np.sqrt(12))
    return a

comparativa = pd.DataFrame({"ingenuo": anual(ingenuo),
                            "realista": anual(real)})
comparativa.round(4)

                ingenuo  realista
bruto            0.0971    0.0670
comision         0.0022    0.0022
horquilla        0.0044    0.0043
impacto          0.0206    0.0203
prestamo         0.0084    0.0044
rebate           0.0105    0.0105
credito          0.0000    0.0000
neto             0.0766    0.0506
rotacion_anual   6.2178    6.2478
sharpe_bruto     0.8059    0.5541
sharpe_neto      0.6337    0.4159

Read from top to bottom, the table tells the whole story:

• The naive gross return (9.7% a year) sells the bankruptcies short during their last year. The borrowing constraint alone takes three points from it: the realistic gross return is 6.7%, close to the ≈ 8% that the README states for the planted premium between quintiles, minus the crashes.
• Impact (2.0% a year) is the dominant cost: the portfolio turns over about six times its value a year, and with 20 million USD the orders in the small stocks are a sizable fraction of their volume. The half spread (0.4%) and the commission (0.2%) are smaller.
• Borrowing costs 0.4% in the realistic version and twice that in the naive one, which pays rates from 26% to 53% (if it could) for the bankruptcies.
• The rebate adds 1.1% a year: with the cash from the sales as collateral paid at the risk-free rate minus half a point. In years of high rates, a sizable part of the result of a long-short is this interest, not the signal.

The Sharpe goes from 0.81 (naive, gross) to 0.42 (realistic, net). The figure shows how the curves separate: the gap between the naive and the realistic versions opens almost entirely in 2007-2009, with the first two bankruptcies; the gap from gross to net grows steadily, because costs are paid at every rebalance. One dollar from 2007 ends at 5.44 in the naive version, at 3.09 in the realistic gross one and at 2.24 in the net one. The data is synthetic and the premium is planted: what you learn is not how much momentum earns, but that a gross edge of 0.8 Sharpe can end up as little more than half when what it costs to trade it is modeled.

flowchart TB
  S[Monthly signal] --> P[Target weights]
  D[Availability] --> P
  P --> G[Weight turnover]
  G --> CO[Commission]
  G --> H[Half spread]
  G --> I[Square-root impact]
  P --> F[Borrow and rebate]
  CO --> N[Net return]
  H --> N
  I --> N
  F --> N

The diagram summarizes the order of operations in the function: availability comes in before setting the weights (a security that cannot be sold does not enter the portfolio), transaction costs depend on turnover and holding costs, on positions.

Typical errors when modeling costs

• A fixed cost in basis points for everything. It ignores minimums, the dependence on the share price, the spread of small stocks and impact, which grows with capital.
• Whole-sample costs instead of those of the moment. Spread and volatility rise in crises; an average cost understates precisely the periods in which the strategy trades the most.
• Selling short what cannot be borrowed. It gives away the gains from the bankruptcies, as shown.
• Forgetting interest. The rebate and the margin loan can be of the same order as the edge; with rates at 5%, a long-short that “makes 6%” may be earning mostly interest.
• Calibrating impact with the backtest. The coefficient $Y$ is measured with real executions (or taken from the literature as an order of magnitude) and varied to see how much the result depends on it; tuning it until the strategy looks good is overfitting.

Exercises

1. Find the amount above which Brio (fixed per order) is cheaper than Alcor's per-share plan for a 20 USD stock and for a 100 USD stock. Explain why it changes.

for px_ in [20, 100]:
    for n in range(1, 10_000):
        if (comision(planes["Brio/fijo"], n, px_)
                < comision(planes["Alcor/por_accion"], n, px_)):
            print(f"price {px_}: from {n} shares",
                  f"({n * px_:,} USD)")
            break

price 20: from 991 shares (19,820 USD)
price 100: from 991 shares (99,100 USD)

2. Repeat the realistic backtest with 200 million USD of capital. Which costs change, by how much and why?

grande = long_short(capital=200e6)
pd.DataFrame({"20 M": anual(real), "200 M": anual(grande)}).loc[
    ["bruto", "comision", "horquilla", "impacto", "neto",
     "sharpe_neto"]].round(4)

               20 M   200 M
bruto        0.0670  0.0670
comision     0.0022  0.0022
horquilla    0.0043  0.0043
impacto      0.0203  0.0642
neto         0.0506  0.0065
sharpe_neto  0.4159  0.0533

3. A colleague presents a long-short with a Sharpe of 1.2 in 2023-2025 and says that “borrowing and financing are details”. Which three checks would you ask for before believing it?