Order management: states, partial fills, cancellations and reconnections.

What the broker answers

In lesson 4.5.1 the simulated broker always answered, in order and once. A real broker promises none of that. The Interactive Brokers API documentation says so plainly: it is normal to receive duplicate status messages, there is no guarantee that one arrives for each state change (a market order that fills instantly may not produce any) and that is why it recommends also watching the execution messages11Note 1 · Interactive Brokers, “TWS API: Placing Orders”, sections on order states and orderStatus (accessed on October 22, 2026; link in the resources).. The level data include four months of messages from a broker like that: vivo4_eventos_broker.csv, in the order in which they arrived, with the sequence number (seq) the broker assigns to each message of each account.

import heapq

import numpy as np
import pandas as pd

ord_ = pd.read_csv("vivo4_ordenes.csv")
ev = pd.read_csv("vivo4_eventos_broker.csv")
ev["llegada"] = np.arange(len(ev))        # order of arrival
for c in ("ts_broker", "ts_recibido"):
    ev[c] = pd.to_datetime(ev[c])
print(len(ord_), "orders;", len(ev), "messages")
pd.crosstab(ev["evento"], ev["modo"], margins=True)

398 orders; 983 messages
modo       paper  real  All
evento
aceptada     134   262  396
cancelada     16    34   50
ejecutada    118   232  350
parcial       65   120  185
rechazada      1     1    2
All          334   649  983

Five message types: aceptada (acknowledged), parcial (a fill that does not complete the order), ejecutada (the one that completes it), cancelada (canceled) and rechazada (rejected). Each one carries the quantity of that fill (cantidad_ultima), the cumulative quantity, the leaves quantity and the average price of everything filled: the complete state of the order, not just the change. That redundancy is what will save the system when a message is missing.

A limit order from the live account that is partially filled and canceled after seven minutes, and the market order the OMS sends with the remainder:

print(ord_.set_index("id_orden_cliente").loc[
    ["R00047", "R00048"], ["instrumento", "lado", "tipo", "cantidad",
                           "precio_limite", "orden_origen"]])
cortos = {"id_orden_cliente": "orden", "cantidad_ultima": "q",
          "precio_ultimo": "precio", "cantidad_acumulada": "acum",
          "cantidad_pendiente": "pend"}
(ev.loc[ev["id_orden_cliente"].isin(["R00047", "R00048"]),
        ["seq", "evento", *cortos]]
   .rename(columns=cortos).set_index("seq"))

                 instrumento   lado     tipo  cantidad  precio_limite orden_origen
id_orden_cliente
R00047                  ORIG  venta   limite       222         112.71          NaN
R00048                  ORIG  venta  mercado        91            NaN       R00047
        evento   orden    q  precio  acum  pend
seq
99    aceptada  R00047    0     NaN     0   222
110    parcial  R00047  131  112.71   131    91
112  cancelada  R00047    0     NaN   131     0
113   aceptada  R00048    0     NaN     0    91
114  ejecutada  R00048   91  112.71    91     0

R00047 sells 222 ORIG at 112.71 or better. 131 are filled at the limit price; after seven minutes the OMS requests a cancellation and the broker confirms it with the cumulative quantity (131) and the leaves quantity at zero. The OMS then sends R00048, a market order, for the 91 still missing, with orden_origen pointing to the limit order. These are three rules the state machine has to check: the cumulative quantity is the sum of the fills, the leaves quantity is what was ordered minus the cumulative (zero on cancellation) and the remainder is exactly what was not filled.

The state machine of an order

The life cycle of an order has been standardized for decades in the FIX protocol (the OrdStatus field) and each broker uses a dialect of it; the site's research on state machines covers it in detail. The states in this lesson and their equivalents:

The complete lists of Alpaca and Interactive Brokers have more states (done_for_day, expired, replaced…) 22Note 2 · Alpaca, “Orders at Alpaca”, “Order Lifecycle” section (accessed on October 22, 2026; link in the resources).; a system that trades day orders without modifying them can treat them as variants of those in the table, but it has to recognize them: an unknown state is an error, not an else.

stateDiagram-v2
  [*] --> enviada
  enviada --> aceptada
  enviada --> rechazada
  aceptada --> parcial
  aceptada --> ejecutada
  parcial --> parcial
  parcial --> ejecutada
  aceptada --> pend_cancel
  parcial --> pend_cancel
  pend_cancel --> pend_cancel : fill
  pend_cancel --> cancelada
  pend_cancel --> ejecutada : race
  ejecutada --> [*]
  cancelada --> [*]
  rechazada --> [*]

pend_cancel is pendiente_cancelacion: the OMS has requested a cancellation and the broker has not answered yet. It is not a detail: while the request travels, the order can keep filling, and if it fills completely before the cancellation arrives, the broker answers with a fill, not with a cancellation (the “race”). A system that considered the order canceled as soon as it requested it would send the remainder as a market order and buy twice.

The machine is coded as a transition table and a class that applies it and checks the invariants. Any (state, event) pair that is not in the table is an invalid transition and raises an exception: it is not ignored.

FINALES = {"ejecutada", "cancelada", "rechazada"}
T = {
    ("enviada", "aceptada"): "aceptada",
    ("enviada", "rechazada"): "rechazada",
    ("aceptada", "parcial"): "parcial",
    ("aceptada", "ejecutada"): "ejecutada",
    ("parcial", "parcial"): "parcial",
    ("parcial", "ejecutada"): "ejecutada",
    ("aceptada", "pedir_cancelacion"): "pendiente_cancelacion",
    ("parcial", "pedir_cancelacion"): "pendiente_cancelacion",
    ("pendiente_cancelacion", "parcial"): "pendiente_cancelacion",
    ("pendiente_cancelacion", "ejecutada"): "ejecutada",
    ("pendiente_cancelacion", "cancelada"): "cancelada",
}

class Invalida(Exception):
    pass

class Orden:
    def __init__(self, id_, cantidad):
        self.id, self.cantidad = id_, cantidad
        self.estado, self.acumulada, self.valor = "enviada", 0, 0.0
        self.ejecuciones = {}          # id_ejecucion -> (q, price)

    def aplicar(self, e):
        clave = (self.estado, e.evento)
        if clave not in T:
            raise Invalida(f"{self.id}: “{e.evento}” in "
                           f"“{self.estado}”")
        if e.evento in ("parcial", "ejecutada"):
            if e.id_ejecucion in self.ejecuciones:
                raise Invalida(f"{self.id}: {e.id_ejecucion} repeated")
            nueva = self.acumulada + e.cantidad_ultima
            if nueva != e.cantidad_acumulada:
                raise Invalida(f"{self.id}: cumulative {self.acumulada}"
                               f" + {e.cantidad_ultima} != "
                               f"{e.cantidad_acumulada}")
            self.ejecuciones[e.id_ejecucion] = (e.cantidad_ultima,
                                                e.precio_ultimo)
            self.acumulada = e.cantidad_acumulada
            self.valor += e.cantidad_ultima * e.precio_ultimo
        if e.evento == "cancelada" and (
                e.cantidad_acumulada != self.acumulada):
            raise Invalida(f"{self.id}: cumulative {self.acumulada}"
                           f" != {e.cantidad_acumulada} on cancel")
        if e.evento in ("parcial", "ejecutada", "cancelada"):
            pend = 0 if e.evento == "cancelada" else (
                self.cantidad - self.acumulada)
            if e.cantidad_pendiente != pend:
                raise Invalida(f"{self.id}: leaves "
                               f"{e.cantidad_pendiente} != {pend}")
        self.estado = T[clave]

The order starts in enviada when the OMS sends it (not when the broker acknowledges it) and the OMS adds its own event, pedir_cancelacion (request cancel), at the time it requested it (ts_cancelacion in vivo4_ordenes.csv). The other events are the broker's.

Processing messages as they arrive

The simplest system applies each message in order of arrival. The OMS stream (submissions and cancel requests) is merged with the broker's by system time: ts_envio and ts_cancelacion for the OMS ones, ts_recibido for the broker's.

def eventos_oms(ordenes):
    """The events the OMS itself generates, with their time."""
    filas = []
    for o in ordenes.itertuples():
        if pd.notna(o.ts_cancelacion):
            filas.append(dict(t=pd.Timestamp(o.ts_cancelacion),
                              id_orden_cliente=o.id_orden_cliente,
                              evento="pedir_cancelacion"))
    return pd.DataFrame(filas)

def procesar(flujo, ordenes):
    """flujo: table with t, id_orden_cliente, evento and fields."""
    libro = {o.id_orden_cliente: Orden(o.id_orden_cliente, o.cantidad)
             for o in ordenes.itertuples()}
    errores = []
    for e in flujo.itertuples():
        o = libro[e.id_orden_cliente]
        try:
            o.aplicar(e)
        except Invalida as x:
            errores.append((o.estado, e.evento, str(x)))
    return libro, errores

def tipo_error(estado, evento, texto):
    if estado == "enviada" and evento in ("parcial", "ejecutada"):
        return "fill before acknowledgment"
    if estado in FINALES:
        return "message after a final state"
    return "cumulative or leaves mismatch"

def mezclar(broker, ordenes, col_t):
    b = broker.assign(t=broker[col_t])
    return (pd.concat([b, eventos_oms(ordenes)], ignore_index=True)
            .sort_values("t", kind="stable"))

libro_n, err_n = procesar(mezclar(ev, ord_, "ts_recibido"), ord_)
print(len(err_n), "errors")
print(pd.Series([tipo_error(*x) for x in err_n]).value_counts()
      .to_string())
print([x[2] for x in err_n[:2]])

53 errors
fill before acknowledgment       28
cumulative or leaves mismatch    21
message after a final state       4
['P00007: “parcial” in “enviada”', 'P00007: cumulative 0 + 241.0 != 409.0']

Fifty-three invalid transitions. The first 28 are fills that arrive before their order's acknowledgment: the broker acknowledged and filled in milliseconds and the two messages traveled with different latencies. The 21 cumulative ones are almost all a consequence of those: the machine rejects the early fill and the order's next message no longer adds up; among them are also the one for the partial fill that arrives after the final one (seq 393, anomaly A09 in the data) and those for the two fills that never arrived. The last 4 are the messages repeated on reconnecting, which reach orders that are already finished. With this processing, a system that ignored the errors would have orders in absurd states; one that counted all the fills, false positions.

A sequencer by sequence number

The broker numbers the messages of each account without gaps. That makes it possible to rebuild the order: a sequencer waits for the seq following the last one delivered, holds those that arrive early and discards those it has already delivered. If the missing one does not arrive within a timeout, it declares it a gap and moves on: you cannot wait forever.

flowchart TB
  L[seq n arrives] --> Q{n vs expected}
  Q -- lower --> D[Discard repeated]
  Q -- equal --> E[Deliver and flush]
  Q -- higher --> R[Hold]
  R --> V{Timeout expires}
  V -- yes --> H[Declare gap]
  H --> E

The timeout is measured with the message clock (ts_recibido), not the computer's: the sequencer receives the time of each arrival and checks expired timeouts before processing it. That way the result is the same on any machine, loaded or not; in live trading, an event loop timer would pass it the time every so often.

class Secuenciador:
    def __init__(self, plazo=pd.Timedelta(seconds=2)):
        self.esperado, self.plazo = 1, plazo
        self.retenidos = {}          # seq -> (arrival, message)
        self.salida, self.repetidos, self.huecos = [], [], []
        self.vistos = set()

    def _vaciar(self, t):
        while self.esperado in self.retenidos:
            _, m = self.retenidos.pop(self.esperado)
            self.salida.append((t, m))
            self.vistos.add(self.esperado)
            self.esperado += 1

    def vencer(self, t):
        """Declares the gaps whose timeout expired before t."""
        while self.retenidos:
            limite = min(x for x, _ in self.retenidos.values()) \
                + self.plazo
            if limite > t:
                return
            nuevo = min(self.retenidos)
            self.huecos.extend(range(self.esperado, nuevo))
            self.esperado = nuevo
            self._vaciar(limite)

    def llegar(self, t, m):
        self.vencer(t)
        if m.seq in self.vistos or m.seq in self.retenidos:
            self.repetidos.append(m.seq)
            return
        if m.seq < self.esperado:            # arrives after its gap
            self.salida.append((t, m))
            self.vistos.add(m.seq)
            return
        self.retenidos[m.seq] = (t, m)
        self._vaciar(t)

def secuenciar(mensajes):
    s = Secuenciador()
    for m in mensajes.itertuples():
        s.llegar(m.ts_recibido, m)
    s.vencer(pd.Timestamp.max.tz_localize("UTC"))
    filas = [m._asdict() | {"t": t} for t, m in s.salida]
    return pd.DataFrame(filas), s

sec, info = {}, {}
for modo, g in ev.groupby("modo", sort=False):
    sec[modo], info[modo] = secuenciar(g)
    print(f"{modo}: delivered {len(sec[modo])}, "
          f"repeated {info[modo].repetidos}, "
          f"gaps {info[modo].huecos}")

paper: delivered 334, repeated [], gaps []
real: delivered 645, repeated [61, 62, 471, 475], gaps [202, 519]

In paper, the 334 messages are delivered with no repeats or gaps. In the live account, the sequencer discards the four repeated messages and declares two gaps, seq 202 and 519: the two messages the broker did not resend on reconnecting. Message 393, which arrived half a second late (after 405), is waited for within the two-second timeout and delivered in its place. How much does waiting cost? The delay between arrival and delivery:

todo = pd.concat(sec.values(), ignore_index=True)
retraso = (todo["t"] - todo["ts_recibido"]).dt.total_seconds() * 1e3
print(f"Delivered immediately: {(retraso == 0).mean():.1%}")
print("Held:", (retraso > 0).sum(), "| median",
      f"{retraso[retraso > 0].median():.1f} ms | maximum",
      f"{retraso.max():,.0f} ms")

Delivered immediately: 93.2%
Held: 67 | median 7.1 ms | maximum 2,000 ms

93.2% of the messages are delivered immediately. The 67 held messages wait a median of 7 ms (the fill that overtook its acknowledgment) and the worst ones, those behind a gap, the full timeout: two seconds. The timeout is a trade-off: shorter, and a message that is merely late is declared a gap; longer, and the system learns late about everything behind it.

Recovering what did not arrive

With the messages in order, the state machine only finds what is really missing:

flujo = pd.concat([pd.concat(sec.values()), eventos_oms(ord_)],
                  ignore_index=True).sort_values("t", kind="stable")
libro, errores = procesar(flujo, ord_)
print(len(errores), "errors")
for x in errores:
    print(" ", x[2])
print(pd.Series({k: o.estado for k, o in libro.items()})
      .value_counts().to_dict())

2 errors
  R00079: cumulative 0 != 112.0 on cancel
  R00214: cumulative 0 + 1016.0 != 1573.0
{'ejecutada': 345, 'cancelada': 49, 'rechazada': 2, 'pendiente_cancelacion': 1, 'aceptada': 1}

Two errors, and they are the two gaps: in R00079 the cancellation arrives with 112 shares cumulative when the system had counted only 0; in R00214, the final fill with 1,573 cumulative when it had counted 0. All the other orders go through the machine without a single invalid transition and end in a final state, except those two, which stay where the error left them.

The message following the gap brings what is needed to rebuild what was lost. The quantity is the difference between the message's cumulative quantity and what was counted plus the message's own fill; the price comes from the average price:

with $A$ and $\overline{p}$ the cumulative quantity and average price of the message, $a$ and $V$ the quantity and value counted up to then, and $q$, $p$ the message's own fill (zero on a cancellation).

class OrdenRecuperable(Orden):
    def aplicar(self, e):
        q = e.cantidad_ultima if e.evento in ("parcial",
                                               "ejecutada") else 0
        if e.cantidad_acumulada > self.acumulada + q:
            perdida = e.cantidad_acumulada - self.acumulada - q
            px = ((e.precio_medio * e.cantidad_acumulada - self.valor
                   - q * (e.precio_ultimo if q else 0)) / perdida)
            self.ejecuciones[f"inferida-{e.seq}"] = (perdida,
                                                     round(px, 2))
            self.acumulada += perdida
            self.valor += perdida * px
            self.inferida = (perdida, round(px, 2))
        super().aplicar(e)

def procesar_rec(flujo, ordenes):
    libro = {o.id_orden_cliente: OrdenRecuperable(o.id_orden_cliente,
                                                  o.cantidad)
             for o in ordenes.itertuples()}
    errores = []
    for e in flujo.itertuples():
        try:
            libro[e.id_orden_cliente].aplicar(e)
        except Invalida as x:
            errores.append(str(x))
    return libro, errores

libro_r, err_r = procesar_rec(flujo, ord_)
print(len(err_r), "errors")
for k, o in libro_r.items():
    if hasattr(o, "inferida"):
        print(k, o.estado, "inferred:", o.inferida)
verdad = pd.read_csv("vivo4_anomalias_verdad.csv")
verdad.loc[verdad["tipo"] == "falta_ejecucion",
           ["id_anomalia", "id_orden_cliente", "descripcion"]]

0 errors
R00079 cancelada inferred: (112.0, 99.06)
R00214 ejecutada inferred: (557.0, 114.03)
   id_anomalia id_orden_cliente                                        descripcion
3          A04           R00079  La ejecución parcial E710000553.01 (112 título...
11         A12           R00214  La ejecución parcial E710001498.01 (557 título...

Zero errors, and the two inferred fills are exactly those of the ground truth in the data: 112 shares at 99.06 in R00079 and 557 at 114.03 in R00214. The inference works because the broker sends the complete state in every message; with a broker that only sent changes there would be nothing to solve for. Even so, an inferred fill is not a confirmed fill: it has no id_ejecucion and no commission. The system must flag it, ask the broker for the day's fills (every API has that query) and replace it with the real one; if it does not match, it is a mismatch for the reconciliation (lesson 4.5.5).

Partial fills and cancellations

An order being filled in several parts is the norm, not the exception. With the book already rebuilt:

n_ej = pd.Series({k: len(o.ejecuciones) for k, o in libro_r.items()})
tabla = ord_.assign(ejecuciones=ord_["id_orden_cliente"].map(n_ej))
print(pd.crosstab(tabla["tipo"], tabla["ejecuciones"]))
lim = tabla[tabla["tipo"] == "limite"].copy()
fin = lim["id_orden_cliente"].map(lambda k: libro_r[k].estado)
acu = lim["id_orden_cliente"].map(lambda k: libro_r[k].acumulada)
lim["destino"] = np.where(fin == "ejecutada", "full",
                          np.where(acu > 0, "partial", "none"))
pd.crosstab(lim["modo"], lim["destino"], margins=True)

ejecuciones   0    1   2   3   4
tipo
cierre        0   56   0   0   0
limite       18   75   7   5   6
mercado       2  165  25  22  17
destino  full  none  partial  All
modo
paper      22     6       10   38
real       39    12       22   73
All        61    18       32  111

Market-on-close orders (MOC) are always filled in a single part, in the auction. Large market orders are split into two to four fills; limit orders, almost always into one, because they are filled against orders that arrive at the price. What sets limit orders apart is their fate: in the live account, 39 of 73 were filled completely, 22 partially and 12 not at all. The 34 that were not filled completely were canceled after seven minutes and their remainder was sent as a market order. The OMS rule for that remainder is checked on all of them:

resto = ord_[ord_["motivo"] == "resto_de_limite"]
pedido = resto["cantidad"].to_numpy()
falta = resto["orden_origen"].map(
    lambda k: libro_r[k].cantidad - libro_r[k].acumulada).to_numpy()
cancel = resto["orden_origen"].map(lambda k: libro_r[k].estado)
print(len(resto), "remainders;", "all from canceled orders:",
      (cancel == "cancelada").all(), "| correct quantity:",
      (pedido == falta).all())

50 remainders; all from canceled orders: True | correct quantity: True

The 50 remainders (16 in paper, 34 in live) come from a canceled limit order and ask for exactly what was missing. Note that the check uses the rebuilt cumulative quantity: R00079 is one of those limit orders, and without the inferred fill of 112 shares the remainder would have looked 112 shares too large. An OMS that computed the remainder with what it had counted, and not with the cumulative quantity the broker confirms on cancellation, would have sold 154 ORTN instead of 42: 112 too many.

Rejections are few and should be treated as what they are, a final answer:

r = ev[ev["evento"] == "rechazada"]
r[["id_orden_cliente", "ts_broker", "motivo"]].merge(
    ord_[["id_orden_cliente", "instrumento", "cantidad"]])

  id_orden_cliente                        ts_broker  ... instrumento cantidad
0           P00005 2025-09-02 19:50:00.862198+00:00  ...        ORTL     1639
1           R00007 2025-10-14 13:35:00.585315+00:00  ...        ORIG     1260

[2 rows x 5 columns]

Two buys rejected for lack of buying power, one in each account and both in the first days, when the portfolio is being built from cash. Retrying the same order would give the same rejection: the OMS does not retry, and the next signal recomputes the difference with what is really there. A rejection for a transient cause (the market has not opened yet, a request limit) is handled differently, and that is why the reason must be stored with the order.

Disconnections and reconnections

The connection with the broker has its own log (vivo4_conexion.csv): each session starts with conectado (connected) and ends with desconectado (disconnected, with the detail fin de sesión, “end of session”), and in between there are seven drops. Each drop follows the same script: the system stops receiving the broker's heartbeat, after three seconds it logs it (latido_perdido), the connection closes, the system reconnects with a new session and the broker resends what the system missed.

con = pd.read_csv("vivo4_conexion.csv", parse_dates=["ts"])
cortes = con[con["evento"] != "conectado"]
cortes = cortes[cortes["detalle"] != "fin de sesión"]
filas = []
for (modo, dia), g in cortes.groupby(["modo", cortes["ts"].dt.date]):
    t_d = g.loc[g["evento"] == "desconectado", "ts"].iloc[0]
    t_r = g.loc[g["evento"] == "reconectado", "ts"].iloc[0]
    cuenta = ev[ev["modo"] == modo]
    rep = cuenta[(cuenta["ts_broker"] < t_r)
                 & (cuenta["ts_recibido"] >= t_r)]
    seqs = range(rep["seq"].min(), rep["seq"].max() + 2) \
        if len(rep) else []
    filas.append(dict(modo=modo, dia=dia,
                      segundos=int((t_r - t_d).total_seconds()),
                      reenviados=len(rep),
                      repetidos=int(rep["seq"].duplicated().sum()),
                      huecos=[h for h in info[modo].huecos
                              if h in seqs]))
pd.DataFrame(filas)

    modo         dia  segundos  reenviados  repetidos huecos
0  paper  2025-09-23        42           0          0     []
1   real  2025-10-21        92          16          2     []
2   real  2025-11-03       137           4          0  [202]
3   real  2025-11-13        72          14          0     []
4   real  2025-12-03       117          18          0     []
5   real  2025-12-09        57          15          2     []
6   real  2025-12-15       107           6          0  [519]

Drops of between 42 and 137 seconds, all at the most delicate time: right when the system sends the closing orders. The paper one fell on a day without orders and left nothing to resend. In those of the live account, on reconnecting the broker resends between 4 and 18 messages generated during the drop; in two of them it sends two fills twice (the repeated messages the sequencer discarded) and in another two it leaves one out (gaps 202 and 519). The header figure summarizes what each drop left behind: repeats, gaps, an out-of-order message and a duplicated order.

The duplicated order is the most expensive one, and it is not the broker's fault. On November 13, the OMS sent R00113 (sell 7 ORUS) at 20:50:00, when the heartbeat was no longer arriving and two seconds before the connection closed. On reconnecting, without the acknowledgment, it considered it lost and resent it as R00114:

dup = ord_.set_index("id_orden_cliente").loc[["R00113", "R00114"]]
print(dup[["ts_envio", "lado", "cantidad", "motivo", "orden_origen"]])
t_rec = pd.Timestamp("2025-11-13 20:51:14", tz="UTC")
m = ev[ev["id_orden_cliente"].isin(["R00113", "R00114"])]
hora = lambda c: m[c].dt.strftime("%H:%M:%S.%f").str[:12]
m.assign(broker=hora("ts_broker"), recibido=hora("ts_recibido"))[
    ["seq", "id_orden_cliente", "evento", "broker", "recibido"]]

                                     ts_envio   lado  cantidad                 motivo orden_origen
id_orden_cliente
R00113            2025-11-13T20:50:00.246596Z  venta         7             rebalanceo          NaN
R00114            2025-11-13T20:51:14.296596Z  venta         7  reenvio_sin_respuesta       R00113
     seq id_orden_cliente     evento        broker      recibido
628  308           R00114   aceptada  20:51:14.345  20:51:14.375
629  309           R00114  ejecutada  20:51:14.389  20:51:14.422
631  295           R00113   aceptada  20:50:00.290  20:51:14.586
634  298           R00113  ejecutada  20:50:00.343  20:51:14.706

The broker had acknowledged and filled R00113 at 20:50:00, but its messages did not arrive before the drop. The OMS resent at 20:51:14.296, right on reconnecting, and the messages for R00113 arrived 290 ms after the resend. Result: two sales of 7 ORUS where the signal asked for one. The correct alternative is to ask before resending. With the state the broker had at the instant of the resend:

def estado_en_broker(id_cliente, t):
    """What the query of the order by its client id would return
    (simulated with the messages the broker had already generated)."""
    m = ev[(ev["id_orden_cliente"] == id_cliente)
           & (ev["ts_broker"] <= t)].sort_values("seq")
    return "unknown" if m.empty else m["evento"].iloc[-1]

def al_reconectar(en_vuelo, t):
    for id_cliente in en_vuelo:
        estado = estado_en_broker(id_cliente, t)
        accion = "send with the same id" if (
            estado == "unknown") else "do not resend"
        print(f"{id_cliente}: the broker says “{estado}” -> {accion}")

al_reconectar(["R00113"], pd.Timestamp("2025-11-13 20:51:14.296",
                                       tz="UTC"))

R00113: the broker says “ejecutada” -> do not resend

The query would have said ejecutada (filled) and there would have been no resend. If the broker did not know the order, the safe thing is to send it with the same client identifier: if the first one arrived, a broker that requires unique client identifiers rejects the second as a duplicate. It is the idempotency key, which lesson 3.7.4 showed with ccxt and which lesson 4.5.5 develops together with persistent state. The order of operations on reconnecting is then:

flowchart TB
  R[Reconnect] --> RE[Receive resend]
  RE --> S[Sequence]
  S --> C[Query in-flight]
  C --> E[Request fills]
  E --> O[Resume trading]

Receive and sequence the resend, query the orders that still have no final state, request the day's fills to confirm the inferred ones and, only then, start sending orders again. Meanwhile, the signal can keep computing, but the OMS does not send.

The heartbeat, with an injected clock

The disconnection is detected by the heartbeat: the broker sends a message every so often even when there is nothing to say, and if three seconds pass without any, the connection is considered dead. Like the sequencer, the heartbeat monitor must not read the computer's clock: it receives the time as an argument, and that way it can be tested with made-up times:

class Latido:
    def __init__(self, limite=pd.Timedelta(seconds=3)):
        self.limite, self.ultimo, self.caido = limite, None, False

    def recibir(self, t):
        self.ultimo, self.caido = t, False

    def comprobar(self, t):
        if not self.caido and t - self.ultimo >= self.limite:
            self.caido = True
            return f"{t.time()} no heartbeat since {self.ultimo.time()}"
        return None

t0 = pd.Timestamp("2025-12-15 20:49:50", tz="UTC")
lat = Latido()
for s in range(0, 6):                      # heartbeats every second
    lat.recibir(t0 + pd.Timedelta(seconds=s))
for s in range(6, 12):                     # the broker goes silent
    aviso = lat.comprobar(t0 + pd.Timedelta(seconds=s))
    if aviso:
        print(aviso)

20:49:58 no heartbeat since 20:49:55

The last heartbeat arrives at 20:49:55 and the monitor fires at 20:49:58: the time at which the connection log in the data records “sin latido del broker en 3 s” (no broker heartbeat for 3 s) in the drop of December 15. The same class, in production, receives the time from an event loop timer; in tests, whatever time you want, without waiting a single second.

Exercises

1. Change the sequencer's timeout to 0.2 seconds. What happens to message 393 of the live account? Does the state machine handle it correctly?

s2 = Secuenciador(plazo=pd.Timedelta(seconds=0.2))
for m in ev[ev["modo"] == "real"].itertuples():
    s2.llegar(m.ts_recibido, m)
s2.vencer(pd.Timestamp.max.tz_localize("UTC"))
print("gaps:", s2.huecos)
orden_393 = [m.seq for _, m in s2.salida].index(393)
print("delivery position of 393:", orden_393, "| before it:",
      [m.seq for _, m in s2.salida][orden_393 - 3:orden_393])
sal = pd.DataFrame([m._asdict() | {"t": t} for t, m in s2.salida])
ord_r = ord_[ord_["modo"] == "real"]
fl = pd.concat([sal, eventos_oms(ord_r)], ignore_index=True)
_, err2 = procesar_rec(fl.sort_values("t", kind="stable"), ord_r)
print(err2)

gaps: [202, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 393, 398, 399, 400, 401, 402, 403, 404, 519]
delivery position of 393: 403 | before it: [402, 403, 404]
['R00162: “parcial” in “ejecutada”']

2. Check that, in the rebuilt book, no order in a final state has a leaves quantity and that the cumulative quantity of each order is the sum of its fills (including the inferred ones).

mal = []
for k, o in libro_r.items():
    suma = sum(q for q, _ in o.ejecuciones.values())
    if suma != o.acumulada:
        mal.append((k, "sum"))
    if o.estado == "ejecutada" and o.acumulada != o.cantidad:
        mal.append((k, "incomplete fill"))
    if o.estado not in FINALES:
        mal.append((k, o.estado))
print(len(libro_r), "orders;", len(mal), "violate them:", mal)

398 orders; 0 violate them: []

3. A limit order is in pendiente_cancelacion and a fill arrives that completes it. What should the OMS do with the remainder it was planning to send as a market order?