Lesson 1.1.1 · 45 min
Market participants and trading venues
Behind every price there is a chain of intermediaries and a dozen places where an order can be matched. This lesson puts them in order and measures them on a full session of a fictitious stock, trade by trade.
- Module
- 1.1 · How markets work
- Duration
- 45 min
- Author
- Jesús Cuesta
- Updated
Level 1 lessons
1.1 · How markets work
1.2 · Asset classes
- 1.2.1 Stocks: splits, dividends and corporate actions
- 1.2.2 ETFs: replication, tracking error, leveraged ETFs and UCITS
- 1.2.3 Futures: specifications, margin, roll and contango
- 1.2.4 Forex: pairs, pips, lots, rollover and CFDs
- 1.2.5 Options: calls, puts, intrinsic value and time value
- 1.2.6 Crypto: spot, perpetuals, funding and custody
- 1.2.7 Fixed income and interest rates; CFDs and leveraged products
1.3 · Market data sources
- 1.3.1 Types of market data: EOD, bars, ticks, order book and reference
- 1.3.2 Free market data sources
- 1.3.3 Retail and prosumer market data APIs
- 1.3.4 Professional market data providers
- 1.3.5 Crypto and on-chain data
- 1.3.6 Brokers as market data sources
- 1.3.7 Market data licensing and terms of use
- 1.3.8 How to choose a market data provider
1.4 · Financial data engineering
- 1.4.1 Downloading over REST and websockets: pagination, rate limits and retries
- 1.4.2 Data cleaning: missing values, bad ticks, gaps and time zones
- 1.4.3 Adjusting for splits and dividends: adjusted and unadjusted prices
- 1.4.4 Universes without survivorship bias: delistings, mergers and symbol changes
- 1.4.5 Building bars: time, tick, volume, dollar and imbalance
- 1.4.6 Continuous futures series: roll rules, difference adjustment and ratio adjustment
- 1.4.7 Storage: CSV, Parquet, partitioning, DuckDB, TimescaleDB and ArcticDB
- 1.4.8 Reproducible pipeline: download, validation, incremental updates and quality tests
1.5 · Fundamental and macro data
1.6 · Financial exploratory analysis
1.7 · Programmed technical analysis
An order passes through many hands
When a retail investor taps “buy” in their broker's app, the order does not go straight “to the exchange”. It travels through a chain of intermediaries, and each one adds a function, a risk and a cost. Knowing the chain explains many of the odd things that show up in the data: trades at half-cent prices, volume that is not in the book, prices that arrive out of order.
Mermaid source
flowchart TB I[Investor] --> B[Broker] B --> S[SOR router] B --> SI[Internalizer] S --> R[Exchange] S --> M[MTF] S --> O[Dark pool] R --> C[CCP clearing house] M --> C O --> C C --> D[CSD depository]
- Investor. Decides what to buy or sell. It can be a person, a fund or a program.
- Broker. The authorized firm that receives the order, checks that there is enough cash or collateral and sends it for execution. In the EU it is an investment firm or a bank; it is obliged to seek best execution for its client (price, cost, speed, likelihood of execution).
- Router (SOR, smart order router). The broker's program that decides which venue to send the order to, or how to split it among several, depending on where the best price and liquidity are at that moment.
- Execution venues. The places where the order is matched with someone else's: exchanges, MTFs, dark pools (all multilateral) or an internalizer (bilateral). The next section tells them apart.
- Clearing house (CCP, central counterparty). After the trade it steps in between buyer and seller: it becomes the buyer to every seller and the seller to every buyer. If one party defaults, the clearing house performs. To cover that risk it requires collateral (margins) from its members.
- Central securities depository (CSD). Keeps the record of who holds each security and delivers against payment: settlement, one or two business days later (lesson 1.1.5).
Who is on the other side
Every purchase has a sale on the other side. Knowing who is usually on the other side helps you understand why a market behaves the way it does. The figure at the top places the participants on two logarithmic axes: how long they hold a position and how large their orders are. These are orders of magnitude, not measurements, but the distances are real: between a market maker and a pension fund there are twelve orders of magnitude of horizon.
| Participant | What they seek | Horizon | How they make (or lose) money |
|---|---|---|---|
| Retail investor | Return on their own money | Minutes to years | From the direction of the price; pays the spread and commissions |
| Asset manager, pension fund | Track an index or beat it with a lot of capital | Months to years | From direction and selection; their problem is moving a lot of volume without moving the price |
| Quant fund, CTA | Exploit statistical regularities | Days to months | From the signal net of costs; suffers when the regularity disappears |
| Market maker | Quote bid and ask continuously | Seconds | From the spread; loses when trading against someone better informed (adverse selection) |
| HFT and arbitrageur | Exploit price differences between venues or products | Microseconds to minutes | From speed and scale; a tiny margin multiplied by millions of trades |
| Broker | Execute on behalf of the client | — | From commissions, interest and, sometimes, order flow |
| Clearing house and depository | Guarantee and settle | — | From fees; their business is that nothing fails |
Two consequences for a trading system. First: almost all immediate liquidity is provided by market makers who do not want to hold positions; if their risk rises (volatility, news), they pull their orders and the spread widens (lesson 1.1.3). Second: a low-frequency system competes on its horizon with asset managers and CTAs, not with HFT. Speed is neither its edge nor its problem; costs and look-ahead bias are.
Where trading happens: exchanges, MTFs, dark pools and internalizers
European regulation (MiFID II and MiFIR) recognizes three types of multilateral trading venue, where many buyers and sellers are matched with each other under non-discretionary rules, and one bilateral figure, the internalizer. Outside all of them lies OTC.
| Type | Who is matched | Pre-trade transparency | Examples of use |
|---|---|---|---|
| Regulated market (exchange) | Third parties with each other, fixed rules | Lit book | Listed stocks, opening and closing auctions |
| MTF | Third parties with each other, fixed rules | Lit book (or blind with a waiver) | Alternative platforms for European stocks |
| OTF | Third parties, with discretionary execution | Depends on the instrument | Only bonds, derivatives, structured products and emission allowances |
| Dark pool | Third parties with each other, no lit book | None, with a waiver | Large orders matched at the midpoint |
| Systematic internalizer | The client against the intermediary's own account | Firm quotes published | Client orders of a large bank or broker |
| OTC | Two parties, bespoke | None | Bespoke derivatives, negotiated blocks |
In Europe a “dark pool” is not a separate legal type: it is a regulated market or an MTF that operates with a pre-trade transparency waiver, for example matching at the reference price (the midpoint of another venue) or matching large-in-scale orders. Since those trades do not contribute to price formation, regulation limits how much volume can be traded that way.
In the US the names change (registered exchanges, ATSs for dark pools, wholesalers that internalize retail orders), but the structure is the same: lit books fragmented across many exchanges and a large share of the volume outside them.
A NOVA session seen by venue
The data for this lesson is a full session of NOVA, a fictitious stock listed in the US, on June 2, 2025: 50,946 trades with their price, size, aggressor side and execution venue. It comes from a book simulated order by order, so trades, quotes and book are consistent with each other. The data is synthetic, but it follows real conventions: a one-cent tick, a session from 9:30 to 16:00 New York time (13:30 to 20:00 UTC) and opening and closing auctions.
import pandas as pd
op = pd.read_parquet("nova_operaciones.parquet")
print(len(op), "trades")
op.head(4)50946 trades
ts precio tamano agresor centro condicion
0 2025-06-02 13:30:00+00:00 53.35 33900 NaN PRINCIPAL subasta_apertura
1 2025-06-02 13:30:00.301051+00:00 53.34 100 venta PRINCIPAL regular
2 2025-06-02 13:30:00.301052+00:00 53.34 100 venta PRINCIPAL regular
3 2025-06-02 13:30:00.407366+00:00 53.34 159 venta PRINCIPAL regularEach row is a trade. agresor says which side took the initiative: the order that arrived and was executed against
one already waiting in the book. In the opening auction it is empty, because an auction has no aggressor (lesson
1.1.2). Grouping by venue:
por_centro = op.groupby("centro").agg(
operaciones=("tamano", "size"),
volumen=("tamano", "sum"),
tamano_medio=("tamano", "mean"),
)
total = por_centro["volumen"].sum()
por_centro["cuota_pct"] = 100 * por_centro["volumen"] / total
por_centro.sort_values("volumen", ascending=False).round(1) operaciones volumen tamano_medio cuota_pct
centro
PRINCIPAL 25776 3564476 138.3 40.1
OSCURO 4386 2358700 537.8 26.5
INTERNALIZADOR 3914 1243861 317.8 14.0
MTF_A 9640 986282 102.3 11.1
MTF_B 7230 735395 101.7 8.3The primary venue, the one with the reference book and the auctions, gathers 40% of the volume. The two MTFs add almost another 20%. And a quarter of the volume is matched in a dark pool: nobody saw it in a book before it happened. NOVA's figures come from a simulation, but the pattern —no venue concentrates all the volume— is that of any liquid stock in the US or in Europe.
The average size tells the other half of the story:
In the venues with a book, a typical trade is about 100 shares: orders executed against what sits at the first level, many of them slices of a larger order split by an algorithm (lesson 1.1.4). In the dark pool, the average size is five times larger. The reason is economic: a large order shown in a lit book moves the price before it is executed, because the others step aside or front-run it. Matching it blind at the midpoint avoids that.
The condicion column separates the special trades:
op.pivot_table(index="centro", columns="condicion",
values="tamano", aggfunc="sum", fill_value=0)condicion bloque regular subasta_apertura subasta_cierre
centro
INTERNALIZADOR 0 1243861 0 0
MTF_A 0 986282 0 0
MTF_B 0 735395 0 0
OSCURO 324900 2033800 0 0
PRINCIPAL 0 2645076 33900 885500Auctions only exist in the primary venue: the opening auction matches little volume and the closing one almost 900,000 shares at a single price, because index funds and many asset managers execute at the close, which is the price they are valued at (lesson 1.1.2). The blocks, all in the dark pool, are a few huge trades:
bloques = op[op["condicion"] == "bloque"]
bloques[["ts", "precio", "tamano", "centro"]] ts precio tamano centro
5478 2025-06-02 13:50:31.017740+00:00 53.375 64100 OSCURO
24035 2025-06-02 16:22:35.166309+00:00 53.795 46100 OSCURO
33823 2025-06-02 17:54:43.496329+00:00 53.255 63000 OSCURO
37493 2025-06-02 18:28:54.725977+00:00 53.195 36800 OSCURO
41959 2025-06-02 19:05:40.906104+00:00 52.655 68200 OSCURO
47010 2025-06-02 19:40:27.200952+00:00 52.570 46700 OSCUROSix trades that add up to 324,900 shares, 3.7% of the day's volume. A rule that filters “abnormal volume” without looking at the condition would take these blocks for signals; they are not: they are institutions moving positions through negotiated trades.
What the dark pool does not show
The dark pool has no visible book, but it does have a price rule: it matches at the midpoint of the best bid and
the best ask of the lit book. Checking it requires joining each trade with the quote prevailing at that instant. The
quotes (nova_cotizaciones.parquet) are the best bid and the best ask of the lit book, one row every time they change:
cot = pd.read_parquet("nova_cotizaciones.parquet")
print(len(cot), "quote changes")
# the prevailing quote is the last one BEFORE the trade
con_cot = pd.merge_asof(op, cot, on="ts", allow_exact_matches=False)
con_cot["medio"] = (con_cot["bid"] + con_cot["ask"]) / 2
con_cot.loc[1:3, ["ts", "precio", "agresor", "centro", "bid", "ask"]]129247 quote changes
ts precio agresor centro bid ask
1 2025-06-02 13:30:00.301051+00:00 53.34 venta PRINCIPAL 53.34 53.36
2 2025-06-02 13:30:00.301052+00:00 53.34 venta PRINCIPAL 53.34 53.36
3 2025-06-02 13:30:00.407366+00:00 53.34 venta PRINCIPAL 53.34 53.36merge_asof is a nearest-key join: for each trade it looks for the last quote with an earlier timestamp. The detail
allow_exact_matches=False matters. When an aggressive order sweeps a level of the book, the quote changes in the
same microsecond as the trade. If the exact match is allowed, that trade is joined with the quote it caused itself:
information from the next instant. It is the look-ahead bias of lesson 0.1.1, now at microsecond scale:
def ejecuta_al_mejor_precio(df):
con_libro = df[df["centro"].isin(["PRINCIPAL", "MTF_A", "MTF_B"])
& (df["condicion"] == "regular")]
lado, precio = con_libro["agresor"], con_libro["precio"]
compra = (lado == "compra") & (precio >= con_libro["ask"])
venta = (lado == "venta") & (precio <= con_libro["bid"])
return (compra | venta).mean()
mal = pd.merge_asof(op, cot, on="ts") # allow_exact_matches=True
print(f"With exact matches: {ejecuta_al_mejor_precio(mal):.2%}")
print(f"Without them: {ejecuta_al_mejor_precio(con_cot):.2%}")With exact matches: 99.91%
Without them: 100.00%With the correct alignment, every aggressive buy pays the ask or more and every sell receives the bid or less, which is how a book works. With the wrong one, a few dozen trades seem to execute at a better price than was available. In an execution cost study (lesson 1.1.5) that error distorts the effective spread.
With the quote properly aligned, the rule of the dark pool and that of the internalizer can be checked:
oscuro = con_cot[con_cot["centro"] == "OSCURO"]
al_medio = (oscuro["precio"] - oscuro["medio"]).abs() < 1e-9
print(f"Dark crosses at the midpoint: {al_medio.mean():.1%}")
inter = con_cot[con_cot["centro"] == "INTERNALIZADOR"]
mejora = (inter["ask"] - inter["precio"]).where(
inter["agresor"] == "compra", inter["precio"] - inter["bid"])
print(f"Internalizer with price improvement: {(mejora > 0).mean():.1%}")
print(f"Average improvement: {100 * mejora.mean():.2f} cents per share")Dark crosses at the midpoint: 100.0%
Internalizer with price improvement: 44.1%
Average improvement: 0.24 cents per shareAll the dark crosses are at the midpoint, as their rule says. The internalizer executes against its own account inside the spread and, in a little under half of the trades, improves on the price of the lit book: that is what it offers in exchange for keeping the order flow. For the client, half a cent of improvement on a $53 stock is little; for the internalizer, multiplied by millions of orders, it is a business.
Mermaid source
flowchart TB
O[Client order] --> T{Is it large?}
T -- yes --> OS[Dark pool or block]
T -- no --> P{Price improvement?}
P -- yes --> IN[Internalizer]
P -- no --> SOR[Lit books]The diagram simplifies the decision of the router and the broker. In practice it is made by algorithms with data from each venue, and the best execution obligation requires being able to justify it.
Clearing and settlement
The trade is only the beginning. Two steps follow that the investor barely sees and that explain some of the data:
Mermaid source
flowchart LR N[Trading] --> C[CCP clearing] --> L[CSD settlement]
- Clearing. The CCP computes each member's net positions (many buys and sells of the same security cancel out) and requires collateral for the outstanding risk.
- Settlement. The CSD delivers the securities and the cash. In the US it happens on the next business day (T+1) since May 28, 2024; in the EU, two days later (T+2), with the move to T+1 scheduled for October 11, 2027.
For a trading system, settlement matters in two places: the cash from a sale is not available until it settles (in a cash account), and the date that determines who receives a dividend is the settlement date, which is why the ex date exists (lessons 1.1.5 and 1.2.1).
What this changes in your data
- Closing price ≠ last trade. The official close is that of the primary venue's auction; later trades on other venues or after hours do not change it.
- The day's volume is split. A provider that only gives the primary exchange's volume (some do) shows 40% of NOVA's real volume. Before comparing volumes, you need to know which tape each source uses (lesson 1.3.1).
- Not all trades are equal. Auctions, blocks and after-hours trades carry condition codes. An indicator such as VWAP (lesson 1.7.1) gives different results with and without them.
- The book is not all the liquidity. What you see at the first level is part of the real interest; the dark pool and the internalizers match without going through it.
Exercises
- Compute each venue's volume share using only the trades with condition
regular, without auctions or blocks. Which venue gains weight and why?
Solution
regular = op[op["condicion"] == "regular"]
por_centro_reg = regular.groupby("centro")["tamano"].sum()
cuota = 100 * por_centro_reg / regular["tamano"].sum()
cuota.sort_values(ascending=False).round(1)centro
PRINCIPAL 34.6
OSCURO 26.6
INTERNALIZADOR 16.3
MTF_A 12.9
MTF_B 9.6
Name: tamano, dtype: float64Without the auctions, the primary venue loses more than five points of share: the closing auction, almost a million shares at a single price, only exists there. All the others gain weight, above all the internalizer. The share “of the primary venue” depends a lot on whether auctions are counted or not, so it is worth always stating which one it is.
- Compare the mean and the median trade size of each venue. What does the difference between the two tell you in the dark pool?
Solution
op.groupby("centro")["tamano"].agg(["mean", "median"]).round(0) mean median
centro
INTERNALIZADOR 318.0 236.0
MTF_A 102.0 100.0
MTF_B 102.0 100.0
OSCURO 538.0 300.0
PRINCIPAL 138.0 100.0In the venues with a book, mean and median are similar: almost all trades are small. In the dark pool, the mean is much larger than the median: the distribution has a long tail of large trades (the blocks) that pull the mean up. To describe a venue's typical size, the median is more robust; to measure how much volume it moves, the sum.
- A system buys 2,000 NOVA shares with a market order at 15:00 New York time. Explain on which venues it could be executed, who would probably be on the other side and which data from this lesson you would use to estimate its cost.
Solution
The broker's router would send it to the lit books (primary venue and MTFs), where it would sweep one or several
levels, or it would split it and look for liquidity in the dark pool and in an internalizer. On the other side there
would mostly be market makers in the lit books, the internalizer itself if it takes the order, and perhaps an asset
manager in the dark pool. To estimate the cost you would need the spread and the depth of the book at that time
(nova_libro_l2.parquet, lesson 1.1.3) and the effective spread of similar trades (nova_operaciones with
nova_cotizaciones, lesson 1.1.5).
Each answer is folded under its question
Self-assessment
A retail buy order is executed at $53.345 when the best bid is 53.34 and the best ask 53.35. Where was it most likely matched?
- AIn the book of the primary exchange, against the ask
- BIn the closing auction
- CIn a dark pool, at the midpoint
- DIt cannot have been executed at that price
Show answer
Answer C. In a dark pool, at the midpoint
A half-cent price, exactly at the midpoint of the spread, is the typical signature of a dark cross pegged to a reference price. In the lit book an aggressive buy pays the ask (53.35).
What distinguishes a systematic internalizer from an MTF?
- AThe internalizer publishes nothing
- BThe MTF only trades bonds
- CNothing: they are two names for the same thing
- DThe internalizer is an investment firm that executes client orders against its own account; the MTF is a multilateral system that matches third parties with each other
Show answer
Answer D. The internalizer is an investment firm that executes client orders against its own account; the MTF is a multilateral system that matches third parties with each other
The MTF (SMN in Spanish terminology) is a multilateral trading venue. The systematic internalizer is not a trading venue: it is the counterparty of its clients, with its own quoting and publication obligations.
What does the central counterparty (CCP) do after a trade?
- AIt steps in between buyer and seller and guarantees settlement, requiring collateral from its members
- BIt decides the price of the trade
- CIt holds the end investor's securities
- DIt sends the order to the venue with the best price
Show answer
Answer A. It steps in between buyer and seller and guarantees settlement, requiring collateral from its members
The CCP becomes the buyer to every seller and the seller to every buyer (novation) and manages the risk with margins. Custody and delivery of securities are handled by the central securities depository (CSD); routing, by the broker.
When joining each trade with the prevailing quote, why
allow_exact_matches=False?- ATo make the code faster
- BBecause a quote with the same timestamp as the trade may be the one that trade has just caused: using it would be information from the next instant
- CBecause pandas does not support exact matches
- DTo remove the auctions
Show answer
Answer B. Because a quote with the same timestamp as the trade may be the one that trade has just caused: using it would be information from the next instant
When an aggressive order sweeps a level, the book changes in the same microsecond. The “prevailing” quote for that trade is the previous one. In the NOVA session, with exact matches 0.1% of the trades against the book seem to execute outside the best price, and without them none do.
In the NOVA session the average volume per trade in the dark pool is about five times that of the MTFs. Why does that make sense?
- ABecause only retail investors trade in the dark pool
- BBecause large orders try not to show their size in a lit book, where they would move the price before being executed
- CBecause the dark pool charges lower commissions per share
- DBy chance
Show answer
Answer B. Because large orders try not to show their size in a lit book, where they would move the price before being executed
Whoever needs to move a large block loses money by announcing it: the others step aside or front-run it. That is why the dark pool concentrates large, institutional orders; in the NOVA session it holds the six blocks of more than 36,000 shares.
Key points
- An order goes through broker, router, execution venue, clearing house and depository. Each link has its function and its cost.
- Participants differ by horizon and size: from the market maker that holds positions for seconds to the pension fund that holds them for years.
- Trading is fragmented: in the NOVA session the primary venue only gathers 40% of the volume; a quarter is matched in a dark pool, at the midpoint and with no visible book.
- The book you see is not all the liquidity: large orders, blocks and internalizers execute outside it.
- To align trades and quotes you have to use the immediately preceding quote; the one at the same instant may be a consequence of the trade.