Crypto and on-chain data.

Three layers of data

In equities there is a main exchange, a consolidated tape and a regulator that requires every trade to be published (lesson 1.1.1). Not in crypto: each centralized exchange (CEX) is at once exchange, broker and, often, custodian, and it publishes its own prices through its own API. Above them sit aggregators that combine many exchanges, and below, the blockchain, which records the actual transfers. These are three layers that answer different questions:

flowchart TB
  C[Blockchain] --> N[On-chain provider]
  C --> X[CEX exchanges]
  X --> A[Aggregators]
  X --> T[Your program]
  A --> T
  N --> T

For a system that trades on an exchange, the price source is that exchange: it is where execution happens. The aggregator is useful for comparison and for the universe (which assets exist, how large their market cap is); on-chain data, for variables that are not in the price.

ccxt: one API for more than a hundred exchanges

Each exchange has its own API, with its own names, formats and limits. ccxt is an open-source library (Python, JavaScript, PHP…) that unifies them: the same method returns the same format on all of them. Its documentation mentions more than a hundred supported exchanges11Note 1 · ccxt documentation, Manual: sections “OHLCV Candlestick Charts” and “Rate Limit” (docs.ccxt.com), accessed on October 21, 2026.. It is not part of the course environment because it needs network access; typical usage is:

import ccxt

exchange = ccxt.binance()            # enableRateLimit=True by default
exchange.load_markets()              # markets and unified symbols
print(exchange.has["fetchOHLCV"], list(exchange.timeframes)[:5])

desde = exchange.parse8601("2025-12-01T00:00:00Z")   # milliseconds
velas = exchange.fetch_ohlcv("BTC/USDT", "1h", since=desde, limit=500)

Symbols are unified: BTC/USDT is bitcoin spot against USDT on any exchange, and a linear perpetual is written BTC/USDT:USDT (base, quote and settlement currency). has["fetchOHLCV"] tells whether the exchange provides candles and timeframes, which intervals it accepts. fetch_ohlcv(symbol, timeframe, since, limit) returns a list of candles, from oldest to most recent, each one as [timestamp in ms UTC, open, high, low, close, volume]¹.

Reading the fetch_ohlcv format

xsn_ccxt_ohlcv.json is a response in that format: the last 200 hourly candles of 2025 for XSN, the course's fictional token. It is a format example, not a real download, but it is processed exactly like one from an exchange:

import json
from pathlib import Path

import pandas as pd

velas = json.loads(Path("xsn_ccxt_ohlcv.json").read_text())
print(type(velas).__name__, len(velas), "candles")
velas[:2]

list 200 candles
[[1766502000000, 47.1217, 47.298, 46.9181, 46.9485, 115011.657], [1766505600000, 46.9485, 47.3096, 46.913, 47.2735, 156031.363]]

A list of lists, with no column names: the order is the specification. The first number is the candle's open time in milliseconds since 1970 in UTC. Converting it with unit="s" would give dates in the year 57,000; forgetting utc=True would give dates without a time zone, which would mix badly with any other series (lesson 1.4.2).

cols = ["ts", "apertura", "maximo", "minimo", "cierre", "volumen"]
df = pd.DataFrame(velas, columns=cols)
df["ts"] = pd.to_datetime(df["ts"], unit="ms", utc=True)
df = df.set_index("ts")
print(df.index[0], "->", df.index[-1])
df.tail(3)

2025-12-23 15:00:00+00:00 -> 2025-12-31 22:00:00+00:00
                           apertura   maximo   minimo   cierre     volumen
ts
2025-12-31 20:00:00+00:00   48.7692  48.8314  48.1571  48.1701  142982.716
2025-12-31 21:00:00+00:00   48.1701  48.2427  48.1480  48.1839  183139.453
2025-12-31 22:00:00+00:00   48.1839  48.2721  48.1744  48.2534  106840.362

Volume comes in units of the base asset (XSN), not in dollars. To compare volumes across assets, or with an aggregator that reports volume in USD, you have to multiply by the price, and that is already an estimate: each trade in the candle had its own price.

Validate before saving

A response that parsed correctly can still be wrong. Three cheap checks worth running on every download:

pasos = df.index.to_series().diff().dropna().value_counts()
print("intervals:", pasos.to_dict())

cuerpo_max = df[["apertura", "cierre"]].max(axis=1)
cuerpo_min = df[["apertura", "cierre"]].min(axis=1)
ok = (df["maximo"] >= cuerpo_max) & (df["minimo"] <= cuerpo_min)
print("consistent OHLC candles:", ok.all())

ap, ci = df["apertura"].to_numpy(), df["cierre"].to_numpy()
seguida = ap[1:] == ci[:-1]
print(f"open = previous close: {seguida.mean():.0%}")

intervals: {Timedelta('0 days 01:00:00'): 199}
consistent OHLC candles: True
open = previous close: 100%

All intervals are one hour, no candle has its high below the body and, in this data, each candle opens at the previous one's close. The last point is typical of a continuous 24/7 market such as crypto; in equities it does not hold from one day to the next (there are auctions and gaps), and on a real exchange it can fail because of a stray trade between two candles.

The fourth check is the most often forgotten: the open candle. Most exchanges also return the current candle, with its close, high, low and volume still changing. If the download runs at 22:30, the 22:00 candle has not finished:

def quitar_vela_abierta(df, ahora, intervalo="1h"):
    fin = df.index + pd.Timedelta(intervalo)
    return df[fin <= ahora]

ahora = pd.Timestamp("2025-12-31 22:30", tz="UTC")
cerradas = quitar_vela_abierta(df, ahora)
print(len(df), "candles;", len(cerradas), "closed at", ahora)
print("last closed:", cerradas.index[-1])

200 candles; 199 closed at 2025-12-31 22:30:00+00:00
last closed: 2025-12-31 21:00:00+00:00

Saving the open candle as final creates a history nobody else can reproduce: whoever downloads it tomorrow will see a different close and a different volume. In a backtest, moreover, it is partial information treated as complete.

Resampling: 4-hour candles

With the DataFrame in UTC time, changing the interval is a resample. The catch is the same as with the open candle: the groups at the edges may be incomplete. The first candle in the file starts at 15:00, so the 12:00 to 16:00 group has only one hour:

reglas = {"apertura": "first", "maximo": "max", "minimo": "min",
          "cierre": "last", "volumen": "sum"}
grupos = df.resample("4h")
v4 = grupos.agg(reglas)
v4["horas"] = grupos["cierre"].count()
print(v4["horas"].value_counts().sort_index().to_dict())
v4 = v4[v4["horas"] == 4].drop(columns="horas")
print(len(v4), "complete 4 h candles")
v4.round(2).tail(3)

{1: 1, 3: 1, 4: 49}
49 complete 4 h candles
                           apertura  maximo  minimo  cierre    volumen
ts
2025-12-31 08:00:00+00:00     48.17   48.77   48.04   48.48  709687.91
2025-12-31 12:00:00+00:00     48.48   48.68   48.33   48.36  539446.43
2025-12-31 16:00:00+00:00     48.36   48.81   48.32   48.77  701424.06

The figure at the top plots the result. Instead of 51 groups, 49 complete 4-hour candles remain: the ones at the edges had 1 and 3 hours. resample groups by the start of the interval (with 4h, pandas aligns the groups with UTC midnight), which is also the convention of ccxt and of most exchanges. Not every provider follows it, as you will see below with CoinGecko.

Paginating with since and limit

An exchange does not return years of candles in a single request: each response has a maximum number of candles (limit), which depends on the exchange. To download a long period you have to request it in pages, advancing since past the last candle received, and wait between requests so as not to exceed the limit. In ccxt, exchange.rateLimit is the pause in milliseconds that the internal limiter keeps between requests (multiplied by the “cost” of each call), and the limiter is enabled by default¹.

flowchart TB
  S[since = start] --> P[fetch_ohlcv]
  P --> V{New candles?}
  V -- no --> F[End]
  V -- yes --> G[Save]
  G --> A[since = last+1h]
  A --> E[Wait rateLimit]
  E --> P

So that the loop really runs without network access, a simulated exchange serves the 200 candles in the file with the same rules: at most 60 candles per call, candles from since onwards and a rateLimit of 100 ms, which here is added to a virtual clock instead of sleeping:

class ExchangeSimulado:
    rateLimit = 100          # ms between requests, as in ccxt
    max_velas = 60

    def __init__(self, velas):
        self.velas = velas
        self.llamadas = 0
        self.reloj_ms = 0

    def fetch_ohlcv(self, symbol, timeframe="1h", since=None,
                    limit=None):
        self.llamadas += 1
        self.reloj_ms += self.rateLimit
        n = min(limit or self.max_velas, self.max_velas)
        nuevas = [v for v in self.velas
                  if since is None or v[0] >= since]
        return nuevas[:n]

def descargar(ex, symbol, since, paso_ms=3_600_000):
    todas = []
    while True:
        lote = ex.fetch_ohlcv(symbol, "1h", since=since, limit=1000)
        if not lote:
            break
        todas += lote
        since = lote[-1][0] + paso_ms
    return todas

ex = ExchangeSimulado(velas)
bajadas = descargar(ex, "XSN/USDT", since=velas[0][0])
print(len(bajadas), "candles in", ex.llamadas, "calls,",
      ex.reloj_ms, "ms of waiting")
print("identical to the original:", bajadas == velas)

200 candles in 5 calls, 500 ms of waiting
identical to the original: True

Four calls with data (60 + 60 + 60 + 20 candles) and a fifth one that comes back empty and ends the loop. Although 1,000 candles were requested per call, the exchange returned only 60: the code must not assume it receives what it asks for, but advance from what it receives. Two frequent mistakes in this loop:

• Advancing since by adding limit × interval instead of starting from the last candle received. If the exchange returns fewer candles than requested, a stretch is skipped without noticing.
• Not deduplicating. If since is set to the last candle and not the next one, each page repeats a candle. With + paso_ms it does not happen; even so, df[~df.index.duplicated()] at the end is a good idea.

At real scale, 200 candles are minutes of waiting; a year of 1-minute candles is 525,600 candles, hundreds of calls per symbol. The rate limit determines how long a historical download takes, and exceeding it usually ends in a “too many requests” error or a temporary IP ban. Lesson 1.4.1 covers retries with increasing backoff.

Gaps: the exchange stops too

A 24/7 market does not guarantee 24/7 candles. Exchanges have maintenance windows, outages and, on illiquid pairs, hours with no trades. Each exchange decides what to return then: some do not return the candle; others, a candle with zero volume. To simulate it, three hours are removed from the download:

con_hueco = df.drop(df.index[100:103])
rejilla = pd.date_range(con_hueco.index[0], con_hueco.index[-1],
                        freq="1h")
faltan = rejilla.difference(con_hueco.index)
print(len(con_hueco), "candles;", len(faltan), "gaps:")
print(list(faltan.strftime("%Y-%m-%d %H:%M")))

197 candles; 3 gaps:
['2025-12-27 19:00', '2025-12-27 20:00', '2025-12-27 21:00']

Comparing against the complete grid is the reliable way to find gaps; counting rows does not tell you where they are. What to do with them is a decision that has to be documented: leave them as missing, or fill them with a flat candle (open = high = low = close = previous close, volume 0) if the exchange was open and there simply were no trades. If it was down, the flat candle is false: nobody could trade at that price. Lesson 1.4.2 covers the full cleaning process.

There is no single XSN price: there is one per exchange

Each exchange has its own book and its own participants, so the price of the same token differs slightly from one venue to another. That is why perpetuals compute funding and liquidations against a composite index of several exchanges (lesson 1.2.6). xsn_1h.parquet contains one exchange's spot and that index:

xsn = pd.read_parquet("xsn_1h.parquet").set_index("ts")
dif_pb = 1e4 * (xsn["spot"] / xsn["indice"] - 1)
dif_pb.describe().round(2)[["mean", "std", "min", "max"]]

mean    -0.07
std      4.00
min    -15.62
max     14.80
dtype: float64

In the course data, the spot deviates from the index by about 4 basis points of standard deviation, with extremes of about 15. That is little for a daily system, but it is of the order of an exchange's commission and a lot for an intraday one. The practical consequence: a backtest must use the prices of the exchange where it will trade, and a signal computed with an aggregator's price may not be executable at that price on any specific exchange.

Aggregators: CoinGecko, CoinMarketCap and Kaiko

Aggregators collect prices and volumes from many exchanges and combine them. They are useful for building a universe (which tokens exist, their market cap, which exchanges list them) and for reference prices, not for execution.

• CoinGecko has a free Demo plan with a key, with 100 calls per minute and 10,000 credits per month, and one year of history, according to its pricing page and its documentation as of October 21, 202622Note 2 · CoinGecko API: pricing, errors and rate limits and Coin OHLC Chart by ID, accessed on October 21, 2026..
• CoinMarketCap has a free Basic plan with 15,000 credits per month; that plan includes one year of daily historical quotes, but not historical OHLCV candles, which start with paid plans33Note 3 · CoinMarketCap API Pricing Plans, accessed on October 21, 2026..
• Kaiko targets institutional clients (trades and order books from more than a hundred exchanges, reference rates and indices) and does not publish prices: it offers custom plans44Note 4 · Kaiko: Pricing and Licenses and Level 1 and Level 2 Data, accessed on October 21, 2026..

Each aggregator has its own format, and it does not always match that of the exchanges. CoinGecko's candle endpoint (/coins/{id}/ohlc) returns five values per candle, with no volume, and its timestamp marks the close of the candle, not the open. It also picks the granularity based on the number of days requested (30 minutes for 1 or 2 days, 4 hours up to 30 days, 4 days beyond that)². With an illustrative sample in that format:

import os

import requests

r = requests.get(
    "https://api.coingecko.com/api/v3/coins/bitcoin/ohlc",
    params={"vs_currency": "usd", "days": 7},
    headers={"x-cg-demo-api-key": os.environ["CG_DEMO_KEY"]},
    timeout=10,
)
r.raise_for_status()
ohlc = r.json()    # [[close ms, open, high, low, close]]

muestra_cg = [   # illustrative sample in the documented format
    [1767225600000, 48.37, 48.83, 48.15, 48.25],
    [1767240000000, 48.25, 48.41, 47.90, 48.02],
]
cg = pd.DataFrame(muestra_cg, columns=cols[:5])
cg["cierre_ts"] = pd.to_datetime(cg["ts"], unit="ms", utc=True)
cg["apertura_ts"] = cg["cierre_ts"] - pd.Timedelta("4h")
cg[["apertura_ts", "cierre_ts", "cierre"]]

                apertura_ts                 cierre_ts  cierre
0 2025-12-31 20:00:00+00:00 2026-01-01 00:00:00+00:00   48.25
1 2026-01-01 00:00:00+00:00 2026-01-01 04:00:00+00:00   48.02

If ccxt candles (open time) are mixed with CoinGecko candles (close time) without converting, one of the two series ends up shifted by a whole interval. It is a textbook look-ahead bias: CoinGecko's “04:00” candle contains prices that, under the ccxt convention, belong to 00:00–04:00.

On-chain data

The blockchain is public: anyone with a node can read every block, every transaction, every address and its balance. From that, on-chain providers compute metrics: active addresses, transferred volume, exchange inflows and outflows, realized profit and loss. Glassnode, for example, documents more than 800 metrics, among them MVRV (market capitalization divided by “realized” capitalization) and SOPR (the degree of realized profit or loss of the coins being moved); full access to its API is in the Professional plan, and the Advanced plan only has a limited API55Note 5 · Glassnode: documentation and pricing, accessed on October 21, 2026..

The chain says which addresses moved which amount, but not who owns them. A metric such as “exchange inflows” depends on which addresses the provider has labeled as belonging to exchanges. A small, fictional example with four transfers:

transfer = pd.DataFrame({
    "origen":  ["w1", "w2", "ex_a", "w3"],
    "destino": ["ex_a", "ex_b", "w4", "w5"],
    "xsn":     [5_000, 3_000, 2_000, 4_000],
})

def flujo_neto(t, etiquetas_exchange):
    entra = t["destino"].isin(etiquetas_exchange)
    sale = t["origen"].isin(etiquetas_exchange)
    return int(t.loc[entra, "xsn"].sum() - t.loc[sale, "xsn"].sum())

print("labeling v1:", flujo_neto(transfer, {"ex_a"}))
print("labeling v2:", flujo_neto(transfer, {"ex_a", "ex_b", "w5"}))

labeling v1: 3000
labeling v2: 10000

The same four transfers give a net flow of 3,000 XSN into exchanges with one labeling and 10,000 with another, which has identified one more exchange address (ex_b) and another one the first did not know about (w5). When the provider improves its labeling, it usually recomputes the history: the series downloaded a year ago is no longer today's. For a backtest it is the same problem as revisions of macro data (lesson 1.5.2): you need to know whether the metric is the one that was known on each date or the one revised afterwards.

Exercises

1. With df, build daily candles (UTC) and keep only the complete days. How many remain and why are they not 9?

g = df.resample("1D")
diarias = g.agg(reglas)
diarias["horas"] = g["cierre"].count()
print(diarias["horas"].tolist())
diarias = diarias[diarias["horas"] == 24]
print(len(diarias), "complete days")

[9, 24, 24, 24, 24, 24, 24, 24, 23]
7 complete days

2. Change ExchangeSimulado.max_velas to 50 and download again. How many calls are needed? What would happen if descargar advanced since by 1000 × 1 h instead of from the last candle?

ExchangeSimulado.max_velas = 50
ex = ExchangeSimulado(velas)
bajadas = descargar(ex, "XSN/USDT", since=velas[0][0])
print(len(bajadas), "candles in", ex.llamadas, "calls")

def descargar_mal(ex, since, limit=1000, paso_ms=3_600_000):
    todas = []
    while lote := ex.fetch_ohlcv("XSN/USDT", "1h", since, limit):
        todas += lote
        since += limit * paso_ms
    return todas

print("advancing by limit:", len(descargar_mal(ex, velas[0][0])))
ExchangeSimulado.max_velas = 60

200 candles in 5 calls
advancing by limit: 50

3. You are going to build a system that trades the XSN perpetual on an exchange. Say which layer you would take the signal price, the funding and the token universe from, and what you would check for each source.