Chunking financial documents: tables, footnotes and XBRL.

Why chunking decides what gets retrieved

A RAG system (retrieval-augmented generation) has two halves: a retriever that picks a few pieces of the documents and a model that writes the answer with them. Lesson 6.3.1 covers the retriever half (embeddings, vector indexes, BM25); this one covers what comes before: how each document is split into pieces (chunks). It is the least glamorous decision in the system and one of the most consequential, because the model only sees what the retriever hands it.

flowchart TB
  D[Documents] --> T[Chunking]
  T --> I[Index]
  P[Question] --> B[Top-k search]
  I --> B
  B --> C[Context]
  C --> L[LLM]
  L --> R[Answer]

Financial documents punish bad chunking in particular:

• Tables. The key figures (sales, debt, the ongoing charges of an ETF) live in tables. Cut in half, a table loses the header row that says what each column is, or the row the question asked for ends up in the neighboring chunk.
• Footnotes. “Adjusted EBITDA¹” means nothing without “¹ excludes restructuring costs of 74.3 million”. In a report the note can sit right below the table or several pages away.
• Templates. Two years of reports from the same company, or the prospectuses of five ETFs from the same manager, are almost identical. A chunk “Ventas | 13.668,9 | 13.378,6” (sales, in Spanish number format) does not say which company or year it belongs to.
• Data that is already structured. The financial statement figures of US companies exist as tagged XBRL facts, with their period and filing date. Looking for them in the text is a detour.

The lesson compares three chunking strategies with an objective measure (recall@k against reference passages) and ends with the structured alternative: parsed HTML and XBRL facts.

The material: documents, canonical passages and gold questions

The level 6 data include 21 synthetic documents in Spanish and in Markdown: the 2023 and 2024 annual reports of four fictitious Spanish issuers, the prospectus and KID of five fictitious UCITS ETFs, and three research notes. The example corpus is in Spanish, and so are the questions, so the fragments quoted below carry an English gloss in parentheses. What makes the documents useful for measuring is the canonical segmentation: 974 passages with their type (heading, text, table, note, list) and their exact position in characters, and 263 questions with the passages that must be cited to answer them.

import re
import unicodedata

import numpy as np
import pandas as pd

docs = pd.read_parquet("rag6_documentos.parquet")
pas = pd.read_parquet("rag6_pasajes.parquet")
preg = pd.read_csv("rag6_preguntas.csv")
TEXTO = dict(zip(docs.id_doc, docs.texto))
TITULO = dict(zip(docs.id_doc, docs.titulo))
P = pas.set_index("id_pasaje")

print(docs.tipo.value_counts().to_dict())
print(pas.tipo.value_counts().to_dict())
print(preg.tipo_pregunta.value_counts().to_dict())
r = P.loc["R01-p013"]
print(r.id_doc, r.seccion, r.tipo, r.inicio, r.fin)
print(TEXTO["R01"][r.inicio:r.fin][:150])

{'informe_anual': 8, 'folleto_etf': 5, 'kid': 5, 'research': 3}
{'texto': 431, 'titulo': 252, 'lista': 119, 'tabla': 105, 'nota': 67}
{'texto': 122, 'tabla': 64, 'no_respondible': 17, 'texto_y_nota': 16, 'nota_pie': 16, 'tabla_y_nota': 13, 'calculo': 8, 'comparacion': 7}
R01 3. Magnitudes principales tabla 1577 1914
| (millones de USD) | 2023 | 2022 |
|---|---|---|
| Ventas | 13.668,9 | 13.378,6 |
| EBITDA ajustado¹ | 3.769,6 | 3.222,8 |
| Resultado neto | 2.442,8

texto[inicio:fin] is the passage: with those offsets you can check, for any chunk of your own, which reference passages it contains and in what proportion. Questions of type texto_y_nota (text and note) and tabla_y_nota (table and note) need two passages, and some comparison questions need five (one per ETF). Question QA002 shows it (how much in lease liabilities Busmar Hogar had at the end of 2023 and whether they are included in net debt; gold answer: 369.2 million dollars, not included): the figure is in note 1 of the financial statements and the qualification in note ² of the key-figures table, many lines earlier.

q = preg.set_index("id_pregunta").loc["QA002"]
print(q.pregunta, "|", q.respuesta_oro)
for pid in q.pasajes_oro.split(";"):
    r = P.loc[pid]
    print(pid, r.inicio, r.seccion[:28], "|",
          TEXTO[r.id_doc][r.inicio:r.fin][:70])

¿Cuántos pasivos por arrendamiento tiene Busmar Hogar al cierre de 2023 y se incluyen en la deuda neta? | 369,2 millones de dólares; no se incluyen
R01-p044 5658 Notas seleccionadas a los es | Nota 1. Arrendamientos. El grupo reconoce pasivos por arrendamiento de
R01-p015 2010 3. Magnitudes principales | ² Deuda financiera neta = deuda financiera bruta − caja y equivalentes

Three ways to chunk

The three strategies produce the same thing: a list of chunks, each with its document and its offset spans (inicio, fin) (start, end). Storing the offsets, and not only the text, is what makes it possible to measure and, later, to cite.

Fixed size. Windows of 500 characters with an overlap of 50. It is the simplest option and knows nothing about the document: it cuts wherever the boundary falls, including in the middle of a row or a figure.

def fijo(doc, n=500, solape=50):
    """Windows of n characters that overlap by `solape`."""
    t, trozos, i = TEXTO[doc], [], 0
    while True:
        f = min(i + n, len(t))
        trozos.append(dict(doc=doc, tramos=[(i, f)]))
        if f == len(t):
            return trozos
        i = f - solape


t = TEXTO["R01"]
r = P.loc["R01-p013"]                  # the key-figures table
for c in fijo("R01"):
    (i, f), = c["tramos"]
    if i < r.fin and f > r.inicio:
        print((i, f), repr(t[max(i, r.inicio):max(i, r.inicio) + 40]))

(1350, 1850) '| (millones de USD) | 2023 | 2022 |\n|---'
(1800, 2300) 'neto | 13.069,0 | 11.890,0 |\n| Dividendo'

The key-figures table ends up split between two windows, and the second one starts in the middle of a row: that chunk has figures without the column header.

Recursive. The idea behind the “recursive” splitters of LangChain or LlamaIndex: try to split on the coarsest separator (a blank line); if a piece is still larger than the maximum, split it on the next one (line break, period, space), and then merge neighboring pieces back while they fit. It respects paragraphs and rows when it can, but it does not know what a table is or which section each chunk belongs to.

SEPARADORES = ["\n\n", "\n", ". ", " "]


def cortes(t, i, f, seps, n):
    """Spans of at most n characters, using the coarsest
    separator that works."""
    if f - i <= n:
        return [(i, f)]
    if not seps:
        return [(k, min(k + n, f)) for k in range(i, f, n)]
    s, bordes = seps[0], [i]
    k = t.find(s, i, f)
    while k != -1:
        bordes.append(k + len(s))
        k = t.find(s, k + len(s), f)
    bordes.append(f)
    piezas = []
    for a, b in zip(bordes[:-1], bordes[1:]):
        if b > a:
            piezas += cortes(t, a, b, seps[1:], n)
    unidos = []                        # merge neighbors while they fit
    for a, b in piezas:
        if unidos and b - unidos[-1][0] <= n:
            unidos[-1] = (unidos[-1][0], b)
        else:
            unidos.append((a, b))
    return unidos


def recursivo(doc, n=500):
    t, trozos = TEXTO[doc], []
    for a, b in cortes(t, 0, len(t), SEPARADORES, n):
        a += len(t[a:b]) - len(t[a:b].lstrip())    # trim whitespace
        b -= len(t[a:b]) - len(t[a:b].rstrip())
        if b > a:
            trozos.append(dict(doc=doc, tramos=[(a, b)]))
    return trozos


print(len(fijo("R01")), len(recursivo("R01")))

23 25

Structure-aware. It uses what the document says about itself: the blocks separated by blank lines, the ## headings, the tables (lines starting with |) and the note marks (¹ ² ³…). Rules:

1. Headings are not chunks: they set the section of what follows.
2. A table is always one whole chunk, even if it exceeds the maximum.
3. A block that cites a note (¹, ²…) is joined to the note: the chunk has two offset spans.
4. Consecutive paragraphs of the same section are merged up to 800 characters.

flowchart TB
  B[Block] --> H{Heading?}
  H -->|yes| S[Sets the section]
  H -->|no| N{Is it a note?}
  N -->|yes| X[Waits for its citation]
  N -->|no| T{Table or citation?}
  T -->|yes| U[Chunk with its note]
  T -->|no| J[Merged with the previous one]

NOTA = "¹²³⁴⁵⁶⁷⁸⁹"


def bloques(t):
    """Paragraphs, tables, headings and notes: whatever is
    separated by blank lines."""
    return [(m.start(), m.end())
            for m in re.finditer(r"[^\n](?:.|\n(?!\n))*", t)]


def seccion(doc, i):
    """Last ## heading before position i."""
    hs = re.findall(r"^##+ (.*)$", TEXTO[doc][:i + 1], re.M)
    return hs[-1] if hs else ""


def estructura(doc, n=800):
    t = TEXTO[doc]
    bl = bloques(t)
    notas = {t[a]: (a, b) for a, b in bl if t[a] in NOTA}
    trozos = []
    for a, b in bl:
        s = t[a:b]
        if s.startswith("#") or s[0] in NOTA:
            continue
        marcas = [notas[c] for c in dict.fromkeys(s) if c in notas]
        fija = s.startswith("|") or bool(marcas)
        ult = trozos[-1] if trozos else None
        if (ult and not fija and not ult["fija"]
                and seccion(doc, a) == seccion(doc, ult["tramos"][0][0])
                and b - ult["tramos"][0][0] <= n):
            ult["tramos"][0] = (ult["tramos"][0][0], b)
            continue
        trozos.append(dict(doc=doc, tramos=[(a, b)] + marcas,
                           fija=fija))
    return trozos


tabla = next(c for c in estructura("R01")
             if c["tramos"][0][0] == P.loc["R01-p013"].inicio)
print(len(estructura("R01")), tabla["tramos"])

29 [(1577, 1914), (1916, 2008), (2010, 2119)]

The key-figures table chunk carries three spans: the table and its two notes. The next figure draws, with the real offsets of that area of the 2023 Busmar Hogar report, the canonical passages and where each strategy cuts.

The last piece is the indexed text. A chunk can be preceded by a context header with the document title and the section (“Busmar Hogar, S.A. — Informe anual 2023 > 3. Magnitudes principales”, that is, annual report 2023 > 3. Key figures). The header can be added to any strategy: for fixed and recursive chunks the last heading before their start is used. That separates the effect of the cuts from the effect of the header.

def con_texto(trozos, cabecera):
    """Adds to each chunk the text that will be indexed."""
    for c in trozos:
        t = TEXTO[c["doc"]]
        cuerpo = "\n\n".join(t[i:f] for i, f in c["tramos"])
        sec = seccion(c["doc"], c["tramos"][0][0])
        cab = f"{TITULO[c['doc']]} > {sec}\n\n" if cabecera else ""
        c["texto"] = cab + cuerpo
    return trozos


ESTRATEGIAS = {"fixed": fijo, "recursive": recursivo,
               "structure": estructura}
TROZOS = {}
for nombre, f in ESTRATEGIAS.items():
    for cab in (False, True):
        trozos = [c for d in TEXTO for c in f(d)]
        TROZOS[(nombre, cab)] = con_texto(trozos, cab)
for (nombre, cab), tr in TROZOS.items():
    largo = np.mean([len(c["texto"]) for c in tr])
    print(f"{nombre:10s} header={cab!s:5s} {len(tr):4d} chunks,"
          f" {largo:5.0f} characters on average")
print(TROZOS[("structure", True)][12]["texto"][:200])

fixed      header=False  270 chunks,   487 characters on average
fixed      header=True   270 chunks,   561 characters on average
recursive  header=False  293 chunks,   404 characters on average
recursive  header=True   293 chunks,   480 characters on average
structure  header=False  376 chunks,   293 characters on average
structure  header=True   376 chunks,   372 characters on average
Busmar Hogar, S.A. — Informe anual 2023 > Estados financieros resumidos

| Balance (millones de USD) | 31-12-2023 |
|---|---|
| Activo no corriente | 19.543,5 |
| Activo corriente (sin caja) | 3.417,2

From a chunk to the passages: offsets and recall@k

To know whether a chunk “contains” a gold passage there is no need to compare texts: intersecting intervals is enough. The cubierta (covered) function computes what fraction of a passage's characters falls inside a set of chunks (the union of their spans, so the overlap of the fixed windows is not counted twice). A passage counts as retrieved if the top $k$ chunks cover at least half of its characters, and the recall@k of a question is the fraction of its gold passages that were retrieved:

with $O$ the gold passages and $c_1, \dots, c_k$ the spans of the $k$ highest-scoring chunks. We also measure whether a question gets all of its passages (full coverage), which is what the table-and-note questions require.

def cubierta(trozos, pid):
    """Fraction of the characters of passage pid inside the union
    of the spans of `trozos`."""
    r = P.loc[pid]
    iv = sorted((max(i, r.inicio), min(f, r.fin)) for c in trozos
                if c["doc"] == r.id_doc for i, f in c["tramos"])
    total, hasta = 0, r.inicio
    for a, b in iv:
        a = max(a, hasta)
        if b > a:
            total, hasta = total + b - a, b
    return total / (r.fin - r.inicio)


for nombre in ESTRATEGIAS:
    tr = [c for c in TROZOS[(nombre, True)] if c["doc"] == "R01"]
    cob = [float(round(cubierta([c], "R01-p013"), 2)) for c in tr]
    print(f"{nombre:10s} chunks touching the table:",
          [x for x in cob if x > 0],
          "| note ²:", max(cubierta([c], "R01-p015") for c in tr))

fixed      chunks touching the table: [0.81, 0.34] | note ²: 1.0
recursive  chunks touching the table: [1.0] | note ²: 1.0
structure  chunks touching the table: [1.0] | note ²: 1.0

With fixed-size chunking the key-figures table is split: one window covers 81% of its characters and the next one 34% (the two overlap by 50 characters). In the other two, a single chunk covers it entirely, and note ² is complete in some chunk under all three strategies, although with recursive chunking not in the table's chunk (previous figure). With or without the header, the offsets are the same: the header is added to the indexed text, not to the document.

The retriever is still missing. We use BM25 (lesson 6.3.1), which scores by word overlap weighted by rarity and normalized by chunk length. It is deterministic, fast and, as will be seen, very competitive on these texts full of proper names and figures. Normalization strips accents and a few stop words, the same for chunks and questions. The 17 questions the documents do not answer have no gold passages and are left out of the recall.

from rank_bm25 import BM25Okapi

VACIAS = set("""de la el en y a los las del que se por con un una
para es al lo su sus o como fue cual cuanto cuantos cuales
segun""".split())


def palabras(s):
    s = unicodedata.normalize("NFKD", s.lower())
    s = "".join(ch for ch in s if not unicodedata.combining(ch))
    return [w for w in re.findall(r"\w+", s) if w not in VACIAS]


resp = preg[preg.respondible].reset_index(drop=True)


def ranking(trozos, kmax=10):
    """The kmax highest-scoring chunks for each question."""
    bm = BM25Okapi([palabras(c["texto"]) for c in trozos])
    return [np.argsort(-bm.get_scores(palabras(q)),
                       kind="stable")[:kmax] for q in resp.pregunta]


def medir(trozos, rk, k):
    filas = []
    for q, orden in zip(resp.itertuples(), rk):
        top = [trozos[j] for j in orden[:k]]
        ok = [cubierta(top, p) >= 0.5
              for p in q.pasajes_oro.split(";")]
        car = sum(len(c["texto"]) for c in top)
        filas.append((q.tipo_pregunta, np.mean(ok), all(ok), car))
    return pd.DataFrame(filas,
                        columns=["tipo", "recall", "completa", "car"])


RK = {clave: ranking(tr) for clave, tr in TROZOS.items()}
filas = []
for (nombre, cab), tr in TROZOS.items():
    for k in (3, 5):
        m = medir(tr, RK[(nombre, cab)], k)
        filas.append((nombre, cab, k, m.recall.mean(),
                      m.completa.mean(), m.car.mean()))
tabla = pd.DataFrame(filas, columns=[
    "chunking", "header", "k", "recall", "complete", "characters"])
print(len(resp), "answerable questions")
tabla.pivot_table(index=["chunking", "header"], columns="k",
                  values=["recall", "characters"]).round(3)

246 answerable questions
                 characters           recall
k                         3         5      3      5
chunking  header
fixed     False    1495.577  2495.577  0.228  0.408
          True     1699.585  2815.459  0.798  0.862
recursive False    1172.317  1975.874  0.217  0.391
          True     1452.394  2407.220  0.829  0.871
structure False     846.776  1561.736  0.254  0.476
          True     1189.667  1959.715  0.873  0.892

Two readings. The first, and the biggest: without a context header no strategy works (between 21.7% and 25.4% recall with k = 3), and with it all three exceed 79%. It is not a property of BM25: the eight annual reports share a template and the questions always say which company and which year they are about. A chunk with “Ventas | 13.668,9” (sales) and no header is as good a match for Busmar Hogar 2023 as for Celra 2024. In practice, the alternative or the complement is filtering by metadata (issuer, fiscal year, document type) before searching; exercise 2 measures it.

The second: with a header, structure-aware chunking retrieves more with less text (87.3% with k = 3 and about 1,190 characters, versus 79.8% for fixed-size chunking with 1,700). But comparing at the same $k$ is not fair to anyone: each strategy delivers chunks of a different size.

At the same context, and question by question

An LLM's context costs: tokens that are paid for, seconds of computation (with the local model of 6.3.3, several per hundred tokens) and noise that distracts the model. The honest comparison puts the characters delivered on the horizontal axis and recall on the vertical axis, varying $k$ from 1 to 10. That is the lesson's header figure.

curvas = []
for nombre in ESTRATEGIAS:
    tr, rk = TROZOS[(nombre, True)], RK[(nombre, True)]
    for k in range(1, 11):
        m = medir(tr, rk, k)
        curvas.append((nombre, k, round(m.car.mean()),
                       round(m.recall.mean(), 3)))
curvas = pd.DataFrame(curvas,
                      columns=["chunking", "k", "chars", "recall"])
curvas.pivot(index="k", columns="chunking", values=["chars", "recall"])

           chars                     recall
chunking   fixed recursive structure  fixed recursive structure
k
1          569.0     490.0     408.0  0.569     0.668     0.688
2         1135.0     975.0     791.0  0.687     0.778     0.787
3         1700.0    1452.0    1190.0  0.798     0.829     0.873
4         2261.0    1928.0    1558.0  0.841     0.837     0.890
5         2815.0    2407.0    1960.0  0.862     0.871     0.892
6         3372.0    2881.0    2360.0  0.885     0.875     0.893
7         3927.0    3359.0    2755.0  0.909     0.878     0.907
8         4488.0    3820.0    3133.0  0.937     0.883     0.909
9         5045.0    4294.0    3492.0  0.952     0.904     0.917
10        5604.0    4772.0    3851.0  0.957     0.915     0.919

Up to about 2,000 characters, structure-aware chunking is on top: with k = 3 (1,190 characters) it retrieves 87.3%, more than fixed-size chunking with k = 4 (2,261 characters, 84.1%). With a lot of context the advantage disappears and fixed-size chunking ends up ahead (95.7% with k = 10), because its large, overlapping windows end up covering almost everything around the answer. Structure-aware chunking plateaus around 92%: what it does not retrieve is not a matter of how many chunks, but of which questions, as the breakdown by type shows.

por_tipo = pd.DataFrame({
    nombre: medir(TROZOS[(nombre, True)], RK[(nombre, True)], 5)
    .groupby("tipo").recall.mean()
    for nombre in ESTRATEGIAS}).round(3)
por_tipo.assign(questions=resp.tipo_pregunta.value_counts())

              fixed  recursive  structure  questions
tipo
calculo       0.500      0.000      0.000          8
comparacion   0.571      0.814      0.200          7
nota_pie      1.000      1.000      1.000         16
tabla         0.812      0.875      0.875         64
tabla_y_nota  1.000      0.885      1.000         13
texto         0.918      0.926      0.992        122
texto_y_nota  0.688      0.750      0.750         16

• Footnotes and table with note. Structure-aware chunking gets 100%: the note travels with the table. Fixed-size chunking does too, by spillover: its 500-character windows usually take in the short table and the note below it. Recursive chunking loses the tables with notes that its cut separates (88.5%).
• Text and note. None goes beyond 75%: the figure is in a paragraph (a note to the financial statements) and the qualification in the note of a table in another section. Joining the note to the table is not enough when the question comes in through the paragraph; the retriever would have to bring both, or the note would have to be indexed on its own as well.
• Calculation. Growth questions (“how much did sales grow in 2023?”) have the key-figures table as gold, but the chairman's letter literally says “un 2,2 % más que en 2022” (2.2% more than in 2022). The retriever finds the answer in another passage and the passage metric counts it as a miss (fixed-size chunking gets half right because some of its windows mix the letter with the start of the table). A reminder that the gold is also a convention.
• Comparison. “Which ETF has the lowest ongoing charges?” needs five prospectuses at once, and the result jumps around with no logic from one strategy to another (from 20% to 81.4%): with k = 5 and five documents, it is enough for two chunks of the same fund to take the place of another. There are only 7 questions and nothing should be read into that ranking. It is a question for a structured table (the five TERs) or for splitting it into five searches, one per fund.

Tables: Markdown, linearized and embeddings

A common recommendation is to linearize tables before indexing them: turn each row into a sentence that repeats the column header (“Ventas: 2023 = 13.668,9; 2022 = 13.378,6.”). The idea is that an embedding model understands a sentence better than a grid with bars. It can be measured: same strategy, same questions, tables in Markdown or linearized, with BM25 and with the small embedding model of the environment (all-MiniLM-L6-v2, 384 dimensions, trained mostly on English text; lesson 6.3.1).

import copy

from sentence_transformers import SentenceTransformer


def linealiza(tabla):
    """One sentence per row, with the header of each column."""
    filas = [[x.strip() for x in linea.strip("|").split("|")]
             for linea in tabla.strip().split("\n")]
    cab = filas[0]
    frases = [f"{f[0]}: " + "; ".join(f"{h} = {v}"
                                      for h, v in zip(cab[1:], f[1:]))
              + "." for f in filas[2:]]
    return f"{cab[0]}. " + " ".join(frases)


md = TROZOS[("structure", True)]
lin = copy.deepcopy(md)
for c in lin:
    t = TEXTO[c["doc"]]
    (i, f), *notas = c["tramos"]
    if t[i] == "|":
        cuerpo = [linealiza(t[i:f])] + [t[a:b] for a, b in notas]
        cab = f"{TITULO[c['doc']]} > {seccion(c['doc'], i)}\n\n"
        c["texto"] = cab + "\n\n".join(cuerpo)
cambia = [j for j, (a, b) in enumerate(zip(md, lin))
          if a["texto"] != b["texto"]]
print(len(cambia), "chunks with a table:")
print(lin[cambia[0]]["texto"].split("\n\n")[1][:150])

st = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2",
                         device="cpu", local_files_only=True)
cod = dict(normalize_embeddings=True, show_progress_bar=False,
           batch_size=64)
Q = st.encode(resp.pregunta.tolist(), **cod)
X_md = st.encode([c["texto"] for c in md], **cod)
X_lin = X_md.copy()
X_lin[cambia] = st.encode([lin[j]["texto"] for j in cambia], **cod)

filas = []
for nombre, tr, X in [("markdown", md, X_md),
                      ("linearized", lin, X_lin)]:
    densa = [np.argsort(-(X @ q), kind="stable")[:10] for q in Q]
    for buscador, rk in [("BM25", ranking(tr)), ("MiniLM", densa)]:
        m = medir(tr, rk, 5)
        filas.append((buscador, nombre, m.recall.mean(),
                      m[m.tipo == "tabla"].recall.mean()))
pd.DataFrame(filas, columns=["retriever", "tables", "recall@5",
                             "recall@5 table"]).round(3)

105 chunks with a table:
(millones de USD). Ventas: 2023 = 13.668,9; 2022 = 13.378,6. EBITDA ajustado¹: 2023 = 3.769,6; 2022 = 3.222,8. Resultado neto: 2023 = 2.442,8; 2022 =
  retriever      tables  recall@5  recall@5 table
0      BM25    markdown     0.892           0.875
1    MiniLM    markdown     0.442           0.406
2      BM25  linearized     0.890           0.844
3    MiniLM  linearized     0.422           0.344

In these documents linearizing does not help: with BM25 the recall of the table questions drops from 87.5% to 84.4%, and with MiniLM from 40.6% to 34.4%. Repeating “2023 =” and “2022 =” in every row dilutes the words that distinguish one row from another. And the small dense model falls far behind BM25 across the board (44% versus 89%): it is an English model, and here the questions and texts are in Spanish and are told apart by proper names, years and figures, exactly what lexical search matches best. It is not a general conclusion about embeddings (a larger multilingual model, or hybrid search, changes the result; lesson 6.3.1), but it is a warning: a chunking recommendation is measured on your own documents and questions.

SEC HTML and XBRL: when the data is already structured

The SEC-style documents in the level 6 data are in HTML, like the real ones: <h2> headings per Item, <table> tables and <sup> notes. With BeautifulSoup structure-aware chunking is straightforward: one chunk per <h2>, with the tables linearized (there is no Markdown to preserve here) and the context header with the issuer, the form and the period.

from bs4 import BeautifulSoup

fil = pd.read_parquet("filings6_documentos.parquet")
d = fil.set_index("id_doc").loc["D000162"]


def tabla_html(tb):
    filas = [[x.get_text(" ", strip=True)
              for x in tr.find_all(["th", "td"])]
             for tr in tb.find_all("tr")]
    cab = filas[0]
    return " ".join(f"{f[0]}: " + "; ".join(
        f"{h} = {v or '-'}" for h, v in zip(cab[1:], f[1:])) + "."
        for f in filas[1:])


sopa = BeautifulSoup(d.texto, "html.parser")
for h2 in sopa.find_all("h2"):
    partes = []
    for el in h2.find_next_siblings():
        if el.name == "h2":
            break
        partes.append(tabla_html(el) if el.name == "table"
                      else el.get_text(" ", strip=True))
    print(f"{d.emisor} > {d.tipo} {d.periodo_fin:%Y-%m-%d} >",
          h2.get_text(strip=True))
    print("   ", " ".join(partes)[:110])

Rapsa Insurance Inc. > 10-Q 2019-03-31 > Part I, Item 1. Financial Statements
    Revenue: Three months = 8,983.5; 3 months year to date = 8,983.5. Net income (loss) (1): Three months = 21.9;
Rapsa Insurance Inc. > 10-Q 2019-03-31 > Part I, Item 2. Management's Discussion and Analysis
    Revenue for the three months ended March 31, 2019 was $8,983.5 million. Assets under management at quarter end
Rapsa Insurance Inc. > 10-Q 2019-03-31 > Part II, Item 1A. Risk Factors
    There have been no material changes to the risk factors disclosed in our most recent Annual Report on Form 10-

But for a question like “what was Rapsa's revenue in the fourth quarter of 2019?” the text is the wrong path. US companies file their financial statements with XBRL tags: each figure is a fact with its concept (us-gaap:Revenues), its unit, its period and, in the course data, the date and time of the filing. The SEC serves them through an API (companyconcept, companyfacts and frames)11Note 1 · SEC, “EDGAR Application Programming Interfaces”, accessed on October 23, 2026. According to that page, frames aggregates one fact per entity using the most recent filing that best fits the requested period: it is convenient for a cross-section, but it is not point-in-time.. With facts, a numerical question is a filter, not a search.

Two details the text does not resolve and the facts do:

• The fourth quarter is not filed. The 10-K gives the year and the September 10-Q the nine-month cumulative figure: the quarter is the difference.
• Amendments rewrite figures. A 10-Q/A changes the earnings of a quarter that was already published. A backtest has to use the value known on each date (point-in-time), filtering by filing date.

xb = pd.read_parquet("filings6_xbrl.parquet").merge(
    fil[["id_doc", "ticker"]], on="id_doc")


def hecho(ticker, concepto, inicio, fin, a_fecha):
    """Latest value of a fact filed up to a_fecha (UTC)."""
    m = ((xb.ticker == ticker) & (xb.concepto == concepto)
         & (xb.periodo_inicio == inicio) & (xb.periodo_fin == fin)
         & (xb.ts_presentacion <= pd.Timestamp(a_fecha, tz="UTC")))
    s = xb[m].sort_values("ts_presentacion")
    return None if s.empty else s.valor.iloc[-1] / 1e6


ing = "us-gaap:Revenues"
anual = hecho("RAPA", ing, "2019-01-01", "2019-12-31", "2020-06-30")
nueve = hecho("RAPA", ing, "2019-01-01", "2019-09-30", "2020-06-30")
ventas = pd.read_csv("acciones6_ventas.csv")
oficial = ventas.query("ticker == 'RAPA' and "
                       "fin_trimestre == '2019-12-31'").ventas.item()
print(f"RAPA Q4 2019: {anual:.1f} - {nueve:.1f} = {anual - nueve:.1f}"
      f" (quarterly sales: {oficial})")

bn = "us-gaap:NetIncomeLoss"
for fecha in ["2020-09-30", "2020-12-31"]:
    print("BEBA, Q2 2020 net income known on", fecha, ":",
          hecho("BEBA", bn, "2020-04-01", "2020-06-30", fecha))

RAPA Q4 2019: 36104.6 - 26958.9 = 9145.7 (quarterly sales: 9145.7)
BEBA, Q2 2020 net income known on 2020-09-30 : 224.5
BEBA, Q2 2020 net income known on 2020-12-31 : 187.1

The subtraction gives exactly the quarter's sales. And BEBA's second-quarter 2020 net income was 224.5 million until a November amendment set it at 187.1. A RAG over the text of today's documents would answer 187.1 to a question asked as of September 2020: look-ahead bias (lesson 6.2.5) through the index. The practical rule is to split the work:

Common mistakes

• Measuring by eye. Trying five questions and picking the strategy “that looks best”. Without reference passages with offsets there is no way to know whether a change helps or hurts; with them, every change is measured in seconds.
• Comparing at the same k. Larger chunks gain recall through size. The comparison is at the same context budget.
• Chunking without a header. In template documents (reports from several years, prospectuses from the same manager) the chunk must say which document and section it belongs to, or be filtered by metadata.
• Splitting tables and separating notes. The row without the column header and the figure without its note are the two most common forms of “correct” answers that are not.
• Losing the offsets. If the index only stores the chunk text, you can neither cite the original document nor map to passages; with the offsets, every answer points to a verifiable span (lesson 6.3.3).
• Asking the text for what is in XBRL. And, conversely, forgetting the filing date: an index built today with amended documents mixes the future into the past.
• Generalizing a recipe. “Linearize the tables”, “use 512-token chunks”, “use dense embeddings”: in these documents, all three make things worse or do not help. Measure with your own corpus and questions.

Exercises

1. Change the size of fixed-size chunking to 300 and to 1,000 characters (10% overlap) and compare its recall with that of structure-aware chunking at about 1,200 characters delivered. Does any fixed size catch up with the structure-aware one?

for n in (300, 500, 1000):
    tr = con_texto([c for d in TEXTO for c in fijo(d, n, n // 10)],
                   True)
    rk = ranking(tr)
    mejor = max(((k, medir(tr, rk, k)) for k in range(1, 11)),
                key=lambda km: (km[1].car.mean() <= 1300,
                                km[1].car.mean()))
    k, m = mejor
    print(f"fixed {n:4d}: k={k:2d} {m.car.mean():6.0f} characters,"
          f" recall {m.recall.mean():.3f}")

fixed  300: k= 3   1114 characters, recall 0.772
fixed  500: k= 2   1135 characters, recall 0.687
fixed 1000: k= 1   1019 characters, recall 0.559

2. Filter by metadata before searching: for each question, keep only the chunks of its document (the question's id_doc column; with several, all of them) and measure the recall@3 of the three strategies without a header. How much of the header's effect does the filter recover?

def ranking_filtrado(trozos, k=3):
    rk = []
    for q in resp.itertuples():
        ids = set(q.id_doc.split(";"))
        sub = [j for j, c in enumerate(trozos) if c["doc"] in ids]
        bm = BM25Okapi([palabras(trozos[j]["texto"]) for j in sub])
        orden = np.argsort(-bm.get_scores(palabras(q.pregunta)),
                           kind="stable")[:k]
        rk.append([sub[j] for j in orden])
    return rk


for nombre in ESTRATEGIAS:
    tr = TROZOS[(nombre, False)]
    m = medir(tr, ranking_filtrado(tr), 3)
    print(f"{nombre:10s} no header, filtered: recall@3"
          f" {m.recall.mean():.3f}")

fixed      no header, filtered: recall@3 0.763
recursive  no header, filtered: recall@3 0.822
structure  no header, filtered: recall@3 0.837

3. Using the XBRL facts, compute the fourth-quarter 2022 revenue of all companies with a 10-K for that year, as known on 2023-06-30, and compare it with acciones6_ventas.csv. How many match to a tenth of a million?

tick = sorted(xb[(xb.forma == "10-K") & (xb.concepto == ing)
                 & (xb.periodo_fin == "2022-12-31")].ticker.unique())
dif = []
for tk in tick:
    a = hecho(tk, ing, "2022-01-01", "2022-12-31", "2023-06-30")
    n9 = hecho(tk, ing, "2022-01-01", "2022-09-30", "2023-06-30")
    v = ventas.query("ticker == @tk and fin_trimestre == '2022-12-31'")
    dif.append(round(a - n9, 1) - v.ventas.item())
dif = np.round(dif, 1)
print(len(tick), "companies;", int((dif == 0).sum()), "match;",
      "max difference", np.abs(dif).max())

60 companies; 60 match; max difference 0.0