Embeddings and vector databases: FAISS, Chroma, pgvector and Qdrant.

Searching by meaning

A question-answering system over documents (RAG, lesson 6.3.3) has two parts: retrieving the passages that contain the answer and writing it with an LLM from them. If retrieval fails, the LLM answers without the information or makes it up. This lesson covers the first part: how to turn text into vectors, how to search them fast and, above all, how to measure whether what has to be found is found.

flowchart TB
  A[Passages] --> B[Embeddings]
  B --> C[Vector index]
  D[Question] --> E[Same model]
  E --> F[Nearest neighbors]
  C --> F
  F --> G[Metadata filter]
  G --> H[Top-k to the LLM]

An embedding is a fixed-dimension vector (384 in all-MiniLM-L6-v2) that the model assigns to a text, trained so that similar texts end up close together. With unit-norm vectors, cosine similarity is the inner product: $\cos(u, v) = u \cdot v$ if $\lVert u \rVert = \lVert v \rVert = 1$. Lesson 6.1.1 used them as features for a classifier; here they are used to search. Before searching, it is worth seeing what they measure. The example corpus of this lesson (sentences, reports, prospectuses and questions) is in Spanish and stays as is, because the lesson measures how a model trained on English handles it; the first cell compares four Spanish sentences and one English one:

import numpy as np
import pandas as pd
from sentence_transformers import SentenceTransformer

st = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2",
                         device="cpu", local_files_only=True)
frases = [
    "La empresa eleva su previsión de ventas para el año",
    "La compañía mejora sus perspectivas de ingresos anuales",
    "La empresa recorta su previsión de ventas para el año",
    "The company raises its full-year sales guidance",
    "El consejo aprueba el pago de un dividendo",
]
V = st.encode(frases, normalize_embeddings=True,
              show_progress_bar=False)
print(V.shape, V.dtype, "· norm:", np.linalg.norm(V, axis=1).round(3))
sim = pd.DataFrame(V @ V.T, index=["eleva", "mejora", "recorta",
                                   "raises", "dividendo"],
                   columns=["eleva", "mejora", "recorta", "raises",
                            "dividendo"])
print(sim.round(2).to_string())

(5, 384) float32 · norm: [1. 1. 1. 1. 1.]
           eleva  mejora  recorta  raises  dividendo
eleva       1.00    0.53     0.95    0.17       0.40
mejora      0.53    1.00     0.55    0.18       0.38
recorta     0.95    0.55     1.00    0.18       0.40
raises      0.17    0.18     0.18    1.00       0.04
dividendo   0.40    0.38     0.40    0.04       1.00

The labels are the key word of each sentence: “eleva” (the company raises its sales forecast for the year), “mejora” (the company improves its annual revenue outlook), “recorta” (the company cuts its sales forecast), “raises” (the English translation of the first) and “dividendo” (the board approves a dividend payment). The result is instructive. “Eleva” and “recorta” the sales forecast, sentences of opposite meaning with almost the same words, have a similarity of 0.95; “eleva” and its paraphrase “la compañía mejora sus perspectivas de ingresos anuales”, with different words, only 0.53. The English translation sits at 0.17: the model is not multilingual. all-MiniLM-L6-v2 was trained on English text, and with Spanish text it behaves almost like a word-fragment comparator. It is the course's model because of its size (22 million parameters, 91 MB) and license (Apache-2.0); for Spanish there are multilingual models of the same family that are not installed in the environment. The lesson measures what this one delivers, which is the usual situation: a generic model on text that is not its own.

The corpus and its evaluation set

The level 6 RAG material is 21 synthetic Markdown documents: 2023 and 2024 annual reports of four issuers, prospectuses and KIDs of five ETFs and three research notes. rag6_pasajes.parquet splits them into reference passages (lesson 6.3.2 covers how to split documents) and rag6_preguntas.csv brings 263 questions with the passages that have to be cited to answer them. Title passages (section headings, which answer nothing) and the 17 questions with no answer in the documents are discarded.

pas = pd.read_parquet("rag6_pasajes.parquet")
docs = pd.read_parquet("rag6_documentos.parquet")
preg = pd.read_csv("rag6_preguntas.csv")
docs["corto"] = (docs["entidad"].str.replace(", S.A.", "")
                 .str.replace(" UCITS ETF", ""))
docs["anio"] = docs["titulo"].str.extract(r"(20\d\d)$")[0].fillna("")
pas = pas.merge(docs[["id_doc", "entidad", "anio"]], on="id_doc")
pas = pas[pas["tipo"] != "titulo"].reset_index(drop=True)
preg = preg[preg["respondible"]].reset_index(drop=True)
oro = [set(x.split(";")) for x in preg["pasajes_oro"]]
print(len(docs), "documents ·", len(pas), "passages ·", len(preg),
      "answerable questions")
print(pas["tipo"].value_counts().to_string())

21 documents · 722 passages · 246 answerable questions
tipo
texto    431
lista    119
tabla    105
nota      67

The metric is recall@k: of each question's gold passages, the fraction that appears among the top $k$ results, averaged over the questions. It is the one that matters in RAG, because what is not retrieved does not reach the LLM. With $k = 5$ it measures what would fit in a typical prompt.

FAISS: exact search

FAISS is a library of in-memory vector indexes. IndexFlatIP stores the vectors as they are and compares the query with all of them (exact search by inner product, which with normalized vectors is the cosine). With 722 passages nothing more is needed:

import faiss

faiss.omp_set_num_threads(1)
P = st.encode(list(pas["texto"]), batch_size=32,
              normalize_embeddings=True, show_progress_bar=False)
Q = st.encode(list(preg["pregunta"]), batch_size=64,
              normalize_embeddings=True, show_progress_bar=False)
indice = faiss.IndexFlatIP(P.shape[1])   # inner product = cosine
indice.add(P)
print(indice.ntotal, "vectors ·", P.nbytes // 1024, "KB in float32")
ids = pas["id_pasaje"].to_numpy()

def recall(rankings, k):
    """Fraction of the gold passages that are among the top k."""
    return round(float(np.mean([len(set(ids[r[:k]]) & g) / len(g)
                                for r, g in zip(rankings, oro)])), 3)

_, denso = indice.search(Q, 20)
print({f"recall@{k}": recall(denso, k) for k in (1, 5, 10)})

722 vectors · 1083 KB in float32
{'recall@1': 0.022, 'recall@5': 0.114, 'recall@10': 0.265}

A recall@5 of 11.4% is very low, and the index is not to blame: it is exact. You have to look at what it returns:

i = 0
print(preg.loc[i, "pregunta"], "· gold:", preg.loc[i, "pasajes_oro"])
for j in denso[i][:4]:
    print(f"  {ids[j]}  {pas.loc[j, 'entidad'][:22]:22s} "
          f"{pas.loc[j, 'texto'][:52]!r}")

¿Cuál es el mayor mercado de Busmar Hogar por ventas en 2023 y qué peso tiene? · gold: R01-p031
  R08-p076  Zaptec Capital, S.A.   'Para 2025, el grupo prevé un crecimiento de las vent'
  R07-p076  Zaptec Capital, S.A.   'Para 2024, el grupo prevé un crecimiento de las vent'
  R03-p076  Celra Semiconductores, 'Para 2024, el grupo prevé un crecimiento de las vent'
  R02-p076  Busmar Hogar, S.A.     'Para 2025, el grupo prevé un crecimiento de las vent'

The question is about the largest market of Busmar Hogar in 2023, and the top four results are the same passage (“Para 2025, el grupo prevé un crecimiento de las ventas…”, “For 2025, the group expects sales growth…”) in reports from Zaptec Capital, Celra Semiconductores and Busmar itself for the following year. The eight annual reports share a template, like real ones from the same sector, and a loose passage does not say which company or which year it belongs to: that information is in the document's title. No embedding can tell apart passages that say the same thing with other figures.

BM25 and hybrid search

The baseline of document search is BM25, a lexical model: it scores each passage by the query words it contains, with more weight on words that are rare in the corpus and a length correction. It knows nothing about synonyms, but it does not confuse “recorta” (cuts) with “eleva” (raises). Hybrid search combines the two rankings; the simplest and most robust way is reciprocal rank fusion (RRF, Cormack, Clarke and Büttcher, 2009), which adds $1/(k_0 + p)$ for each passage in each list, where $p$ is the passage's position in that list, with $k_0 = 60$, the value they fixed in their pilot study. It does not need the scores of the two systems to be comparable, only their positions.

flowchart TB
  P[Question] --> D[Embeddings: top 20]
  P --> L[BM25: top 20]
  D --> R[RRF]
  L --> R
  R --> K[Fused top-k]

import re
import unicodedata
from nltk.corpus import stopwords
from rank_bm25 import BM25Okapi

VACIAS = set(stopwords.words("spanish"))

def tokens(t):
    t = unicodedata.normalize("NFKD", t.lower())
    t = "".join(c for c in t if not unicodedata.combining(c))
    return [w for w in re.findall(r"\w+", t) if w not in VACIAS]

bm25 = BM25Okapi([tokens(t) for t in pas["texto"]])

def rank_bm25(consultas, mascaras=None):
    S = np.array([bm25.get_scores(tokens(c)) for c in consultas])
    if mascaras is not None:
        S[~mascaras] = -np.inf
    return np.argsort(-S, axis=1, kind="stable")[:, :20]

def rrf(*rankings, k0=60, n=20):
    """Reciprocal rank fusion: sum of 1 / (k0 + position)."""
    salida = []
    for rs in zip(*rankings):
        p = {}
        for r in rs:
            for pos, j in enumerate(r):
                p[j] = p.get(j, 0) + 1 / (k0 + pos + 1)
        salida.append(sorted(p, key=lambda j: (-p[j], j))[:n])
    return np.array(salida)

lexico = rank_bm25(preg["pregunta"])
hibrido = rrf(denso, lexico)
tabla = pd.DataFrame({m: [recall(r, k) for k in (1, 5, 10)]
                      for m, r in [("dense", denso), ("BM25", lexico),
                                   ("hybrid", hibrido)]},
                     index=["recall@1", "recall@5", "recall@10"])
print(tabla.round(3).to_string())

           dense   BM25  hybrid
recall@1   0.022  0.073   0.042
recall@5   0.114  0.423   0.301
recall@10  0.265  0.655   0.532

BM25 retrieves 42.3% in the top 5, almost four times more than the embeddings: the questions share terms with their passages (“provisiones para litigios”, litigation provisions; “pasivos por arrendamiento”, lease liabilities; “unidad de creación”, creation unit) and the lexical model exploits them. The fusion ends up between the two (30.1%). But neither solves the underlying problem: knowing which document each passage comes from.

Chroma: metadata and filters

A vector database is more than an index: it stores the vectors together with the text and its metadata (document, issuer, date, type) and lets you filter by them in the query. Chroma is an embedded database, inside the process itself, with where-style filters. Following the README of the ia environment, the collection is created with embedding_function=None and given the embeddings already computed (its default function would download a model, and the course works without a network), with the cosine space in the configuration (by default it uses L2, according to its documentation):

import chromadb

cliente = chromadb.EphemeralClient()
col = cliente.create_collection(
    "pasajes", embedding_function=None,
    configuration={"hnsw": {"space": "cosine"}})
col.add(ids=list(ids), embeddings=P, documents=list(pas["texto"]),
        metadatas=[{"id_doc": a, "entidad": b, "anio": c}
                   for a, b, c in pas[["id_doc", "entidad", "anio"]]
                   .itertuples(index=False)])

def filtro(pregunta):
    """Entity named in the question and, if any, the most recent year."""
    m = docs[[c in pregunta for c in docs["corto"]]]
    anios = re.findall(r"20\d\d", pregunta)
    if anios and (m["anio"] == max(anios)).any():
        m = m[m["anio"] == max(anios)]
    return m

def where(m):
    if m.empty:
        return None
    return {"id_doc": {"$in": list(m["id_doc"])}}

r = col.query(query_embeddings=Q[:1], n_results=3,
              where=where(filtro(preg.loc[0, "pregunta"])),
              include=["distances"])
print(r["ids"][0], np.round(r["distances"][0], 3))

['R01-p076', 'R01-p068', 'R01-p004'] [0.374 0.383 0.397]

The filter comes from the question itself: the entity it names and, if it cites a year, the report for that year. With the filter, the top three results for the Busmar question are already from its 2023 report (R01), although still not the gold passage. Chroma returns cosine distances ($1 - \cos$); they are the same FAISS would give on that subset. The same filter, as a mask, is applied to BM25:

pos = {p: i for i, p in enumerate(ids)}

def rank_chroma(E, consultas, n=20):
    salida = []
    for e, c in zip(E, consultas):
        w = where(filtro(c))
        k = n if w is None else min(n, int(pas["id_doc"].isin(
            w["id_doc"]["$in"]).sum()))
        r = col.query(query_embeddings=e[None], n_results=k, where=w,
                      include=[])
        fila = [pos[x] for x in r["ids"][0]]
        salida.append(fila + [fila[-1]] * (n - len(fila)))
    return np.array(salida)

mascaras = np.array([
    pas["id_doc"].isin(filtro(c)["id_doc"]).to_numpy()
    if not filtro(c).empty else np.ones(len(pas), bool)
    for c in preg["pregunta"]])
denso_f = rank_chroma(Q, preg["pregunta"])
lexico_f = rank_bm25(preg["pregunta"], mascaras)
hibrido_f = rrf(denso_f, lexico_f)
for m, r in [("dense", denso_f), ("BM25", lexico_f),
             ("hybrid", hibrido_f)]:
    print(f"{m:8s}", {f"@{k}": recall(r, k) for k in (1, 5, 10)})

dense    {'@1': 0.091, '@5': 0.464, '@10': 0.633}
BM25     {'@1': 0.302, '@5': 0.752, '@10': 0.887}
hybrid   {'@1': 0.151, '@5': 0.656, '@10': 0.804}

The filter multiplies recall: with $k = 5$, from 11.4% to 46.4% with embeddings and from 42.3% to 75.2% with BM25. One detail remains: once the document is filtered, the company's name in the query no longer tells anything and pulls toward the passages that repeat it (“Busmar Hogar opera en el sector…”, “Busmar Hogar operates in the … sector”). It is removed:

def limpia(p):
    for c in docs["corto"].unique():
        p = p.replace(c, "")
    return p

limpias = [limpia(p) for p in preg["pregunta"]]
Q2 = st.encode(limpias, batch_size=64, normalize_embeddings=True,
               show_progress_bar=False)
denso_l = rank_chroma(Q2, preg["pregunta"])
lexico_l = rank_bm25(limpias, mascaras)
hibrido_l = rrf(denso_l, lexico_l)
for m, r in [("dense", denso_l), ("BM25", lexico_l),
             ("hybrid", hibrido_l)]:
    print(f"{m:8s}", {f"@{k}": recall(r, k) for k in (1, 5, 10)})
porpas = lambda r: [len(set(ids[x[:5]]) & g) / len(g)
                    for x, g in zip(r, oro)]
tipo = (pd.DataFrame({"tipo": preg["tipo_pregunta"],
                      "dense": porpas(denso_l),
                      "BM25": porpas(lexico_l),
                      "hybrid": porpas(hibrido_l)})
        .groupby("tipo")
        .agg(n=("dense", "size"), dense=("dense", "mean"),
             BM25=("BM25", "mean"), hybrid=("hybrid", "mean")))
print(tipo.round(2).sort_values("n", ascending=False).to_string())

dense    {'@1': 0.307, '@5': 0.545, '@10': 0.698}
BM25     {'@1': 0.546, '@5': 0.866, '@10': 0.899}
hybrid   {'@1': 0.468, '@5': 0.731, '@10': 0.82}
                n  dense  BM25  hybrid
tipo
texto         122   0.71  0.90    0.72
tabla          64   0.42  0.88    0.75
nota_pie       16   0.50  1.00    0.88
texto_y_nota   16   0.50  1.00    0.84
tabla_y_nota   13   0.19  1.00    1.00
calculo         8   0.00  0.00    0.00
comparacion     7   0.23  0.29    0.47

The figure at the top summarizes the path. With the filter and the clean query, BM25 reaches 86.6% in the top 5, the embeddings 54.5% and the fusion 73.1%. By question type, BM25 retrieves every footnote and every table with a note; the embeddings fail mostly on tables (42%), where the text is figures and vertical bars. Calculation questions (“¿cuánto crecieron las ventas?”, how much did sales grow?) are found by nobody: their gold passage is the table of key figures, which does not contain the word “crecieron” (grew). Cross-document comparison questions (7) need five passages from five documents and no method gets to half.

At scale: approximate indexes

With 722 vectors, comparing against all of them is instantaneous. With millions (all the news of a decade, every EDGAR filing chunked), exact search costs one distance per vector and per query. Approximate indexes compare against a fraction:

• IVF (inverted file): groups the vectors with k-means into nlist cells and, for each query, only scans the nprobe nearest cells.
• HNSW (Malkov and Yashunin): a layered neighbor graph; the search descends the graph from an entry point and efSearch sets how many candidates it keeps.
• PQ (product quantization): splits each vector into chunks and stores each chunk as the index of a centroid; with IVF, it saves memory as well as time.

The cell builds 50,000 synthetic 384-dimensional vectors grouped into 5,000 topics (a corpus with structure, like a real one; with real embeddings the figures change, the pattern does not) and measures, for each index and parameter, what fraction of the 10 exact neighbors it retrieves and how many distances it computes per query. The number of distances is the deterministic cost measure; time depends on the machine and its load, and behaves proportionally. FAISS is pinned to one thread so that the HNSW graph is always the same.

rng = np.random.default_rng(0)
N, dim, grupos = 50_000, 384, 5_000
centros = rng.standard_normal((grupos, dim)).astype("float32")
X = centros[rng.integers(0, grupos, N)] + rng.standard_normal(
    (N, dim)).astype("float32")
X /= np.linalg.norm(X, axis=1, keepdims=True)
Xq = centros[rng.integers(0, grupos, 500)] + rng.standard_normal(
    (500, dim)).astype("float32")
Xq /= np.linalg.norm(Xq, axis=1, keepdims=True)
exacto = faiss.IndexFlatIP(dim)
exacto.add(X)
_, verdad = exacto.search(Xq, 10)

def r10(I):
    return np.mean([len(set(a) & set(b)) / 10
                    for a, b in zip(I, verdad)])

cuantif = faiss.IndexFlatIP(dim)
ivf = faiss.IndexIVFFlat(cuantif, dim, 256, faiss.METRIC_INNER_PRODUCT)
ivf.cp.seed = 0
ivf.train(X[:20_000])
ivf.add(X)
tam = np.array([ivf.invlists.list_size(i) for i in range(256)])
filas = [("exact", "-", 1.0, N)]
for nprobe in (1, 4, 16, 64):
    ivf.nprobe = nprobe
    _, I = ivf.search(Xq, 10)
    _, listas = cuantif.search(Xq, nprobe)
    filas.append(("IVF", f"nprobe={nprobe}", r10(I),
                  256 + tam[listas].sum(1).mean()))
hnsw = faiss.IndexHNSWFlat(dim, 16, faiss.METRIC_INNER_PRODUCT)
hnsw.hnsw.efConstruction = 40
hnsw.add(X)
for ef in (16, 32, 64, 128, 256):
    hnsw.hnsw.efSearch = ef
    faiss.cvar.hnsw_stats.reset()
    _, I = hnsw.search(Xq, 10)
    filas.append(("HNSW", f"efSearch={ef}", r10(I),
                  faiss.cvar.hnsw_stats.ndis / len(Xq)))
pq = faiss.IndexIVFPQ(faiss.IndexFlatIP(dim), dim, 256, 48, 8,
                      faiss.METRIC_INNER_PRODUCT)
pq.cp.seed = 0
pq.train(X[:20_000])
pq.add(X)
tam_pq = np.array([pq.invlists.list_size(i) for i in range(256)])
for nprobe in (16, 64):
    pq.nprobe = nprobe
    _, I = pq.search(Xq, 10)
    _, listas = pq.quantizer.search(Xq, nprobe)
    filas.append(("IVF-PQ", f"nprobe={nprobe}", r10(I),
                  256 + tam_pq[listas].sum(1).mean()))
ann = pd.DataFrame(filas, columns=["index", "parameter", "recall@10",
                                   "distances per query"])
ann["distances per query"] = ann["distances per query"].round()
print(ann.round(3).astype({"distances per query": int})
      .to_string(index=False))
print("bytes per vector: exact", dim * 4, "· IVF-PQ", pq.code_size)

 index    parameter  recall@10  distances per query
 exact            -      1.000                50000
   IVF     nprobe=1      0.427                  453
   IVF     nprobe=4      0.556                 1041
   IVF    nprobe=16      0.712                 3386
   IVF    nprobe=64      0.894                12763
  HNSW  efSearch=16      0.646                  582
  HNSW  efSearch=32      0.796                  993
  HNSW  efSearch=64      0.860                 1707
  HNSW efSearch=128      0.904                 3067
  HNSW efSearch=256      0.941                 5679
IVF-PQ    nprobe=16      0.658                 3386
IVF-PQ    nprobe=64      0.792                12763
bytes per vector: exact 1536 · IVF-PQ 48

HNSW dominates IVF on this data: with about 3,000 distances per query, HNSW retrieves 90.4% of the exact top 10 and IVF 71.2%; IVF needs about 12,800 to reach 89.4%. Neither reaches 100% without approaching the exact cost. The price of HNSW lies elsewhere: building the graph is the slowest part of the cell and the graph takes memory on top of the vectors. IVF-PQ scans the same cells as IVF (the same distances, but each one is a sum of 48 values from a table instead of a 384-dimensional product) and stores 48 bytes per vector against the 1,536 of a float32 vector (32 times less), at the cost of recall: 79.2% with 64 cells, against 89.4% for uncompressed IVF. The FAISS guide to choosing an index sums up the criterion: IndexFlat is the only one that guarantees exact results and is enough with few searches; if exact results are not needed and memory is not tight, HNSW (about $d \cdot 4 + M \cdot 2 \cdot 4$ bytes per vector: 1,664 here, with $M = 16$); if it is tight, product quantization.

pgvector and Qdrant

FAISS and Chroma run inside the process. In production, vectors usually live in a service:

• pgvector adds to PostgreSQL a vector type (up to 2,000 indexable dimensions), the cosine distance <=> and HNSW indexes (by default m = 16, ef_construction = 64 and hnsw.ef_search = 40) and IVFFlat. Its advantage is that the passages, the metadata and the permissions are in the same database and are filtered with SQL. Its documentation warns that with approximate indexes the filter is applied after the index scan, so a selective filter can return fewer results than requested; since version 0.8.0 there are iterative scans (hnsw.iterative_scan) that keep searching until the list is complete.
• Qdrant is a dedicated server (REST on port 6333 and gRPC on 6334, with Docker locally) with payload filters inside the search; the Python client creates collections with their distance and queries with query_points.

Neither of the two is run in the course (they need a server). The code, according to their documentation (accessed 2026-10-23):

# pgvector: PostgreSQL with the vector extension (not run)
# SQL, once:
#   CREATE EXTENSION vector;
#   CREATE TABLE pasajes (id text PRIMARY KEY, id_doc text,
#                         texto text, embedding vector(384));
#   CREATE INDEX ON pasajes USING hnsw (embedding vector_cosine_ops);
import psycopg
from pgvector.psycopg import register_vector

with psycopg.connect("dbname=research") as conn:
    register_vector(conn)
    for (i, doc, txt), e in zip(pas[["id_pasaje", "id_doc", "texto"]]
                                .itertuples(index=False), P):
        conn.execute("INSERT INTO pasajes VALUES (%s, %s, %s, %s)",
                     (i, doc, txt, e))
    conn.execute("SET hnsw.ef_search = 100")
    conn.execute("SET hnsw.iterative_scan = relaxed_order")
    filas = conn.execute(
        "SELECT id, texto FROM pasajes WHERE id_doc = %s "
        "ORDER BY embedding <=> %s LIMIT 5", ("R01", Q[0])).fetchall()

# Qdrant: self-hosted or managed server (not run in the course)
from qdrant_client import QdrantClient
from qdrant_client.models import (Distance, FieldCondition, Filter,
                                  MatchValue, PointStruct, VectorParams)

qd = QdrantClient(url="http://localhost:6333")
qd.create_collection(
    collection_name="pasajes",
    vectors_config=VectorParams(size=384, distance=Distance.COSINE))
qd.upsert(collection_name="pasajes", wait=True, points=[
    PointStruct(id=k, vector=e.tolist(),
                payload={"id_pasaje": i, "id_doc": doc})
    for k, (i, doc, e) in enumerate(zip(ids, pas["id_doc"], P))])
res = qd.query_points(
    collection_name="pasajes", query=Q[0].tolist(), limit=5,
    query_filter=Filter(must=[FieldCondition(
        key="id_doc", match=MatchValue(value="R01"))]),
    with_payload=True).points

Exercises

1. Prepend to each passage the title of its document (for example, “Busmar Hogar, S.A. — Informe anual 2023”, annual report 2023) before computing its embedding and repeat the dense search without filters. How much does recall@5 rise?

titulos = pas["id_doc"].map(docs.set_index("id_doc")["titulo"])
P2 = st.encode(list(titulos + ". " + pas["texto"]), batch_size=32,
               normalize_embeddings=True, show_progress_bar=False)
idx2 = faiss.IndexFlatIP(P2.shape[1])
idx2.add(P2)
_, denso2 = idx2.search(Q, 20)
print({f"recall@{k}": recall(denso2, k) for k in (1, 5, 10)})

{'recall@1': 0.199, 'recall@5': 0.515, 'recall@10': 0.583}

2. Weight the fusion: give BM25 twice the weight of the embeddings in RRF (adding $2/(k_0 + p)$ for BM25) and measure recall@5 with the filter and the clean query.

def rrf_pesos(A, B, wa=1.0, wb=2.0, k0=60, n=20):
    salida = []
    for ra, rb in zip(A, B):
        p = {}
        for r, w in ((ra, wa), (rb, wb)):
            for pos_, j in enumerate(r):
                p[j] = p.get(j, 0) + w / (k0 + pos_ + 1)
        salida.append(sorted(p, key=lambda j: (-p[j], j))[:n])
    return np.array(salida)

for wb in (1, 2, 4):
    print(f"BM25 weight {wb}: recall@5 = "
          f"{recall(rrf_pesos(denso_l, lexico_l, wb=wb), 5)}")

BM25 weight 1: recall@5 = 0.731
BM25 weight 2: recall@5 = 0.81
BM25 weight 4: recall@5 = 0.813

3. An asset manager wants a search engine over 15 years of research notes (about 2 million passages), with per-team permissions and filters by date and issuer, and already stores everything in PostgreSQL. Which option would you choose and what would you measure first?