---
title: "6.2.4 Lightweight fine-tuning: LoRA and QLoRA"
description: "LoRA with peft on MiniLM to classify headlines, the parameter and memory math of LoRA and QLoRA, and a comparison with FinBERT and a logistic regression."
url: "https://pythonparatrading.com/en/python-trading-course/level-6/lightweight-fine-tuning-lora-qlora/"
lang: "en"
image: "https://pythonparatrading.com/og/en/python-trading-course/level-6/lightweight-fine-tuning-lora-qlora.png"
---

# 6.2.4 · Lightweight fine-tuning for financial tasks: LoRA and QLoRA

> Tuning every weight of a language model takes memory almost nobody has. LoRA freezes the model and trains two small matrices per layer; QLoRA also stores the base in 4 bits. This lesson does the parameter and memory math with the real Qwen2.5 configuration, trains LoRA adapters on MiniLM on a CPU to classify the sentiment of synthetic headlines, and compares them with an untrained FinBERT and a logistic regression on embeddings, with a time-based split and a placebo.

- Course: [Level 6: Generative AI, LLMs, agents, big data and alternative data](https://pythonparatrading.com/en/python-trading-course/level-6/)
- Module: 6.2 · LLMs for research and trading
- Duration: 70 min
- Prerequisites: [6.1.1 · Tokenization, embeddings, financial dictionaries and FinBERT](https://pythonparatrading.com/en/python-trading-course/level-6/tokenization-embeddings-loughran-mcdonald-finbert/), [5.7.1 · PyTorch: tensors, autograd and the training loop](https://pythonparatrading.com/en/python-trading-course/level-5/pytorch-tensors-autograd-training-loop/)

## By the end of this lesson

- Explain the LoRA decomposition (W + (α/r)·B·A), why B starts at zero and why the adapter can be merged at no inference cost.
- Count trainable parameters and estimate the memory of full fine-tuning, LoRA and QLoRA (NF4 and double quantization) for a 0.5B and a 7B model.
- Train LoRA adapters with peft on a small encoder on a CPU, with a time-based split and validation after each epoch.
- Compare the fine-tune with two cheap baselines (an untouched pretrained model and a logistic regression on embeddings) using a paired test.
- Recognize adapter overfitting with a shuffled-label placebo and know what a good result on template data does not prove.

## Three ways to adapt a model to a task

A pretrained model knows a lot about language and little about the specific task: separating good, bad and neutral
headlines for an issuer, extracting a KPI from a 10-Q or classifying the tone of an analyst's question. There are three
ways to bring it closer, from least to most expensive:

```mermaid
flowchart TB
  A[Pretrained] --> B[Use as is]
  A --> C[Train on top]
  A --> D[Tune weights]
  D --> E[Full fine-tuning]
  D --> F[LoRA and QLoRA]
```

1. **Use it as is** (zero-shot): FinBERT classifies financial sentiment without seeing a single example of your own, or
   an LLM receives the task in the prompt (lessons 6.1.1 and 6.2.3). Zero cost, but the model brings the biases of its
   corpus: the language, the style and what was considered positive at the time.
2. **Freeze it and train on top**: the embeddings are computed once and a linear model is fitted (lesson 6.1.1). It is
   cheap and hard to overfit, but the embeddings do not move: if the encoder does not separate two classes well, the
   logistic regression cannot fix it.
3. **Tune the weights**: full fine-tuning moves every parameter and needs to store a gradient and the optimizer state
   for each one. Parameter-efficient fine-tuning (PEFT) methods move only a part. LoRA is the most widely used.

This lesson focuses on the third, with the other two as baselines, and measures it on a small task that fits on a CPU:
the sentiment of the headlines in the level 6 synthetic corpus, whose ground truth is known.

## The task: headline sentiment with 300 labels

Each original news item in `noticias6_corpus.parquet` has its `sentimiento_verdadero` (true sentiment) in
`noticias6_verdad.csv`: negative, zero for neutral events (shareholder meetings, appointments, conferences) or positive.
The class is its sign. Only the headline is used (a median of 18 tokens with the MiniLM tokenizer) so that training fits
on a CPU in seconds, with a **time-based split**: 300 headlines from 2019-2022 to train, 200 from 2023 to validate and
choose hyperparameters, and 600 from 2024-2025 as the test set, which is not looked at until the end. The sample corpus
is synthetic and mostly in Spanish, with the rest in English; it stays as is, because the lesson measures precisely how
each model copes with the language.

```python
import numpy as np
import pandas as pd
import torch

torch.manual_seed(0)
torch.use_deterministic_algorithms(True)
corpus = pd.read_parquet("noticias6_corpus.parquet")
verdad = pd.read_csv("noticias6_verdad.csv")
d = corpus.merge(verdad, on="id_noticia")
d = d[d["tipo_noticia"] == "original"].copy()
s = d["sentimiento_verdadero"].round(2)
d["y"] = np.sign(s).astype(int) + 1   # 0 neg., 1 neutral, 2 pos.
d["anio"] = d["ts"].dt.year
tr = d[d["anio"] <= 2022].sample(300, random_state=0)
va = d[d["anio"] == 2023].sample(200, random_state=0)
te = d[d["anio"] >= 2024].sample(600, random_state=0)
for nombre, x in [("train", tr), ("validation", va),
                  ("test", te)]:
    print(f"{nombre:13s} {len(x):5d} headlines · "
          f"{x['idioma'].eq('es').mean():.0%} in Spanish · "
          f"classes {np.bincount(x['y'], minlength=3)}")
print(te[["titular", "y"]].head(4).to_string(index=False))
```
```output
train           300 headlines · 77% in Spanish · classes [ 96  82 122]
validation      200 headlines · 74% in Spanish · classes [69 45 86]
test            600 headlines · 77% in Spanish · classes [207 138 255]
                                                                titular  y
                    Incendio en la planta de Funvia Petroleum en Bilbao  0
           Contrato de 1.606,9 millones de dólares para Rapsa Insurance  2
                            Centis Health lanza la plataforma Centis V6  2
Bezvia Gas llama a revisión a 3.000 unidades de la plataforma Bezvia X4  0
```

Three hundred labels is a realistic figure: it is what an analyst reviews in an afternoon. Three in four headlines are in
Spanish, the language MiniLM and FinBERT know worst, since both were trained on English text. The most frequent class in
the test set is the positive one (255 of 600): a classifier that always said “positive” would be right 42.5% of the
time.

<aside class="callout" data-kind="aviso">

The headlines come from templates by event type (“Incendio en la planta de…”, “Fire at the … plant”; “Contrato de…
millones de dólares para…”, “… million dollar contract for…”). Separating their sentiment is much easier than with real
headlines, where the same fact is told in a thousand ways and the sentiment depends on what the market expected. The
figures in this lesson compare methods with one another under the same conditions; they are not an estimate of what can
be achieved with real news.

</aside>

## LoRA: training a low-rank correction

A linear layer computes $y = W x$ with $W$ of size $d \times k$. Full fine-tuning learns an update $\Delta W$ of the same
size. LoRA (Hu et al., 2021) starts from the observation that this update has, in practice, a low effective rank, and
writes it as the product of two narrow matrices:

$$
y = W x + \frac{\alpha}{r} B A x, \qquad A \in \mathbb{R}^{r \times k},\ B \in \mathbb{R}^{d \times r},\ r \ll \min(d, k)
$$

$W$ is frozen; only $A$ and $B$ are trained. The factor $\alpha / r$ sets the scale of the correction (with
`lora_alpha=16` and `r=8`, it equals 2) so the rank can be changed without retuning the learning rate.

```mermaid
flowchart LR
  X[x] --> W[W frozen]
  X --> A[A: r x k]
  A --> B[B: d x r]
  W --> S[sum]
  B --> S
  S --> Y[y]
```

The parameter count is immediate: $d \cdot k$ for $W$ and $r (d + k)$ for the adapter. In a MiniLM attention matrix
($384 \times 384$):

```python
d_entrada = d_salida = 384
for r in (1, 8, 64):
    completa = d_entrada * d_salida
    lora = r * (d_entrada + d_salida)
    print(f"r = {r:2d}: W has {completa:,} parameters; "
          f"A and B, {lora:,} ({lora / completa:.1%})")
```
```output
r =  1: W has 147,456 parameters; A and B, 768 (0.5%)
r =  8: W has 147,456 parameters; A and B, 6,144 (4.2%)
r = 64: W has 147,456 parameters; A and B, 49,152 (33.3%)
```

With `peft` you choose which layers get adapters (`target_modules`). The original LoRA recipe puts them on the `query`
and `value` projections of the attention; the cell does so in all six MiniLM layers:

```python
import transformers
from transformers import AutoModel, AutoTokenizer
from peft import LoraConfig, get_peft_model

transformers.logging.set_verbosity_error()
NOMBRE = "sentence-transformers/all-MiniLM-L6-v2"
tok = AutoTokenizer.from_pretrained(NOMBRE)
cfg = LoraConfig(r=8, lora_alpha=16, lora_dropout=0.1,
                 target_modules=["query", "value"])
base = get_peft_model(AutoModel.from_pretrained(NOMBRE), cfg)
total = sum(p.numel() for p in base.parameters())
entren = sum(p.numel() for p in base.parameters()
             if p.requires_grad)
print(f"parameters: {total:,} · trainable: {entren:,} "
      f"({entren / total:.2%})")
capa = base.base_model.model.encoder.layer[0].attention.self.query
print(type(capa).__name__, tuple(capa.lora_A["default"].weight.shape),
      tuple(capa.lora_B["default"].weight.shape),
      "B = 0:", bool((capa.lora_B["default"].weight == 0).all()))
```
```output
parameters: 22,786,944 · trainable: 73,728 (0.32%)
Linear (8, 384) (384, 8) B = 0: True
```

73,728 of 22.8 million parameters are trained (0.32%): 6 layers × 2 matrices × 6,144. The `query` layer is now a peft
`Linear` with its `lora_A` (8×384) and its `lora_B` (384×8). **B starts at zero** and A at random: that way $BA = 0$
and, before training, the adapted model gives exactly the same outputs as the original. The fine-tune starts from the
pretrained model, not from a model randomly broken.

Three properties make LoRA practical:

- **Memory**: there are no gradients or Adam state for the frozen weights (the next section quantifies it).
- **One model, many adapters**: the same MiniLM can serve one adapter for headlines, another for analyst questions and
  another for the risk factors of a 10-K, each a few hundred KB.
- **No inference cost**: after training, $W' = W + \frac{\alpha}{r} BA$ is computed once and the model is back to its
  original matrices. The paper highlights this against other adapters, which add layers and latency.

## How much memory: from 0.5B to 7B, with and without quantization

For an LLM the math matters more. The cell builds Qwen2.5-0.5B (the configuration of the course's local model) and a
configuration with the published dimensions of Qwen2.5-7B (28 layers, a hidden size of 3,584, 18,944 in the
intermediate layer, attention with 28 query heads and 4 key-value heads) **on the `meta` device**: PyTorch creates the
structure without allocating memory for the weights, so a 7B model can be counted on a laptop. The adapters go on the
seven linear projections of each block, with rank 8.

```python
from transformers import AutoConfig, AutoModelForCausalLM
from transformers import Qwen2Config

def cuenta(config, modulos, r=8):
    """Total, linear-layer and LoRA parameters (no weights)."""
    with torch.device("meta"):
        m = AutoModelForCausalLM.from_config(config)
    total = sum(p.numel() for p in m.parameters())
    lineales = sum(mod.weight.numel()
                   for n, mod in m.model.layers.named_modules()
                   if isinstance(mod, torch.nn.Linear))
    m = get_peft_model(m, LoraConfig(r=r, lora_alpha=2 * r,
                                     target_modules=modulos))
    lora = sum(p.numel() for p in m.parameters() if p.requires_grad)
    return total, lineales, lora

TODAS = ["q_proj", "k_proj", "v_proj", "o_proj",
         "gate_proj", "up_proj", "down_proj"]
q05 = AutoConfig.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
q7 = Qwen2Config(hidden_size=3584, intermediate_size=18944,
                 num_hidden_layers=28, num_attention_heads=28,
                 num_key_value_heads=4, vocab_size=152064,
                 tie_word_embeddings=False)
filas = {}
for nombre, c in [("Qwen2.5-0.5B", q05), ("Qwen2.5-7B", q7)]:
    filas[nombre] = cuenta(c, TODAS)
print(pd.DataFrame(
    {k: {"total (M)": t / 1e6, "linear (M)": li / 1e6,
         "LoRA r=8 (M)": lo / 1e6, "LoRA (%)": 100 * lo / t}
     for k, (t, li, lo) in filas.items()}).T.round(2))
```
```output
              total (M)  linear (M)  LoRA r=8 (M)  LoRA (%)
Qwen2.5-0.5B     494.03      357.83          4.40      0.89
Qwen2.5-7B      7615.62     6525.29         20.19      0.27
```

The totals match the Qwen2.5-7B-Instruct model card (7.61B parameters, 6.53B excluding embeddings)[^1]. Rank-8 LoRA in
every linear layer trains 20.2 million: 0.27% of the 7B model and 0.89% of the 0.5B one (in the small one, the
embeddings weigh more).

Training memory has four parts: the weights, the gradients, the optimizer state and the activations. With Adam in fp32,
each trainable parameter costs 16 bytes (weight, gradient and two 4-byte moments). The cell compares three
configurations, **without counting the activations**, which depend on the batch size and the sequence length:

- **Full**: everything trainable in fp32 with Adam, 16 bytes per parameter.
- **LoRA**: the frozen base in bf16 (2 bytes per parameter) and 16 bytes per adapter parameter.
- **QLoRA**: the frozen linear layers in NF4 (4 bits) with double quantization of the constants, the rest in bf16 and
  the adapter as in LoRA.

```python
def memoria_gb(total, lineales, lora):
    """GB to train: full, LoRA (bf16 base), QLoRA (NF4)."""
    adam = 4 + 4 + 8          # fp32 weight, gradient, two moments
    completo = total * adam
    lora_bf16 = total * 2 + lora * adam
    resto = total - lineales  # embeddings, norms and output in bf16
    bits = 4 + 0.127          # NF4 + double-quantized constants
    qlora = lineales * bits / 8 + resto * 2 + lora * adam
    return [x / 1e9 for x in (completo, lora_bf16, qlora)]

mem = pd.DataFrame({k: memoria_gb(*v) for k, v in filas.items()},
                   index=["full fp32+Adam", "LoRA (bf16 base)",
                          "QLoRA (NF4 base)"]).round(1)
print(mem.to_string())
```
```output
                  Qwen2.5-0.5B  Qwen2.5-7B
full fp32+Adam             7.9       121.8
LoRA (bf16 base)           1.1        15.6
QLoRA (NF4 base)           0.5         5.9
```

*Fig. — Memory to fine-tune Qwen2.5-7B (7,616 million parameters) by method, excluding activations.* Horizontal bars: full fine-tuning in fp32 with Adam, 121.8 GB; rank-8 LoRA on all linear layers with the frozen base in bf16, 15.6 GB; QLoRA with the base in 4-bit NF4 and double quantization, highlighted, 5.9 GB.

For the 7B model, full fine-tuning needs 121.8 GB just for weights, gradients and optimizer: several data center GPUs.
LoRA brings it down to 15.6 GB, of which 15.2 are the base in bf16. QLoRA compresses precisely that base and reaches
5.9 GB. The adapter state is the same in both (20 million × 16 bytes = 0.3 GB).

## QLoRA: the base in 4 bits

QLoRA (Dettmers et al., 2023) trains LoRA adapters on a model whose base is quantized to 4 bits. The paper introduces
three pieces: the **NF4** type (4-bit NormalFloat, with its 16 levels spread according to the quantiles of a normal
distribution, because the weights of a network are distributed approximately normally), **double quantization**
(quantizing the scale constants of each block of 64 weights as well) and **paged optimizers** (which move the optimizer
state to system memory during spikes). According to the paper:

- Fine-tuning a 65B LLaMA in 16 bits needs more than 780 GB of GPU memory; QLoRA does it on a single 48 GB GPU without
  losing performance relative to 16-bit fine-tuning.
- Scale constants in fp32 cost $32/64 = 0.5$ bits per parameter; with double quantization, $8/64 + 32/(64 \cdot 256) =
  0.127$ bits, about 0.37 bits less (about 3 GB in a 65B model). That is the 4.127 in the cell.
- To match full fine-tuning, adapters have to go on **all** the linear layers, not just `query` and `value`; the rank,
  on the other hand, barely mattered in their experiments.

During computation, each block is dequantized to bf16 on the fly: QLoRA saves memory, not operations. The following code
is the one from the transformers and peft documentation (accessed 2026-10-23); it is not run in the course:
bitsandbytes is not in the `ia` environment and, with a 0.5B model that fits entirely in memory, quantization would add
nothing to measure.

```python skip
# QLoRA with bitsandbytes (not in the environment; not run)
import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import (LoraConfig, get_peft_model,
                  prepare_model_for_kbit_training)

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",          # 4-bit NormalFloat
    bnb_4bit_use_double_quant=True,     # quantizes the constants
    bnb_4bit_compute_dtype=torch.bfloat16,
)
modelo = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct", quantization_config=bnb,
    device_map="auto")
modelo = prepare_model_for_kbit_training(modelo)
cfg = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
                 target_modules="all-linear", task_type="CAUSAL_LM")
modelo = get_peft_model(modelo, cfg)
modelo.print_trainable_parameters()
```

`target_modules="all-linear"` is peft's shortcut for “all the linear layers” (except the output layer), which is what
the paper recommends. The transformers documentation lists bitsandbytes backends for NVIDIA GPUs, Intel XPU, Intel Gaudi
and CPU, and warns that with weights in 8 or 4 bits only **extra** parameters can be trained: the adapters.

## Two baselines before training anything

A fine-tune is only worth what it improves over the cheap alternative. The first baseline is FinBERT as is, without
seeing a single label of your own (lesson 6.1.1; its weights are licensed CC-BY-NC-SA-4.0). To keep the run short, it is
measured on the first 100 test headlines, a random subsample:

```python
from transformers import pipeline

finbert = pipeline("text-classification", model="ProsusAI/finbert",
                   device="cpu")
muestra = te.iloc[:100]
codigo = {"negative": 0, "neutral": 1, "positive": 2}
salida = finbert(list(muestra["titular"]), batch_size=32)
pred_fb = np.array([codigo[r["label"]] for r in salida])
acierto_fb = pred_fb == muestra["y"].to_numpy()
print(f"FinBERT: {acierto_fb.mean():.1%} on {len(muestra)} headlines")
for idioma in ("en", "es"):
    m = muestra["idioma"].to_numpy() == idioma
    print(f"  {idioma}: {acierto_fb[m].mean():.1%} "
          f"({m.sum()} headlines)")
```
```output
FinBERT: 36.0% on 100 headlines
  en: 68.2% (22 headlines)
  es: 26.9% (78 headlines)
```

FinBERT gets 36.0% right, below the 42.5% of always answering the majority class. In English it reaches 68.2% (only 22
headlines, so very noisy), and in Spanish it sinks to 26.9%: its vocabulary chops Spanish into English fragments (lesson
6.1.1). A financial-domain model is not a model of **this** domain: another language, another headline style and another
notion of neutral.

The second baseline is the one from 6.1.1: the frozen MiniLM embeddings (mean over tokens and unit norm, which is what
`sentence-transformers` does with this model) and a logistic regression. The cell reuses the model from the previous
section with the adapter disabled (`disable_adapter()`), so the embeddings are those of the original MiniLM. The
regularization is chosen on validation:

```python
from sklearn.linear_model import LogisticRegression

def lotes(textos, n=100):
    textos = list(textos)
    for k in range(0, len(textos), n):
        yield tok(textos[k:k + n], truncation=True,
                  max_length=48, padding=True, return_tensors="pt")

def embeber(modelo, x):
    """Mean over tokens (no padding) and unit norm, as in MiniLM."""
    h = modelo(input_ids=x["input_ids"],
               attention_mask=x["attention_mask"]).last_hidden_state
    m = x["attention_mask"].unsqueeze(-1).float()
    e = (h * m).sum(1) / m.sum(1)
    return torch.nn.functional.normalize(e, dim=-1)

base.eval()
with torch.inference_mode(), base.disable_adapter():
    E = {k: torch.cat([embeber(base, x) for x in lotes(df["titular"])])
         .numpy() for k, df in [("tr", tr), ("va", va), ("te", te)]}
for C in (1, 10, 100, 1000):
    lr = LogisticRegression(C=C, max_iter=3000).fit(E["tr"], tr["y"])
    print(f"C = {C:4d}: validation {lr.score(E['va'], va['y']):.1%}")
```
```output
C =    1: validation 77.0%
C =   10: validation 84.5%
C =  100: validation 86.0%
C = 1000: validation 87.5%
```

Validation accuracy rises with $C$ (less regularization) up to 87.5% with $C = 1{,}000$; with $C = 100$ it is 86.0%,
three headlines out of 200 fewer. The best on validation is chosen, $C = 1{,}000$. On the test set:

```python
lr = LogisticRegression(C=1000, max_iter=3000).fit(E["tr"], tr["y"])
pred_lr = lr.predict(E["te"])
print(f"logistic: {np.mean(pred_lr == te['y']):.1%} on test")
```
```output
logistic: 89.3% on test
```

89.3% with 300 labels is a demanding baseline. Any fine-tune has to beat it comfortably to make up for its cost and its
risk.

## Training the adapters

The classifier is MiniLM with adapters, the same mean over tokens and a linear layer from 384 to 3 classes (the head,
fully trainable: 1,155 more parameters). The embedding is multiplied by 10 before the head because, with unit norm, its
components are very small and the head would take a long time to separate the classes. It is trained with AdamW,
batches of 32, a learning rate of $2 \cdot 10^{-3}$ (high for full fine-tuning, normal for LoRA) and four epochs, and at
the end of each one the loss is measured on 100 validation headlines from 2023. Each batch is trimmed to the length of
its longest headline, with no padding columns that add nothing: with headlines of a median of 18 tokens, that halves the
computation.

```python
class Clasificador(torch.nn.Module):
    def __init__(self, base):
        super().__init__()
        self.base = base
        self.cabeza = torch.nn.Linear(384, 3)

    def forward(self, x):
        return self.cabeza(10 * embeber(self.base, x))

def recorta(x):
    """Remove the padding columns this batch does not need."""
    n = int(x["attention_mask"].sum(1).max())
    return {c: v[:, :n] for c, v in x.items()}

def entrenar(modelo, df, y, epocas, lr, lote=32, val=va.iloc[:100]):
    x_tr = next(lotes(df["titular"], len(df)))
    x_va = next(lotes(val["titular"], len(val)))
    y_tr, y_va = torch.tensor(y), torch.tensor(val["y"].to_numpy())
    opt = torch.optim.AdamW(
        [p for p in modelo.parameters() if p.requires_grad], lr=lr)
    g = torch.Generator().manual_seed(0)
    perd = torch.nn.functional.cross_entropy
    curva = []
    for ep in range(1, epocas + 1):
        modelo.train()
        orden, suma = torch.randperm(len(y_tr), generator=g), 0.0
        for k in range(0, len(orden), lote):
            b = orden[k:k + lote]
            x_b = recorta({c: v[b] for c, v in x_tr.items()})
            l = perd(modelo(x_b), y_tr[b])
            opt.zero_grad()
            l.backward()
            opt.step()
            suma += l.item() * len(b)
        modelo.eval()
        with torch.inference_mode():
            z = modelo(x_va)
        curva.append((ep, suma / len(y_tr), perd(z, y_va).item(),
                      (z.argmax(1) == y_va).float().mean().item()))
        print("epoch {} · train {:.2f} · validation {:.2f}"
              " · validation accuracy {:.1%}".format(*curva[-1]))
    return curva

torch.manual_seed(0)
clf = Clasificador(base)
curva_lora = entrenar(clf, tr, tr["y"].to_numpy(), epocas=4, lr=2e-3)
```
```output
epoch 1 · train 1.04 · validation 0.93 · validation accuracy 50.0%
epoch 2 · train 0.72 · validation 0.64 · validation accuracy 77.0%
epoch 3 · train 0.44 · validation 0.40 · validation accuracy 87.0%
epoch 4 · train 0.24 · validation 0.25 · validation accuracy 91.0%
```

The validation loss falls with the training loss over the four epochs (from 0.93 to 0.25) and validation accuracy goes
from 50.0% to 91.0%. There is no sign of overfitting yet: validation keeps improving, and with more epochs it would
probably improve somewhat more. Training stops at four so the lesson runs in a few minutes of CPU, and for the same
reason the per-epoch validation uses 100 headlines and the test 600. In a real problem, the number of epochs is chosen
on validation (early stopping) and, once chosen, the test set is looked at only once.

## Result on the 2024-2025 test set

```python
from scipy.stats import binomtest

clf.eval()
with torch.inference_mode():
    pred_lora = torch.cat([clf(x).argmax(1)
                           for x in lotes(te["titular"])]).numpy()
y_te = te["y"].to_numpy()
ok_lr, ok_lo = pred_lr == y_te, pred_lora == y_te
tabla = pd.DataFrame({
    "logistic": [ok_lr.mean()] + [ok_lr[te["idioma"] == i].mean()
                                  for i in ("en", "es")],
    "LoRA": [ok_lo.mean()] + [ok_lo[te["idioma"] == i].mean()
                              for i in ("en", "es")]},
    index=["all (600)", "English", "Spanish"])
print((100 * tabla).round(1).to_string())
print(f"FinBERT's 100: FinBERT {acierto_fb.mean():.1%} · logistic "
      f"{ok_lr[:100].mean():.1%} · LoRA {ok_lo[:100].mean():.1%}")
solo_lo = int((ok_lo & ~ok_lr).sum())
solo_lr = int((ok_lr & ~ok_lo).sum())
p = binomtest(solo_lo, solo_lo + solo_lr).pvalue
print(f"only LoRA right: {solo_lo} · only logistic: {solo_lr}"
      f" · exact McNemar p = {p:.1e}")
```
```output
           logistic  LoRA
all (600)      89.3  94.3
English        87.6  91.2
Spanish        89.8  95.2
FinBERT's 100: FinBERT 36.0% · logistic 89.0% · LoRA 95.0%
only LoRA right: 54 · only logistic: 24 · exact McNemar p = 9.0e-04
```

LoRA gets 94.3% of the 600 test headlines right, against 89.3% for the logistic regression. The figure at the top
summarizes the three baselines. Since the two models are evaluated on the same headlines, the right comparison is
**paired**: the McNemar test looks only at the disagreements. Only LoRA gets 54 headlines right and only the logistic
regression, 24; if they were equally good, those 78 disagreements would split like coin flips, and a 54-24 split or a
more extreme one has a probability of $9.0 \times 10^{-4}$.

Where matters as much as how much. In English the improvement is 3.6 points (87.6% to 91.2%, with 137 headlines); in
Spanish, 5.4 (89.8% to 95.2%). The adapters mostly correct what the frozen encoder did not separate well. By event type:

```python
por_evento = (pd.DataFrame({"evento": te["evento"], "logistic": ok_lr,
                            "LoRA": ok_lo})
              .groupby("evento").agg(["mean", "size"]))
por_evento = pd.DataFrame({
    "n": por_evento[("LoRA", "size")],
    "logistic": (100 * por_evento[("logistic", "mean")]).round(0),
    "LoRA": (100 * por_evento[("LoRA", "mean")]).round(0)})
print(por_evento.sort_values("logistic").head(6).to_string())
```
```output
                    n  logistic   LoRA
evento
incidente          13      46.0   62.0
oferta_compra       6      50.0  100.0
rating_rebaja      19      74.0  100.0
previsiones        16      81.0   88.0
analista_baja      56      82.0   86.0
retirada_producto  17      82.0   76.0
```

The events the logistic regression classifies worst are fires and incidents (46% against 62% with LoRA), takeover offers
and rating downgrades: headlines whose sentiment depends on one or two words (“rebaja”, downgrade; “eleva”, raises;
“incendio”, fire) that an embedding of the whole sentence dilutes. LoRA does not win everywhere: on product recalls it
gets fewer right than the logistic regression (76% against 82%). And beware of the sample sizes: 6 takeover offers and
13 incidents are too few to claim anything event by event.

## Merging and serving the adapter

There are two options for serving the model: save only the adapter and load it on top of the original MiniLM, or merge
it into the weights with `merge_and_unload()`:

```python
x = next(lotes(te["titular"].iloc[:50], 50))
with torch.inference_mode():
    antes = clf(x)
clf.base = clf.base.merge_and_unload()
with torch.inference_mode():
    despues = clf(x)
dif = (antes - despues).abs().max().item()
print(type(clf.base).__name__, "· max logit difference < 1e-4:",
      dif < 1e-4)
print(f"adapter size in float32: {entren * 4 / 1024:.0f} KB"
      f" (the whole model: {total * 4 / 1024**2:.0f} MB)")
```
```output
BertModel · max logit difference < 1e-4: True
adapter size in float32: 288 KB (the whole model: 87 MB)
```

After merging, the model is a normal `BertModel` again, with the same outputs (differences on the order of float32
rounding) and no extra operations. Unmerged, the adapter takes 288 KB against the model's 87 MB: with LLMs of billions
of parameters, that difference is what makes it possible to keep one adapter per task, per client or per training date
on top of a single copy of the base model.

## The risk: memorizing instead of learning

73,728 parameters look like few, but they are far more than 300 examples. A placebo makes it clear: the same adapters
(without dropout) and the same head, trained on 100 headlines whose labels have been **shuffled at random** (four epochs,
to keep the run short). There is nothing to learn:

```python
torch.manual_seed(0)
cfg0 = LoraConfig(r=8, lora_alpha=16, lora_dropout=0.0,
                  target_modules=["query", "value"])
placebo = Clasificador(
    get_peft_model(AutoModel.from_pretrained(NOMBRE), cfg0))
pocos = tr.iloc[:100]
barajadas = np.random.default_rng(0).permutation(pocos["y"].to_numpy())
curva_placebo = entrenar(placebo, pocos, barajadas, epocas=4, lr=5e-3)
```
```output
epoch 1 · train 1.20 · validation 1.24 · validation accuracy 21.0%
epoch 2 · train 0.96 · validation 1.35 · validation accuracy 17.0%
epoch 3 · train 0.81 · validation 1.51 · validation accuracy 11.0%
epoch 4 · train 0.74 · validation 1.66 · validation accuracy 11.0%
```

*Fig. — Training and validation loss by epoch for MiniLM with LoRA: with the real labels (300 headlines) and with shuffled labels (100).* Two panels. With real labels both losses fall together: training from 1.04 to 0.24 and validation from 0.93 to 0.25 in four epochs. With shuffled labels, training loss falls from 1.20 to 0.74 while validation loss, highlighted, rises from 1.24 to 1.66: the model memorises noise.

The training loss falls from 1.20 to 0.74 in four epochs: the model memorizes which arbitrary label goes with each
headline. The validation loss **rises** from 1.24 to 1.66 and validation accuracy drops from 21% to 11%, far below the
36% of always predicting the most frequent class of those 100 headlines. Worse than chance: since the headlines are
templated, what is memorized is a false rule (“fires are good news”) that is applied confidently to new headlines of the
same type. Without a separate validation set, the placebo's training curve is indistinguishable from that of a model
that learns.

The defenses are those of levels 2 and 5, applied to a language model:

- **Time-based split** and a test set looked at once. A 2024 headline with the same template as one from 2020 is
  correlated with it; a random split would mix near-duplicates between training and test and give inflated figures.
- **Cheap baselines** on the same test set: an untouched model and a logistic regression on embeddings. If the fine-tune
  does not beat them with a paired test, it is not worth it.
- **Placebos and controls**: shuffled labels, or a task unrelated to the text. A fine-tune that “learns” there too is
  measuring its capacity to memorize.
- **Count the experiments**: rank, layers, learning rate, epochs and seeds are hyperparameters. Choosing the best of 50
  on the same validation set has the selection problem of 2.5.4 and 5.5.4.
- **The training text has a date too**: fine-tuning a model with news up to 2025 and backtesting with news from 2022 is
  look-ahead. With LLMs the problem goes beyond your own labels and is the subject of lesson 6.2.5.

## Exercises

1. With the lesson's `cuenta` function, compare rank-64 LoRA on `q_proj` and `v_proj` only with rank-8 LoRA on every
   linear layer of Qwen2.5-7B. Which one trains more parameters and how much memory would each need with QLoRA?

**Solution.**

```python
for nombre, mods, r in [("r=64 on q, v", ["q_proj", "v_proj"], 64),
                        ("r=8 on all", TODAS, 8)]:
    t, li, lo = cuenta(q7, mods, r=r)
    print(f"{nombre:14s} LoRA {lo / 1e6:5.1f} M · QLoRA "
          f"{memoria_gb(t, li, lo)[2]:.1f} GB")
```
```output
r=64 on q, v   LoRA  20.2 M · QLoRA 5.9 GB
r=8 on all     LoRA  20.2 M · QLoRA 5.9 GB
```

They tie: 20.2 million parameters and 5.9 GB with QLoRA. It is not a rounding coincidence. Per block, rank 64 on
`q_proj` (3,584 to 3,584) and `v_proj` (3,584 to 512, because of the 4 key-value heads) gives $64 \cdot (7{,}168 +
4{,}096) = 720{,}896$ parameters, and rank 8 on the seven projections gives $8 \cdot (2 \cdot 7{,}168 + 2 \cdot 4{,}096
+ 3 \cdot 22{,}528) = 720{,}896$. With the same budget, the question is where to put it, and the QLoRA paper answers
on every linear layer: concentrating it in the attention did not match full fine-tuning in their experiments, and the
rank barely mattered. Memory barely depends on this choice because the quantized base dominates it.

2. Compute the confusion matrix of the logistic regression and of LoRA on the test set. Which confusions does LoRA
   correct?

**Solution.**

```python
from sklearn.metrics import confusion_matrix

nombres = ["neg", "neutral", "pos"]
for m, p in [("logistic", pred_lr), ("LoRA", pred_lora)]:
    print(m)
    print(pd.DataFrame(confusion_matrix(y_te, p), index=nombres,
                       columns=nombres).to_string())
```
```output
logistic
         neg  neutral  pos
neg      176        6   25
neutral    4      133    1
pos       20        8  227
LoRA
         neg  neutral  pos
neg      175        1   31
neutral    0      138    0
pos        1        1  253
```

Rows are the true class and columns, the predicted one. The logistic regression does not go wrong mainly with the
neutral ones, as one might expect, but with **polarity**: 25 negatives classified as positive and 20 positives as
negative. It is what lesson 6.3.1 anticipates: the MiniLM embedding of “La empresa eleva su previsión de ventas” (“The
company raises its sales forecast”) and that of “La empresa recorta su previsión de ventas” (“The company cuts its
sales forecast”) have a cosine similarity of 0.95, because the frozen model measures what the sentence is about more
than which way it points. LoRA gets every neutral one right and 253 of 255 positives, but still confuses negatives with
positives (31, more than the logistic regression): with four epochs it has learned to recognize good news well and bad
news not quite yet. It is a reminder that an aggregate improvement (89.3% to 94.3%) can hide a specific error that gets
worse, and in trading, sign errors are the expensive ones.

3. A colleague fine-tunes LoRA with 3,000 headlines from 2019-2025 split at random between training and test and gets
   99.5% accuracy. They propose using the model to trade tomorrow's news. What is left to check?

**Solution.**

Almost everything. The random split mixes nearly identical headlines (the same template, another company or another
date) between training and test, so the 99.5% partly measures memory; a test set later in time than all the training
data is needed. The comparison with cheap baselines on that test set is missing (a logistic regression on embeddings
with the same labels), and so is a paired test. A placebo is missing, to know how much the model can memorize with those
labels. And, above all, getting the sentiment right is not making money: the price reaction to a news item happens
within minutes (lesson 6.1.3), so the signal has to exist **before** the market incorporates it, with the publication
time and the costs of level 2. Finally, these headlines are templated: with real text, the accuracy of any method will
be much lower.

[^1]: Qwen2.5-7B-Instruct model card on Hugging Face (accessed 2026-10-23): 7.61B parameters, 6.53B excluding
    embeddings, 28 layers, 28 query heads and 4 key-value heads, Apache-2.0 license.

## Self-assessment

1. A 384×384 linear layer is adapted with rank-8 LoRA. How many new parameters are trained in that layer?

   - A. 147,456, the same as the original matrix
   - B. 3,072, half of r·(d + k)
   - C. 6,144: A is 8×384 and B is 384×8
   - D. 64, the square of the rank

   *Answer C: 6,144: A is 8×384 and B is 384×8.* LoRA replaces the update ΔW (384×384) with the product B·A, where A is r×384 and B is 384×r: r·(d + k) = 8·768 = 6,144 parameters, 4.2% of the matrix. The original matrix stays frozen.

2. When the model is created with peft, the B matrix of each adapter is zero. What for?

   - A. So that, before training, the adapted model gives exactly the same output as the original and the fine-tune starts from the pretrained model
   - B. To save memory
   - C. So that the gradient of A is zero at the first step and it never moves
   - D. It is an initialization bug that has to be fixed

   *Answer A: So that, before training, the adapted model gives exactly the same output as the original and the fine-tune starts from the pretrained model.* With B = 0, B·A = 0 and the output does not change; A is initialized at random so that the gradient of B is not zero. In the lesson, the logistic regression uses the embeddings with the adapter disabled, and those of the freshly created adapter would be identical.

3. According to the lesson's math, why does QLoRA make it possible to fine-tune Qwen2.5-7B with about 6 GB when LoRA with the base in bf16 needs about 16 GB?

   - A. Because it trains fewer parameters than LoRA
   - B. Because it does not use Adam
   - C. Because it eliminates activations
   - D. Because it stores the frozen weights of the linear layers in NF4 (about 4.1 bits per parameter with double quantization) instead of 16 bits; the adapters and the optimizer are the same

   *Answer D: Because it stores the frozen weights of the linear layers in NF4 (about 4.1 bits per parameter with double quantization) instead of 16 bits; the adapters and the optimizer are the same.* The 20 million LoRA parameters and their Adam state weigh the same in both cases (0.3 GB). The difference is in the base: 6,525 million linear weights go from 2 bytes to about 0.52. The math does not include activations, which depend on the batch and the sequence length.

4. With 300 training headlines, LoRA gets 94.3% right and the logistic regression on frozen embeddings 89.3% on the same test set of 600. Only LoRA gets 54 headlines right and only the logistic regression, 24. What can be concluded?

   - A. Nothing, because 5 points could be noise with 600 headlines
   - B. That the difference is significant on this data (exact McNemar, p ≈ 0.001) and that adapting the encoder helps mostly in Spanish; not that it will repeat with real texts
   - C. That LoRA always beats a logistic regression
   - D. That the logistic regression is badly regularized

   *Answer B: That the difference is significant on this data (exact McNemar, p ≈ 0.001) and that adapting the encoder helps mostly in Spanish; not that it will repeat with real texts.* The paired test uses only the disagreements (54 against 24). The improvement is larger in Spanish (89.8% to 95.2%), the language MiniLM knows worse, than in English (87.6% to 91.2%). The headlines come from templates by event type: it is a much easier problem than real text.

5. The same adapters, trained with shuffled labels, bring the training loss down from 1.20 to 0.74 in four epochs while the validation loss rises from 1.24 to 1.66. What does the placebo teach?

   - A. That the model has found a hidden signal
   - B. That the learning rate is too low
   - C. That 73,728 parameters are enough to memorize 100 labels unrelated to the text: a falling training loss proves nothing without a validation set separated in time
   - D. That more epochs are needed

   *Answer C: That 73,728 parameters are enough to memorize 100 labels unrelated to the text: a falling training loss proves nothing without a validation set separated in time.* With random labels there is nothing to learn; what goes down is memory. Validation accuracy falls to 11%, far below the 36% of always predicting the most frequent class: the model has learned a false rule (which label goes with each event type) and applies it with confidence. It is the same overfitting lesson as 5.8.3 with another model.

## Key points

- LoRA freezes W and trains a rank-r B·A: in MiniLM, 73,728 parameters out of 22.8 million (0.32%); in Qwen2.5-7B with r = 8 in every linear layer, 20.2 million (0.27%).
- Memory without activations for Qwen2.5-7B: 121.8 GB with full fine-tuning in fp32 and Adam, 15.6 GB with LoRA and the base in bf16, 5.9 GB with QLoRA (NF4 and double quantization, 4.127 bits per linear weight).
- On synthetic headlines with 300 labels, LoRA on MiniLM gets 94.3% right in 2024-2025, against 89.3% for a logistic regression on the frozen embeddings and 36.0% for an untrained FinBERT (which does not understand Spanish).
- The adapter is merged into W after training: the resulting model gives the same outputs and adds no latency; unmerged, it takes 288 KB against the model's 87 MB.
- The shuffled-label placebo memorizes: the training loss falls and the validation loss rises. Time-based validation, a cheap baseline and a placebo are the minimum before believing a fine-tune.

## Resources

### Documentation

- [PEFT (Hugging Face): LoRA guide](https://huggingface.co/docs/peft/main/en/developer_guides/lora) · huggingface.co
- [Transformers (Hugging Face): quantization with bitsandbytes (NF4, double quantization)](https://huggingface.co/docs/transformers/main/en/quantization/bitsandbytes) · huggingface.co
- [ProsusAI/finbert: model card (CC-BY-NC-SA-4.0 license)](https://huggingface.co/ProsusAI/finbert) · huggingface.co

### Books

- [Hu, E. J. et al. (2021): LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685](https://arxiv.org/abs/2106.09685) · arxiv.org
- [Dettmers, T. et al. (2023): QLoRA: Efficient Finetuning of Quantized LLMs. arXiv:2305.14314](https://arxiv.org/abs/2305.14314) · arxiv.org

### Data

- [Level 6 data: news corpus (Parquet)](https://pythonparatrading.com/curso/datos/nivel-6/noticias6_corpus.parquet) · Download · noticias6_corpus.parquet

Previous lesson: [6.2.3 · Guidance, KPIs and tone: what an LLM extracts well](https://pythonparatrading.com/en/python-trading-course/level-6/extracting-guidance-kpis-and-tone-with-llms/)

Next lesson: [6.2.5 · Look-ahead bias in LLMs: the model that already knows the future](https://pythonparatrading.com/en/python-trading-course/level-6/look-ahead-bias-in-llms/)