---
title: "Level 6: generative AI, LLMs, agents and alternative data"
description: "Level 6 of the Python Trading Course: financial NLP, LLMs, RAG, agents with guardrails, AI as a copilot, big data and alternative data."
url: "https://pythonparatrading.com/en/python-trading-course/level-6/"
lang: "en"
image: "https://pythonparatrading.com/og/en/python-trading-course/level-6.png"
---

# Level 6: Generative AI, LLMs, agents, big data and alternative data

> LLMs as assistant analysts and engineers, not as autonomous traders: financial text, RAG, agents with deterministic guardrails, big data and alternative data with due diligence.

- Hours: 90–110
- Prerequisite: Level 5 or equivalent ML
- Profile: Quant or ML engineer
- Course: [Python Trading Course](https://pythonparatrading.com/en/python-trading-course/)

## Classical and modern financial NLP

- `6.1.1` Tokenisation, embeddings; financial dictionaries (Loughran-McDonald), FinBERT.
- `6.1.2` Sources: news, 10-K, 10-Q and 8-K (EDGAR), CNMV material disclosures (hechos relevantes), earnings call transcripts, social media.
- `6.1.3` Event studies with text signals; precise time alignment (publication timestamps); topic modelling.

## LLMs for research and trading

- `6.2.1` How LLMs work (context, temperature, hallucinations); commercial APIs versus local models (Ollama, vLLM).
- `6.2.2` Structured prompting and JSON outputs validated with Pydantic.
- `6.2.3` Information extraction: guidance, KPIs and tone of conference calls; LLM sentiment versus FinBERT.
- `6.2.4` Lightweight fine-tuning (LoRA, QLoRA) for financial tasks.
- `6.2.5` Look-ahead bias in LLMs: the model “knows the future” of its training corpus; backtests with cut-off dates and point-in-time models.

## Financial RAG

- `6.3.1` Embeddings and vector databases (FAISS, pgvector, Chroma, Qdrant).
- `6.3.2` Chunking financial documents: tables, footnotes, XBRL.
- `6.3.3` RAG over annual reports, ETF prospectuses and KIDs and in-house research; evaluation (faithfulness, citation, coverage).

## AI agents for trading

- `6.4.1` Agents with tools (tool calling), ReAct, planning.
- `6.4.2` Frameworks: LangGraph, LlamaIndex, provider SDK agents; MCP (Model Context Protocol) to connect data and brokers.
- `6.4.3` TradingAgents-style multi-agent architectures: fundamental, sentiment, news and technical analysts, bull and bear debate, trader and risk manager.
- `6.4.4` Evidence and limits: in LiveTradeBench (50 live sessions, 21 LLMs), the Spearman correlation between LMArena score and cumulative return on equities was only 0.054 (−0.38 on Polymarket). GPT-4.1 gained more than 6% on equities and lost more than 30% on Polymarket. AlphaForgeBench documents “drastically different” actions across runs of the same model, even with deterministic decoding. Deep learning models that are strong in backtest degrade in live trading.
- `6.4.5` Guardrails: the agent proposes and the deterministic risk engine decides; hard limits, human approval, audit log.
- `6.4.6` Research agents: hypothesis generation, assisted alpha mining, code and backtest review.

*Fig. — LiveTradeBench: correlation between LMArena score and the returns of 21 LLMs.* Two bars with the Spearman correlation between the general LMArena score of 21 language models and their cumulative return over 50 live sessions: 0.054 in stocks, close to zero, and −0.38 in Polymarket, negative.

## AI as the quant's copilot

- `6.5.1` Coding assistants (Copilot, Claude Code, Codex, Cursor): a tests-first workflow.
- `6.5.2` Automatic review of backtests for biases; migrations (pandas 2 → 3, backtrader → NautilusTrader).
- `6.5.3` Risks: plausible but incorrect code, invented dependencies, leaked secrets.

## Big data for trading

- `6.6.1` When it is “big”: tick data, L2 and L3 books, options (OPRA), global universes.
- `6.6.2` Columnar formats (Parquet, Arrow); lazy and streaming Polars; out-of-core DuckDB.
- `6.6.3` Dask and Spark (PySpark) for clusters; Ray to parallelise experiments.
- `6.6.4` Time series databases: TimescaleDB, QuestDB, ClickHouse, ArcticDB; kdb+ as professional background.
- `6.6.5` Cloud storage (S3, GCS), lakehouse (Delta, Iceberg) and GPU (RAPIDS, cuDF) as background.

## Alternative data

- `6.7.1` Landscape and market size: Neudata figures and caveats about consultancy reports with widely varying estimates.
- `6.7.2` Types: web scraping (prices, jobs, reviews), web and app traffic, card transactions, receipts, geolocation, satellite, shipping (AIS), weather, patents, public data, on-chain.
- `6.7.3` Responsible web scraping: requests and httpx, BeautifulSoup, Playwright; robots.txt, terms of use, GDPR.
- `6.7.4` Providers and due diligence (Neudata, Nasdaq Data Link, Eagle Alpha): point-in-time history, panel biases, legality (inside information, personal data).
- `6.7.5` From data to signal: mapping to tickers (entity resolution), sales nowcasting, signal backtest, alpha decay.
- `6.7.6` Case studies: satellite imagery with computer vision (CNN, ViT) and supply chain graphs.

## Level project

A research system that combines RAG over filings and transcripts, an LLM sentiment signal validated with an event study and a backtest, and an agent that writes daily reports, with guardrails and no direct access to orders.

Previous level: [Level 5: Machine learning and deep learning (Keras/PyTorch)](https://pythonparatrading.com/en/python-trading-course/level-5/)

Next level: [Level 7: Production, MLOps, regulation, career and capstone](https://pythonparatrading.com/en/python-trading-course/level-7/)