SYS-03RepositoryJupyter Notebook

Quantstack

Research and backtesting in one container. Once the image is built, one command starts JupyterLab with Zipline and ArcticDB.

Stars
7
Language
Jupyter Notebook

What it is

Quantstack is a research and backtesting platform for algorithmic trading with Zipline, ArcticDB and JupyterLab, packaged in Docker. The image starts from Miniconda on Debian.

It solves an environment problem. Research libraries were not designed to be used together, and in a team each member works with a different operating system and configuration. The container fixes the system, the package manager and the tools for everyone. Once the image is built, docker compose up -d brings the environment up.

What's included

  • docker-compose.yml: notebook service on port 8888, with JUPYTER_TOKEN (default testing). It mounts quantarmy_repo/ as the working folder (/WorkingData).
  • DockerSource/Dockerfile: continuumio/miniconda3 image. Installs JupyterLab and TA-Lib with conda, and the rest of req.txt with pip. Copies the Zipline extension, the connector and the .env file into the image.
  • DockerSource/req.txt: zipline-reloaded, pyfolio-reloaded, alphalens-reloaded, empyrical-reloaded and exchange_calendars; arcticdb and yfinance; pandas, numpy, scipy, scikit-learn, matplotlib, plotly, seaborn and pyarrow; JupyterLab extensions (git, LSP, system monitor, execution time).
  • DockerSource/extension.py: registers the qa_datalake bundle with the NYSE calendar.
  • DockerSource/qa_datalake.py: ArcticDB → Zipline connector. Reads prices, dividends and splits for the ETFs in the table from ArcticDB, from 2010, and writes them into the bundle.
  • DockerSource/overrides.json: JupyterLab settings (theme, fonts).
  • quantarmy_repo/: notebooks. backtest/ (template template_zipline_jupyterlab.ipynb and the volatility example 01-bt-zipline-variance.ipynb), datalake/ (connection, contents and exploration of the datalake), models/ (01_101_ALPHAS.ipynb) and tools/ (zipline_db_management.ipynb). analisis/, theory/ and howtos/ hold an empty notebook for now.

ETFs in the qa_datalake bundle:

TickerDescription
XLE.USEnergy Select Sector
XLF.USFinancial Select Sector
XLI.USIndustrial Select Sector
XLK.USTechnology Select Sector
XLP.USConsumer Staples Select Sector
XLU.USUtilities Select Sector
XLV.USHealth Care Select Sector
XLY.USConsumer Discretionary Select Sector
XLB.USMaterials Select Sector
XLC.USCommunication Services Select Sector
ITA.USiShares U.S. Aerospace & Defense

Installation and first run

Requirements: Docker and Docker Compose, and access credentials for the QA Datalake (S3/ArcticDB).

git clone https://github.com/quantarmyz/quantstack.git
cd quantstack

Before building, create DockerSource/.env with the datalake credentials. The Dockerfile copies it into the image (/root/.env).

IN [01]DockerSource/.envdotenv
ENDPOINT=your-endpoint.com
DB=your-bucket
ACCESS_KEY=your-access-key
SECRET_KEY=your-secret-key

Build the image and start the stack, in the background with -d:

docker compose build --no-cache
docker compose up -d

JupyterLab runs at http://localhost:8888, with the password testing. To change it, edit JUPYTER_TOKEN in docker-compose.yml.

From a JupyterLab terminal, ingest the datalake data into the Zipline bundle:

zipline ingest -b qa_datalake

Then, in backtest/, template_zipline_jupyterlab.ipynb is the starting point for a new backtest.

How it fits

Quantstack is the central piece of the lab chain. Its connector reads from the Datalake on ArcticDB, with the same four environment variables, and feeds a Zipline bundle. Part 4 of the Big Data Management series explains that connector. For the report, the image ships pyfolio-reloaded; QAStats, the lab's reporting library, is installed separately.

Fig. — Data, backtest and report: Datalake → Quantstack (Zipline) → QAStats

Other systems