Lesson 0.4.4 · 50 min

Files (CSV, JSON, YAML, TOML) and dates with time zones

A trading system's data lives in files, and its times live in time zones. This lesson covers the four usual text formats with pathlib and the zoneinfo module, with which the New York open stops being "at 15:30" and becomes what it is: a local time that moves.

Fig. — Madrid time of the New York Stock Exchange open (09:30 in New York), every Monday of 2026
04/06
Module
0.4 · Intermediate Python
Duration
50 min
Updated
Level 0 lessons
  1. 0.1 · Welcome and expectations

    1. 0.1.1 Systematic, algorithmic and quantitative trading versus discretionary trading
    2. 0.1.2 Realistic expectations: costs, time and risk of loss
    3. 0.1.3 Course structure, entry tests, personal repository and learning with AI
  2. 0.2 · Modern working environment

    1. 0.2.1 Installing Python with uv: versions and support cycle
    2. 0.2.2 Virtual environments and dependencies: uv init, uv add, pyproject.toml and lockfiles
    3. 0.2.3 Editors (VS Code, Cursor, PyCharm) and notebooks (Jupyter, JupyterLab, marimo)
    4. 0.2.4 Basic terminal, paths, environment variables and .env files
    5. 0.2.5 Git and GitHub: commits, branches, pull requests and .gitignore
    6. 0.2.6 Introduction to AI coding assistants: prompts, review and tests
  3. 0.3 · Language fundamentals

    1. 0.3.1 Variables, types and operators; floating point and decimal for prices and ticks
    2. 0.3.2 Strings, f-strings and formatting numbers and dates
    3. 0.3.3 Lists, tuples, dictionaries, sets and comprehensions
    4. 0.3.4 Control flow: if, elif, else, for, while and match-case
    5. 0.3.5 Functions: arguments, *args and **kwargs, lambdas and scope
    6. 0.3.6 Exceptions; modules and packages, with Python 3.15 lazy imports
  4. 0.4 · Intermediate Python

    1. 0.4.1 Object-oriented programming: classes, inheritance, composition and dataclasses
    2. 0.4.2 Typing: type hints, mypy or ty, and Pydantic for configurations
    3. 0.4.3 Iterators, generators, decorators and context managers
    4. 0.4.4 Files (CSV, JSON, YAML, TOML) and dates with time zones
    5. 0.4.5 Professional logging and testing with pytest
    6. 0.4.6 Concurrency: threading, multiprocessing, asyncio and free-threading
  5. 0.5 · Numerical computing and data

    1. 0.5.1 NumPy 2: arrays, broadcasting, vectorization, linear algebra and random generators
    2. 0.5.2 pandas 3.0: Series, DataFrame, time indexes, Copy-on-Write, the str type and pd.col()
    3. 0.5.3 Selection, filtering, groupby, merge, pivot, resample and moving windows
    4. 0.5.4 Polars: lazy versus eager API, expressions, performance and when to pick it over pandas
    5. 0.5.5 DuckDB and essential SQL: SELECT, JOIN, GROUP BY and window functions on Parquet
  6. 0.6 · Visualisation

    1. 0.6.1 Object-oriented Matplotlib, Seaborn and Plotly: price, drawdown and candlestick charts
    2. 0.6.2 Quick dashboards with Streamlit: a monitoring dashboard in a single script
  7. 0.7 · Minimum mathematics and statistics

    1. 0.7.1 Simple and log returns, compound interest and descriptive statistics
    2. 0.7.2 Probability and distributions, correlation versus causation and simple regression
    3. 0.7.3 Intuitive linear algebra: vectors and matrices as groundwork for portfolios

Paths with pathlib

A trading system reads prices, configurations and trades, and writes reports, logs and backtest results. Before opening a file you need to know where it is, and paths written as text ("datos/" + simbolo + ".csv") break easily: too many or too few slashes, different separators on Windows and Linux, duplicated extensions.

pathlib.Path represents a path as an object. The / operator joins parts, and its attributes give the name, the extension or the folder without manipulating text:

IN [01]python
from pathlib import Path

for p in sorted(Path(".").glob("*.csv")):
    print(f"{p.name:<22} {p.stem:<16} {p.stat().st_size:>8,} B")

informes = Path("informes") / "2025" / "sint"
informes.mkdir(parents=True, exist_ok=True)
destino = informes / "resumen.csv"
print(destino, destino.suffix, destino.parent.exists())
OUT [01]The level's data files
cartera_sint.csv       cartera_sint       49,207 B
operaciones_sint.csv   operaciones_sint    2,440 B
sint_diario.csv        sint_diario        60,757 B
informes/2025/sint/resumen.csv .csv True

glob("*.csv") lists the files that match the pattern (rglob also searches subfolders). mkdir with parents=True creates the intermediate folders, and with exist_ok=True it does not fail if they already exist: a reporting script can run every day without checking anything first.

Relative paths, such as Path(".") or "sint_diario.csv", are resolved from the working directory, the place the program was launched from (lesson 0.2.4). A script that works in the terminal and fails in a task scheduler almost always has that problem. The usual solution is to build paths from the location of the script itself: Path(__file__).parent / "datos".

CSV: everything arrives as text

CSV is the most common format for prices and trades: one row per record, columns separated by commas and a header. The standard library's csv module takes care of quotes and separators; DictReader delivers each row as a dictionary with the header's keys. What it does not do is convert types: everything arrives as text.

IN [02]python
import csv

with open("operaciones_sint.csv", newline="", encoding="utf-8") as f:
    filas = list(csv.DictReader(f))

print(len(filas), list(filas[0]))
print(filas[0]["cantidad"], type(filas[0]["cantidad"]).__name__)

def pnl(fila: dict[str, str]) -> float:
    signo = 1 if fila["lado"] == "largo" else -1
    bruto = signo * int(fila["cantidad"]) * (
        float(fila["precio_salida"]) - float(fila["precio_entrada"]))
    return bruto - float(fila["comision"])

por_simbolo: dict[str, float] = {}
for fila in filas:
    s = fila["simbolo"]
    por_simbolo[s] = por_simbolo.get(s, 0.0) + pnl(fila)
for s, total in sorted(por_simbolo.items()):
    print(f"{s:<5} {total:>10,.2f}")
OUT [02]Reading the SINT trades
40 ['id', 'fecha_entrada', 'fecha_salida', 'simbolo', 'lado', 'cantidad', 'precio_entrada', 'precio_salida', 'comision']
200 str
ORO    -3,237.96
RVAR       17.09
SINT    4,405.08
TECH      -35.02

"200" is text: adding to it would raise an error and multiplying it would repeat the string (lesson 0.4.2). That is why every numeric field goes through int() or float() before any arithmetic. pandas (lesson 0.5.2) infers the types on its own, and even so it is worth checking them.

Writing is symmetrical. csv.writer takes rows as lists and handles separators and quotes. The newline="" argument to open is not decorative: without it, on Windows each row would end with an extra line break.

IN [03]python
with destino.open("w", newline="", encoding="utf-8") as f:
    w = csv.writer(f)
    w.writerow(["simbolo", "pnl"])
    for s, total in sorted(por_simbolo.items()):
        w.writerow([s, round(total, 2)])

print(destino.read_text(encoding="utf-8"))
OUT [03]Writing a summary
simbolo,pnl
ORO,-3237.96
RVAR,17.09
SINT,4405.08
TECH,-35.02

A typical trap in Spain, and in many other European locales: CSV files exported from a spreadsheet set to Spanish use ; as the separator and a decimal comma (142,22). float("142,22") raises ValueError. It is read with csv.reader(f, delimiter=";") and the comma is replaced by a period before converting, or with pd.read_csv(sep=";", decimal=","). What you must never do is remove the comma: 14222 is a different price.

JSON: no dates or decimals

JSON is the format of APIs: almost all brokers and data providers respond in JSON. It supports objects (dictionaries), lists, text, numbers, true, false and null. Nothing else: no dates, no Decimal, no sets.

IN [04]python
import json
from datetime import date

resumen = {"estrategia": "cruce-50-200", "fecha": date(2025, 12, 31),
           "operaciones": len(filas),
           "pnl": {s: round(v, 2) for s, v in por_simbolo.items()}}
try:
    json.dumps(resumen)
except TypeError as e:
    print("TypeError:", e)

resumen["fecha"] = resumen["fecha"].isoformat()
texto = json.dumps(resumen, ensure_ascii=False, indent=2)
Path("resumen.json").write_text(texto, encoding="utf-8")
leido = json.loads(Path("resumen.json").read_text(encoding="utf-8"))
print(leido["fecha"], type(leido["fecha"]).__name__)
print(leido == resumen)
OUT [04]Saving a summary as JSON
TypeError: Object of type date is not JSON serializable
2025-12-31 str
True

The date is saved as ISO 8601 text (2025-12-31) and comes back as text: whoever reads the JSON has to know that the field is a date and convert it with date.fromisoformat. That is the work Pydantic does (lesson 0.4.2) when it validates an API response: the model says the field is a date and Pydantic converts it.

Two more details. ensure_ascii=False keeps accented characters as they are instead of escapes such as \u00f3. And json.dumps writes NaN for a not-a-number float, which is not standard JSON: many readers, browsers included, reject it. With allow_nan=False the error is raised when writing, not when reading.

YAML and TOML: configurations

Configurations are written by hand, and JSON is awkward for that: it does not allow comments and requires quotes around everything. The two usual formats are YAML and TOML. YAML is read with PyYAML (lesson 0.4.2); TOML, with tomllib, part of the standard library since Python 3.11.

YAML guesses types from the text, and sometimes it guesses wrong. PyYAML follows version 1.1 of the standard, where yes, no, on and off are booleans. There are companies listed under those symbols:

IN [05]python
import yaml

Path("config.yaml").write_text("""\
simbolos: [SPY, ON, NO, "ON"]
inicio: 2021-01-04
version_modelo: 1.10
""", encoding="utf-8")
print(yaml.safe_load(Path("config.yaml").read_text(encoding="utf-8")))
OUT [05]YAML guesses the types
{'simbolos': ['SPY', True, False, 'ON'], 'inicio': datetime.date(2021, 1, 4), 'version_modelo': 1.1}

ON (onsemi, on the Nasdaq) has become True and NO False; only the quoted "ON" is still text. The unquoted date has become a date, and version 1.10 the number 1.1. In a list of symbols the error raises no exception: the strategy simply does not trade those assets, or fails much later when requesting the price of True. That is why configurations are validated with a model (lesson 0.4.2) that requires list[str].

TOML requires quotes around all text, so it does not have that problem. It is the format of pyproject.toml (lesson 0.2.2). tomllib.load needs the file opened in binary mode ("rb"):

IN [06]python
import tomllib

Path("estrategia.toml").write_text("""\
[estrategia]
nombre = "cruce"
simbolos = ["SPY", "ON", "NO"]
inicio = 2021-01-04

[costes]
comision_pb = 1.5
deslizamiento_pb = 3.0
""", encoding="utf-8")
with open("estrategia.toml", "rb") as f:
    cfg = tomllib.load(f)
print(cfg["estrategia"])
print(sum(cfg["costes"].values()))
OUT [06]TOML with tomllib
{'nombre': 'cruce', 'simbolos': ['SPY', 'ON', 'NO'], 'inicio': datetime.date(2021, 1, 4)}
4.5

The symbols are still text and the date is still a date. tomllib only reads: writing TOML requires an external package (tomli-w). That fits how it is used: a configuration is written by hand and the program reads it.

FormatReading in PythonCommentsTypesTypical use
CSVcsv (stdlib), pandasNoAll textPrices, trades
JSONjson (stdlib)NoBasic, no datesAPIs, results
YAMLPyYAML (safe_load)YesGuesses them (careful)Configurations
TOMLtomllib (stdlib, read-only)YesExplicit, with datespyproject.toml, configurations

Dates with time zones: zoneinfo

A datetime without a time zone (naive) is a clock time without a place: "March 9, 2026, 09:30" does not say whether it is in New York, Madrid or Tokyo. One with a zone (aware) does represent a specific instant. zoneinfo, part of the standard library since Python 3.9, provides the zones of the IANA database, with their full history of clock changes, by name: America/New_York, Europe/Madrid, Asia/Tokyo.

The NYSE core session runs from 09:30 to 16:00, New York time.1Note 1 · NYSE, "Holidays & Trading Hours": Core Trading Session: 9:30 a.m. to 4:00 p.m. ET. Link in "Resources". Converting the open to other zones is a call to astimezone:

IN [07]python
from datetime import datetime, time, timedelta
from zoneinfo import ZoneInfo

NY = ZoneInfo("America/New_York")
MAD = ZoneInfo("Europe/Madrid")
UTC = ZoneInfo("UTC")

for dia in (date(2026, 3, 2), date(2026, 3, 9), date(2026, 3, 30)):
    apertura = datetime.combine(dia, time(9, 30), NY)
    print(apertura.isoformat(), "→",
          apertura.astimezone(MAD).strftime("%H:%M Madrid"), "|",
          apertura.astimezone(UTC).strftime("%H:%M UTC"))
OUT [07]The New York open in Madrid and in UTC
2026-03-02T09:30:00-05:00 → 15:30 Madrid | 14:30 UTC
2026-03-09T09:30:00-04:00 → 14:30 Madrid | 13:30 UTC
2026-03-30T09:30:00-04:00 → 15:30 Madrid | 13:30 UTC

The same open, at the same New York time, falls at 15:30 Madrid time on March 2, at 14:30 on the 9th and again at 15:30 on the 30th. The shift comes from the clock-change rules: the United States moves its clocks forward on the second Sunday of March and back on the first Sunday of November;2Note 2 · NIST, "Daylight Saving Time Rules": daylight saving time begins on the second Sunday of March at 2:00 a.m. and ends on the first Sunday of November at 2:00 a.m., local time. Link in "Resources". the European Union does it on the last Sunday of March and the last Sunday of October.3Note 3 · Directive 2000/84/EC, transposed into Spanish law by Royal Decree 236/2002: the last Sunday of March and the last Sunday of October, at 1:00 UTC. Link in "Resources". Between those dates, the difference between the two cities is 5 hours instead of 6. In 2026, on business days:

IN [08]python
def diferencia(dia: date) -> timedelta:
    """Hours between Madrid and New York at the NYSE open."""
    ap = datetime.combine(dia, time(9, 30), NY)
    return ap.astimezone(MAD).utcoffset() - ap.utcoffset()

dias = [date(2026, 1, 1) + timedelta(days=i) for i in range(365)]
habiles = [d for d in dias if d.weekday() < 5]
raros = [d for d in habiles if diferencia(d) != timedelta(hours=6)]
print(len(raros), "business days with", diferencia(raros[0]))
tramos = [[raros[0]]]
for ayer, hoy in zip(raros, raros[1:]):
    if (hoy - ayer).days > 3:
        tramos.append([])
    tramos[-1].append(hoy)
for t in tramos:
    print(f"  from {t[0]} to {t[-1]}: {len(t)} days")
OUT [08]The days of 2026 with a 5-hour difference
20 business days with 5:00:00
  from 2026-03-09 to 2026-03-27: 15 days
  from 2026-10-26 to 2026-10-30: 5 days

Twenty business days in 2026 on which New York opens an hour earlier for someone trading from Madrid: from March 9 to 27 and from October 26 to 30. The header figure plots every Monday of 2026 with the Madrid time of the open; the four highlighted weeks are the shifted ones.

The consequences are practical. A program that sends orders "at 15:25 Madrid time, five minutes before the open" arrives an hour late in those weeks. A filter that discards "the first half hour of New York" using fixed Madrid times is applied at the wrong time. And a backtest with intraday bars in Madrid local time mixes two session schedules without anyone noticing. The solution is always the same: express each market's hours in its own zone and convert.

Times that do not exist and times that repeat

The clock change creates two anomalies in local time. In Madrid, on March 29, 2026 the clock jumps from 02:00 to 03:00: 02:30 on that day does not exist. On October 25, it goes from 03:00 back to 02:00: 02:30 happens twice. zoneinfo raises no errors; the fold attribute (0 or 1) chooses between the two occurrences of a repeated time:

IN [09]python
inexistente = datetime(2026, 3, 29, 2, 30, tzinfo=MAD)
print(inexistente.astimezone(UTC).astimezone(MAD))

for fold in (0, 1):
    repetida = datetime(2026, 10, 25, 2, 30, tzinfo=MAD, fold=fold)
    print(fold, repetida.astimezone(UTC).strftime("%H:%M UTC"),
          repetida.tzname())

try:
    print(datetime(2026, 3, 9, 15, 0) < datetime.now(UTC))
except TypeError as e:
    print("TypeError:", e)
OUT [09]Nonexistent and repeated times in Madrid
2026-03-29 03:30:00+02:00
0 00:30 UTC CEST
1 01:30 UTC CET
TypeError: can't compare offset-naive and offset-aware datetimes

The nonexistent time, after a round trip through UTC, becomes 03:30 summer time. The repeated one has two different instants, one hour apart: 00:30 and 01:30 UTC. And Python refuses to compare a date without a zone with one that has a zone, rather than assume anything.

Markets almost never trade at those hours, but data does get generated: cryptocurrency files, which trade 24 hours a day, or a server's logs in local time, have a duplicated hour every fall and a gap every spring. The rules that avoid almost all problems:

  • Store in UTC. Timestamps of data, orders and logs go in UTC, which has no clock changes.
  • Convert for display. Local time is for people: reports, charts, alerts.
  • Define market hours in their own zone. "09:30 America/New_York", not "15:30 Madrid time".
  • Never mix naive and aware. If a file comes with dates without a zone, decide when reading it which zone they are in and convert them.

The gap between closes is computed as before, each one in its own zone:

IN [10]python
dia = date(2025, 12, 31)
cierre_bme = datetime.combine(dia, time(17, 30), MAD)
cierre_nyse = datetime.combine(dia, time(16, 0), NY)
print(cierre_bme.astimezone(UTC).strftime("BME   %H:%M UTC"))
print(cierre_nyse.astimezone(UTC).strftime("NYSE  %H:%M UTC"))
print("NYSE closes", cierre_nyse - cierre_bme, "later")
OUT [10]Two closes with the same date
BME   16:30 UTC
NYSE  21:00 UTC
NYSE closes 4:30:00 later

Subtracting aware dates gives the real duration, because Python converts them to UTC before subtracting. Subtracting two naive local times on clock-change days would give an hour too many or too few.

Exercises

  1. Compute at what Madrid time the Tokyo Stock Exchange opens (09:00, Asia/Tokyo) on March 2 and March 30, 2026. Why does it change, if Japan has no daylight saving time?
Solution
IN [11]python
TOK = ZoneInfo("Asia/Tokyo")
for dia in (date(2026, 3, 2), date(2026, 3, 30)):
    ap = datetime.combine(dia, time(9, 0), TOK)
    print(dia, ap.astimezone(MAD).strftime("%H:%M Madrid (%Z)"))
OUT [11]Tokyo from Madrid
2026-03-02 01:00 Madrid (CET)
2026-03-30 02:00 Madrid (CEST)

Japan does not change its clocks; Madrid does. In winter (CET, UTC+1) Tokyo is 8 hours ahead; in summer (CEST, UTC+2), 7. A market's schedule moves as seen from outside even though the market changes nothing.

  1. Read operaciones_sint.csv and write to largas.json only the long SINT trades, with cantidad as an integer, the prices as floats and the dates as ISO text. Check how many there are by reading the JSON.
Solution
IN [12]python
largas = [
    {"entrada": f["fecha_entrada"], "salida": f["fecha_salida"],
     "cantidad": int(f["cantidad"]),
     "precio_entrada": float(f["precio_entrada"]),
     "precio_salida": float(f["precio_salida"])}
    for f in filas if f["simbolo"] == "SINT" and f["lado"] == "largo"
]
Path("largas.json").write_text(json.dumps(largas, indent=2),
                               encoding="utf-8")
datos = json.loads(Path("largas.json").read_text(encoding="utf-8"))
print(len(datos), datos[0])
OUT [12]Filtering a CSV into JSON
24 {'entrada': '2021-09-02', 'salida': '2021-10-12', 'cantidad': 180, 'precio_entrada': 108.64, 'precio_salida': 108.79}

The dates already arrive from the CSV as ISO text, so they are copied as they are. Quantities and prices are converted before writing: the JSON stores numbers, and whoever reads it will not have to convert them.

  1. Write a function en_sesion(instante, zona, abre, cierra) that says whether an aware datetime falls within a market's hours expressed in its own zone. Check 15:00 Madrid time on March 16 and March 30, 2026 against the NYSE.
Solution
IN [13]python
def en_sesion(instante: datetime, zona: ZoneInfo,
              abre: time, cierra: time) -> bool:
    local = instante.astimezone(zona)
    return local.weekday() < 5 and abre <= local.time() < cierra

for dia in (16, 30):
    t = datetime(2026, 3, dia, 15, 0, tzinfo=MAD)
    print(t.date(), en_sesion(t, NY, time(9, 30), time(16, 0)))
OUT [13]Is the market open?
2026-03-16 True
2026-03-30 False

At 15:00 Madrid time on March 16 the NYSE has already been open for half an hour (it is 10:00 in New York); on March 30 it has not opened yet (it is 09:00). The function knows nothing about holidays: a complete market calendar (level 1) adds the list of days without a session.

Footnotes

  1. NYSE, "Holidays & Trading Hours": Core Trading Session: 9:30 a.m. to 4:00 p.m. ET. Link in "Resources". ↩

  2. NIST, "Daylight Saving Time Rules": daylight saving time begins on the second Sunday of March at 2:00 a.m. and ends on the first Sunday of November at 2:00 a.m., local time. Link in "Resources". ↩

  3. Directive 2000/84/EC, transposed into Spanish law by Royal Decree 236/2002: the last Sunday of March and the last Sunday of October, at 1:00 UTC. Link in "Resources". ↩

  4. BME Exchange, "Horario Bolsa Española" (Spanish stock exchange trading hours): open market from 9:00 to 17:30 and closing auction from 17:30 to 17:35 (https://www.bolsasymercados.es/es/bme-exchange/horario-del-mercado.html). ↩

Each answer is folded under its question

Self-assessment

  1. A YAML file contains simbolos: [SPY, ON, NO]. What does yaml.safe_load return?

    • A['SPY', 'ON', 'NO']
    • BA syntax error
    • C['SPY', None, None]
    • D['SPY', True, False]
    Show answer

    Answer D. ['SPY', True, False]

    PyYAML follows YAML 1.1, where on, off, yes and no (in any combination of upper and lower case) are booleans. Symbols that match those words must be put in quotes.

  2. At what Madrid time does the NYSE open on Monday, March 16, 2026?

    • A15:30, as always
    • B14:30, because the United States has already moved its clocks forward and Europe has not yet
    • C16:30
    • D09:30
    Show answer

    Answer B. 14:30, because the United States has already moved its clocks forward and Europe has not yet

    The United States changes its clocks on the second Sunday of March (March 8, 2026) and the EU on the last one (March 29). Between the two dates, the difference is 5 hours instead of 6.

  3. Why is it a good idea to store timestamps in UTC?

    • ABecause it takes up less space
    • BBecause markets open in UTC
    • CBecause UTC has no clock changes: each instant has a single representation and differences between timestamps are always correct
    • DBecause Python does not support other zones
    Show answer

    Answer C. Because UTC has no clock changes: each instant has a single representation and differences between timestamps are always correct

    In local time there are times that do not exist (the spring-forward change) and times that repeat (the fall-back change). In UTC that does not happen: store in UTC and convert to local time only for display.

  4. You compare datetime(2026, 3, 9, 15, 0) (no zone) with an open created with tzinfo=ZoneInfo("America/New_York"). What happens?

    • ATypeError: dates with and without a time zone cannot be compared
    • BPython assumes the first one is in UTC
    • CPython assumes the first one is in the computer's local time
    • DIt always returns False
    Show answer

    Answer A. TypeError: dates with and without a time zone cannot be compared

    A date without a zone (naive) does not represent a specific instant. Python refuses to compare it with one that has a zone (aware) rather than guess.

Key points

  • pathlib.Path builds paths with /, lists files with glob and reads or writes text in one line. Relative paths depend on the working directory.
  • CSV delivers text: every column has to be converted. JSON does not support dates or Decimal: they are serialized as ISO text.
  • YAML is convenient and treacherous: ON and NO are booleans and 1.10 is the number 1.1. TOML requires quotes around text and does not have that problem; tomllib reads it but does not write it.
  • The United States and the EU change their clocks on different dates: in 2026, the NYSE opens at 14:30 Madrid time from March 9 to 27 and from October 26 to 30.
  • Store in UTC, convert for display and never mix dates with and without a zone. The closes of two markets on the same date do not happen at the same time.

Resources

Data