Lesson 0.4.4 · 50 min
Files (CSV, JSON, YAML, TOML) and dates with time zones
A trading system's data lives in files, and its times live in time zones. This lesson covers the four usual text formats with pathlib and the zoneinfo module, with which the New York open stops being "at 15:30" and becomes what it is: a local time that moves.
- Module
- 0.4 · Intermediate Python
- Duration
- 50 min
- Author
- Jesús Cuesta
- Updated
Level 0 lessons
0.1 · Welcome and expectations
0.2 · Modern working environment
- 0.2.1 Installing Python with uv: versions and support cycle
- 0.2.2 Virtual environments and dependencies: uv init, uv add, pyproject.toml and lockfiles
- 0.2.3 Editors (VS Code, Cursor, PyCharm) and notebooks (Jupyter, JupyterLab, marimo)
- 0.2.4 Basic terminal, paths, environment variables and .env files
- 0.2.5 Git and GitHub: commits, branches, pull requests and .gitignore
- 0.2.6 Introduction to AI coding assistants: prompts, review and tests
0.3 · Language fundamentals
- 0.3.1 Variables, types and operators; floating point and decimal for prices and ticks
- 0.3.2 Strings, f-strings and formatting numbers and dates
- 0.3.3 Lists, tuples, dictionaries, sets and comprehensions
- 0.3.4 Control flow: if, elif, else, for, while and match-case
- 0.3.5 Functions: arguments, *args and **kwargs, lambdas and scope
- 0.3.6 Exceptions; modules and packages, with Python 3.15 lazy imports
0.4 · Intermediate Python
- 0.4.1 Object-oriented programming: classes, inheritance, composition and dataclasses
- 0.4.2 Typing: type hints, mypy or ty, and Pydantic for configurations
- 0.4.3 Iterators, generators, decorators and context managers
- 0.4.4 Files (CSV, JSON, YAML, TOML) and dates with time zones
- 0.4.5 Professional logging and testing with pytest
- 0.4.6 Concurrency: threading, multiprocessing, asyncio and free-threading
0.5 · Numerical computing and data
- 0.5.1 NumPy 2: arrays, broadcasting, vectorization, linear algebra and random generators
- 0.5.2 pandas 3.0: Series, DataFrame, time indexes, Copy-on-Write, the str type and pd.col()
- 0.5.3 Selection, filtering, groupby, merge, pivot, resample and moving windows
- 0.5.4 Polars: lazy versus eager API, expressions, performance and when to pick it over pandas
- 0.5.5 DuckDB and essential SQL: SELECT, JOIN, GROUP BY and window functions on Parquet
0.6 · Visualisation
0.7 · Minimum mathematics and statistics
Paths with pathlib
A trading system reads prices, configurations and trades, and writes reports, logs and backtest results. Before opening
a file you need to know where it is, and paths written as text ("datos/" + simbolo + ".csv") break easily: too many
or too few slashes, different separators on Windows and Linux, duplicated extensions.
pathlib.Path represents a path as an object. The / operator joins parts, and its attributes give the name, the
extension or the folder without manipulating text:
from pathlib import Path
for p in sorted(Path(".").glob("*.csv")):
print(f"{p.name:<22} {p.stem:<16} {p.stat().st_size:>8,} B")
informes = Path("informes") / "2025" / "sint"
informes.mkdir(parents=True, exist_ok=True)
destino = informes / "resumen.csv"
print(destino, destino.suffix, destino.parent.exists())cartera_sint.csv cartera_sint 49,207 B
operaciones_sint.csv operaciones_sint 2,440 B
sint_diario.csv sint_diario 60,757 B
informes/2025/sint/resumen.csv .csv Trueglob("*.csv") lists the files that match the pattern (rglob also searches subfolders). mkdir with parents=True
creates the intermediate folders, and with exist_ok=True it does not fail if they already exist: a reporting script
can run every day without checking anything first.
Relative paths, such as Path(".") or "sint_diario.csv", are resolved from the working directory, the place the
program was launched from (lesson 0.2.4). A script that works in the terminal and fails in a task scheduler almost
always has that problem. The usual solution is to build paths from the location of the script itself:
Path(__file__).parent / "datos".
CSV: everything arrives as text
CSV is the most common format for prices and trades: one row per record, columns separated by commas and a header. The
standard library's csv module takes care of quotes and separators; DictReader delivers each row as a dictionary
with the header's keys. What it does not do is convert types: everything arrives as text.
import csv
with open("operaciones_sint.csv", newline="", encoding="utf-8") as f:
filas = list(csv.DictReader(f))
print(len(filas), list(filas[0]))
print(filas[0]["cantidad"], type(filas[0]["cantidad"]).__name__)
def pnl(fila: dict[str, str]) -> float:
signo = 1 if fila["lado"] == "largo" else -1
bruto = signo * int(fila["cantidad"]) * (
float(fila["precio_salida"]) - float(fila["precio_entrada"]))
return bruto - float(fila["comision"])
por_simbolo: dict[str, float] = {}
for fila in filas:
s = fila["simbolo"]
por_simbolo[s] = por_simbolo.get(s, 0.0) + pnl(fila)
for s, total in sorted(por_simbolo.items()):
print(f"{s:<5} {total:>10,.2f}")40 ['id', 'fecha_entrada', 'fecha_salida', 'simbolo', 'lado', 'cantidad', 'precio_entrada', 'precio_salida', 'comision']
200 str
ORO -3,237.96
RVAR 17.09
SINT 4,405.08
TECH -35.02"200" is text: adding to it would raise an error and multiplying it would repeat the string (lesson 0.4.2). That is
why every numeric field goes through int() or float() before any arithmetic. pandas (lesson 0.5.2) infers the
types on its own, and even so it is worth checking them.
Writing is symmetrical. csv.writer takes rows as lists and handles separators and quotes. The newline="" argument
to open is not decorative: without it, on Windows each row would end with an extra line break.
with destino.open("w", newline="", encoding="utf-8") as f:
w = csv.writer(f)
w.writerow(["simbolo", "pnl"])
for s, total in sorted(por_simbolo.items()):
w.writerow([s, round(total, 2)])
print(destino.read_text(encoding="utf-8"))simbolo,pnl
ORO,-3237.96
RVAR,17.09
SINT,4405.08
TECH,-35.02A typical trap in Spain, and in many other European locales: CSV files exported from a spreadsheet set to Spanish use
; as the separator and a decimal comma (142,22). float("142,22") raises ValueError. It is read with
csv.reader(f, delimiter=";") and the comma is replaced by a period before converting, or with
pd.read_csv(sep=";", decimal=","). What you must never do is remove the comma: 14222 is a different price.
JSON: no dates or decimals
JSON is the format of APIs: almost all brokers and data providers respond in JSON. It supports objects (dictionaries),
lists, text, numbers, true, false and null. Nothing else: no dates, no Decimal, no sets.
import json
from datetime import date
resumen = {"estrategia": "cruce-50-200", "fecha": date(2025, 12, 31),
"operaciones": len(filas),
"pnl": {s: round(v, 2) for s, v in por_simbolo.items()}}
try:
json.dumps(resumen)
except TypeError as e:
print("TypeError:", e)
resumen["fecha"] = resumen["fecha"].isoformat()
texto = json.dumps(resumen, ensure_ascii=False, indent=2)
Path("resumen.json").write_text(texto, encoding="utf-8")
leido = json.loads(Path("resumen.json").read_text(encoding="utf-8"))
print(leido["fecha"], type(leido["fecha"]).__name__)
print(leido == resumen)TypeError: Object of type date is not JSON serializable
2025-12-31 str
TrueThe date is saved as ISO 8601 text (2025-12-31) and comes back as text: whoever reads the JSON has to know that the
field is a date and convert it with date.fromisoformat. That is the work Pydantic does (lesson 0.4.2) when it
validates an API response: the model says the field is a date and Pydantic converts it.
Two more details. ensure_ascii=False keeps accented characters as they are instead of escapes such as \u00f3. And
json.dumps writes NaN for a not-a-number float, which is not standard JSON: many readers, browsers included, reject
it. With allow_nan=False the error is raised when writing, not when reading.
YAML and TOML: configurations
Configurations are written by hand, and JSON is awkward for that: it does not allow comments and requires quotes
around everything. The two usual formats are YAML and TOML. YAML is read with PyYAML (lesson 0.4.2); TOML, with
tomllib, part of the standard library since Python 3.11.
YAML guesses types from the text, and sometimes it guesses wrong. PyYAML follows version 1.1 of the standard, where
yes, no, on and off are booleans. There are companies listed under those symbols:
import yaml
Path("config.yaml").write_text("""\
simbolos: [SPY, ON, NO, "ON"]
inicio: 2021-01-04
version_modelo: 1.10
""", encoding="utf-8")
print(yaml.safe_load(Path("config.yaml").read_text(encoding="utf-8"))){'simbolos': ['SPY', True, False, 'ON'], 'inicio': datetime.date(2021, 1, 4), 'version_modelo': 1.1}ON (onsemi, on the Nasdaq) has become True and NO False; only the quoted "ON" is still text. The unquoted date
has become a date, and version 1.10 the number 1.1. In a list of symbols the error raises no exception: the
strategy simply does not trade those assets, or fails much later when requesting the price of True. That is why
configurations are validated with a model (lesson 0.4.2) that requires list[str].
TOML requires quotes around all text, so it does not have that problem. It is the format of pyproject.toml
(lesson 0.2.2). tomllib.load needs the file opened in binary mode ("rb"):
import tomllib
Path("estrategia.toml").write_text("""\
[estrategia]
nombre = "cruce"
simbolos = ["SPY", "ON", "NO"]
inicio = 2021-01-04
[costes]
comision_pb = 1.5
deslizamiento_pb = 3.0
""", encoding="utf-8")
with open("estrategia.toml", "rb") as f:
cfg = tomllib.load(f)
print(cfg["estrategia"])
print(sum(cfg["costes"].values())){'nombre': 'cruce', 'simbolos': ['SPY', 'ON', 'NO'], 'inicio': datetime.date(2021, 1, 4)}
4.5The symbols are still text and the date is still a date. tomllib only reads: writing TOML requires an external
package (tomli-w). That fits how it is used: a configuration is written by hand and the program reads it.
| Format | Reading in Python | Comments | Types | Typical use |
|---|---|---|---|---|
| CSV | csv (stdlib), pandas | No | All text | Prices, trades |
| JSON | json (stdlib) | No | Basic, no dates | APIs, results |
| YAML | PyYAML (safe_load) | Yes | Guesses them (careful) | Configurations |
| TOML | tomllib (stdlib, read-only) | Yes | Explicit, with dates | pyproject.toml, configurations |
Dates with time zones: zoneinfo
A datetime without a time zone (naive) is a clock time without a place: "March 9, 2026, 09:30" does not say whether
it is in New York, Madrid or Tokyo. One with a zone (aware) does represent a specific instant. zoneinfo, part of the
standard library since Python 3.9, provides the zones of the IANA database, with their full history of clock changes,
by name: America/New_York, Europe/Madrid, Asia/Tokyo.
The NYSE core session runs from 09:30 to 16:00, New York time.1Note 1 · NYSE, "Holidays & Trading Hours": Core Trading Session: 9:30 a.m. to 4:00 p.m. ET. Link in "Resources". Converting the open to other zones is a call to
astimezone:
from datetime import datetime, time, timedelta
from zoneinfo import ZoneInfo
NY = ZoneInfo("America/New_York")
MAD = ZoneInfo("Europe/Madrid")
UTC = ZoneInfo("UTC")
for dia in (date(2026, 3, 2), date(2026, 3, 9), date(2026, 3, 30)):
apertura = datetime.combine(dia, time(9, 30), NY)
print(apertura.isoformat(), "→",
apertura.astimezone(MAD).strftime("%H:%M Madrid"), "|",
apertura.astimezone(UTC).strftime("%H:%M UTC"))2026-03-02T09:30:00-05:00 → 15:30 Madrid | 14:30 UTC
2026-03-09T09:30:00-04:00 → 14:30 Madrid | 13:30 UTC
2026-03-30T09:30:00-04:00 → 15:30 Madrid | 13:30 UTCThe same open, at the same New York time, falls at 15:30 Madrid time on March 2, at 14:30 on the 9th and again at 15:30 on the 30th. The shift comes from the clock-change rules: the United States moves its clocks forward on the second Sunday of March and back on the first Sunday of November;2Note 2 · NIST, "Daylight Saving Time Rules": daylight saving time begins on the second Sunday of March at 2:00 a.m. and ends on the first Sunday of November at 2:00 a.m., local time. Link in "Resources". the European Union does it on the last Sunday of March and the last Sunday of October.3Note 3 · Directive 2000/84/EC, transposed into Spanish law by Royal Decree 236/2002: the last Sunday of March and the last Sunday of October, at 1:00 UTC. Link in "Resources". Between those dates, the difference between the two cities is 5 hours instead of 6. In 2026, on business days:
def diferencia(dia: date) -> timedelta:
"""Hours between Madrid and New York at the NYSE open."""
ap = datetime.combine(dia, time(9, 30), NY)
return ap.astimezone(MAD).utcoffset() - ap.utcoffset()
dias = [date(2026, 1, 1) + timedelta(days=i) for i in range(365)]
habiles = [d for d in dias if d.weekday() < 5]
raros = [d for d in habiles if diferencia(d) != timedelta(hours=6)]
print(len(raros), "business days with", diferencia(raros[0]))
tramos = [[raros[0]]]
for ayer, hoy in zip(raros, raros[1:]):
if (hoy - ayer).days > 3:
tramos.append([])
tramos[-1].append(hoy)
for t in tramos:
print(f" from {t[0]} to {t[-1]}: {len(t)} days")20 business days with 5:00:00
from 2026-03-09 to 2026-03-27: 15 days
from 2026-10-26 to 2026-10-30: 5 daysTwenty business days in 2026 on which New York opens an hour earlier for someone trading from Madrid: from March 9 to 27 and from October 26 to 30. The header figure plots every Monday of 2026 with the Madrid time of the open; the four highlighted weeks are the shifted ones.
The consequences are practical. A program that sends orders "at 15:25 Madrid time, five minutes before the open" arrives an hour late in those weeks. A filter that discards "the first half hour of New York" using fixed Madrid times is applied at the wrong time. And a backtest with intraday bars in Madrid local time mixes two session schedules without anyone noticing. The solution is always the same: express each market's hours in its own zone and convert.
Times that do not exist and times that repeat
The clock change creates two anomalies in local time. In Madrid, on March 29, 2026 the clock jumps from 02:00 to 03:00:
02:30 on that day does not exist. On October 25, it goes from 03:00 back to 02:00: 02:30 happens twice.
zoneinfo raises no errors; the fold attribute (0 or 1) chooses between the two occurrences of a repeated time:
inexistente = datetime(2026, 3, 29, 2, 30, tzinfo=MAD)
print(inexistente.astimezone(UTC).astimezone(MAD))
for fold in (0, 1):
repetida = datetime(2026, 10, 25, 2, 30, tzinfo=MAD, fold=fold)
print(fold, repetida.astimezone(UTC).strftime("%H:%M UTC"),
repetida.tzname())
try:
print(datetime(2026, 3, 9, 15, 0) < datetime.now(UTC))
except TypeError as e:
print("TypeError:", e)2026-03-29 03:30:00+02:00
0 00:30 UTC CEST
1 01:30 UTC CET
TypeError: can't compare offset-naive and offset-aware datetimesThe nonexistent time, after a round trip through UTC, becomes 03:30 summer time. The repeated one has two different instants, one hour apart: 00:30 and 01:30 UTC. And Python refuses to compare a date without a zone with one that has a zone, rather than assume anything.
Markets almost never trade at those hours, but data does get generated: cryptocurrency files, which trade 24 hours a day, or a server's logs in local time, have a duplicated hour every fall and a gap every spring. The rules that avoid almost all problems:
- Store in UTC. Timestamps of data, orders and logs go in UTC, which has no clock changes.
- Convert for display. Local time is for people: reports, charts, alerts.
- Define market hours in their own zone. "09:30
America/New_York", not "15:30 Madrid time". - Never mix naive and aware. If a file comes with dates without a zone, decide when reading it which zone they are in and convert them.
The gap between closes is computed as before, each one in its own zone:
dia = date(2025, 12, 31)
cierre_bme = datetime.combine(dia, time(17, 30), MAD)
cierre_nyse = datetime.combine(dia, time(16, 0), NY)
print(cierre_bme.astimezone(UTC).strftime("BME %H:%M UTC"))
print(cierre_nyse.astimezone(UTC).strftime("NYSE %H:%M UTC"))
print("NYSE closes", cierre_nyse - cierre_bme, "later")BME 16:30 UTC
NYSE 21:00 UTC
NYSE closes 4:30:00 laterSubtracting aware dates gives the real duration, because Python converts them to UTC before subtracting. Subtracting two naive local times on clock-change days would give an hour too many or too few.
Exercises
- Compute at what Madrid time the Tokyo Stock Exchange opens (09:00,
Asia/Tokyo) on March 2 and March 30, 2026. Why does it change, if Japan has no daylight saving time?
Solution
TOK = ZoneInfo("Asia/Tokyo")
for dia in (date(2026, 3, 2), date(2026, 3, 30)):
ap = datetime.combine(dia, time(9, 0), TOK)
print(dia, ap.astimezone(MAD).strftime("%H:%M Madrid (%Z)"))2026-03-02 01:00 Madrid (CET)
2026-03-30 02:00 Madrid (CEST)Japan does not change its clocks; Madrid does. In winter (CET, UTC+1) Tokyo is 8 hours ahead; in summer (CEST, UTC+2), 7. A market's schedule moves as seen from outside even though the market changes nothing.
- Read
operaciones_sint.csvand write tolargas.jsononly the long SINT trades, withcantidadas an integer, the prices as floats and the dates as ISO text. Check how many there are by reading the JSON.
Solution
largas = [
{"entrada": f["fecha_entrada"], "salida": f["fecha_salida"],
"cantidad": int(f["cantidad"]),
"precio_entrada": float(f["precio_entrada"]),
"precio_salida": float(f["precio_salida"])}
for f in filas if f["simbolo"] == "SINT" and f["lado"] == "largo"
]
Path("largas.json").write_text(json.dumps(largas, indent=2),
encoding="utf-8")
datos = json.loads(Path("largas.json").read_text(encoding="utf-8"))
print(len(datos), datos[0])24 {'entrada': '2021-09-02', 'salida': '2021-10-12', 'cantidad': 180, 'precio_entrada': 108.64, 'precio_salida': 108.79}The dates already arrive from the CSV as ISO text, so they are copied as they are. Quantities and prices are converted before writing: the JSON stores numbers, and whoever reads it will not have to convert them.
- Write a function
en_sesion(instante, zona, abre, cierra)that says whether an awaredatetimefalls within a market's hours expressed in its own zone. Check 15:00 Madrid time on March 16 and March 30, 2026 against the NYSE.
Solution
def en_sesion(instante: datetime, zona: ZoneInfo,
abre: time, cierra: time) -> bool:
local = instante.astimezone(zona)
return local.weekday() < 5 and abre <= local.time() < cierra
for dia in (16, 30):
t = datetime(2026, 3, dia, 15, 0, tzinfo=MAD)
print(t.date(), en_sesion(t, NY, time(9, 30), time(16, 0)))2026-03-16 True
2026-03-30 FalseAt 15:00 Madrid time on March 16 the NYSE has already been open for half an hour (it is 10:00 in New York); on March 30 it has not opened yet (it is 09:00). The function knows nothing about holidays: a complete market calendar (level 1) adds the list of days without a session.
Footnotes
-
NYSE, "Holidays & Trading Hours": Core Trading Session: 9:30 a.m. to 4:00 p.m. ET. Link in "Resources". ↩
-
NIST, "Daylight Saving Time Rules": daylight saving time begins on the second Sunday of March at 2:00 a.m. and ends on the first Sunday of November at 2:00 a.m., local time. Link in "Resources". ↩
-
Directive 2000/84/EC, transposed into Spanish law by Royal Decree 236/2002: the last Sunday of March and the last Sunday of October, at 1:00 UTC. Link in "Resources". ↩
-
BME Exchange, "Horario Bolsa Española" (Spanish stock exchange trading hours): open market from 9:00 to 17:30 and closing auction from 17:30 to 17:35 (https://www.bolsasymercados.es/es/bme-exchange/horario-del-mercado.html). ↩
Each answer is folded under its question
Self-assessment
A YAML file contains
simbolos: [SPY, ON, NO]. What doesyaml.safe_loadreturn?- A
['SPY', 'ON', 'NO'] - BA syntax error
- C
['SPY', None, None] - D
['SPY', True, False]
Show answer
Answer D.
['SPY', True, False]PyYAML follows YAML 1.1, where
on,off,yesandno(in any combination of upper and lower case) are booleans. Symbols that match those words must be put in quotes.- A
At what Madrid time does the NYSE open on Monday, March 16, 2026?
- A15:30, as always
- B14:30, because the United States has already moved its clocks forward and Europe has not yet
- C16:30
- D09:30
Show answer
Answer B. 14:30, because the United States has already moved its clocks forward and Europe has not yet
The United States changes its clocks on the second Sunday of March (March 8, 2026) and the EU on the last one (March 29). Between the two dates, the difference is 5 hours instead of 6.
Why is it a good idea to store timestamps in UTC?
- ABecause it takes up less space
- BBecause markets open in UTC
- CBecause UTC has no clock changes: each instant has a single representation and differences between timestamps are always correct
- DBecause Python does not support other zones
Show answer
Answer C. Because UTC has no clock changes: each instant has a single representation and differences between timestamps are always correct
In local time there are times that do not exist (the spring-forward change) and times that repeat (the fall-back change). In UTC that does not happen: store in UTC and convert to local time only for display.
You compare
datetime(2026, 3, 9, 15, 0)(no zone) with an open created withtzinfo=ZoneInfo("America/New_York"). What happens?- A
TypeError: dates with and without a time zone cannot be compared - BPython assumes the first one is in UTC
- CPython assumes the first one is in the computer's local time
- DIt always returns
False
Show answer
Answer A.
TypeError: dates with and without a time zone cannot be comparedA date without a zone (naive) does not represent a specific instant. Python refuses to compare it with one that has a zone (aware) rather than guess.
- A
Key points
pathlib.Pathbuilds paths with/, lists files withgloband reads or writes text in one line. Relative paths depend on the working directory.- CSV delivers text: every column has to be converted. JSON does not support dates or
Decimal: they are serialized as ISO text. - YAML is convenient and treacherous:
ONandNOare booleans and1.10is the number 1.1. TOML requires quotes around text and does not have that problem;tomllibreads it but does not write it. - The United States and the EU change their clocks on different dates: in 2026, the NYSE opens at 14:30 Madrid time from March 9 to 27 and from October 26 to 30.
- Store in UTC, convert for display and never mix dates with and without a zone. The closes of two markets on the same date do not happen at the same time.