--- title: "6.1.3 Text event studies and topic modeling" description: "Align each news item with its reaction session, measure the error of using the date, CAR by tone, LDA and BERTopic topics and 10-K changes." url: "https://pythonparatrading.com/en/python-trading-course/level-6/text-event-studies-and-topic-modeling/" lang: "en" image: "https://pythonparatrading.com/og/en/python-trading-course/level-6/text-event-studies-and-topic-modeling.png" --- # 6.1.3 · Event studies with text signals, timestamp alignment and topic modeling > A text signal is only worth what it moves the price after you are able to trade it. This lesson turns the publication instant of each news item into its reaction session with the real NYSE calendar, measures how wrong you are when you use the date, studies the abnormal return by tone, groups the news by topic and repeats the method with the changes in the risk factors of 10-Ks. - Course: [Level 6: Generative AI, LLMs, agents, big data and alternative data](https://pythonparatrading.com/en/python-trading-course/level-6/) - Module: 6.1 · Classical and modern financial NLP - Duration: 75 min - Prerequisites: [0.4.4 · Files (CSV, JSON, YAML, TOML) and dates with time zones](https://pythonparatrading.com/en/python-trading-course/level-0/python-files-pathlib-time-zones/), [1.1.2 · Sessions, auctions, holidays and calendars](https://pythonparatrading.com/en/python-trading-course/level-1/trading-sessions-auctions-and-market-calendars/), [2.3.5 · Factors (value, quality, low volatility, size) and event strategies](https://pythonparatrading.com/en/python-trading-course/level-2/factor-investing-and-event-strategies/), [6.1.1 · Tokenization, embeddings, financial dictionaries and FinBERT](https://pythonparatrading.com/en/python-trading-course/level-6/tokenization-embeddings-loughran-mcdonald-finbert/) ## By the end of this lesson - Assign each timestamped document its reaction session (before the open, during the session, after the close, no session, half-day sessions) with `exchange_calendars`. - Measure the bias of date alignment against instant alignment and explain which time slots it comes from. - Build an event study with text signals: CAR by tone terciles, with pre-event controls and without counting the same story twice. - Extract topics with LDA, check them against the event types and use the topic to separate news that moves the price from news that does not. - Measure a document signal (similarity between the risk sections of consecutive 10-Ks) with its drift window. ## An event study with text Lesson 2.3.5 built event studies with dates: earnings announcements, index additions, dividends. The mechanics are the same with text (abnormal return, CAR, windows before and after), with two differences that change the results: - **The event has a time.** A news item at 7:00 and another at 17:40 on the same day do not enter the same close-to-close return. With daily data, the question is in which session the price can react. - **The signal is a noisy score.** The sentiment comes from a dictionary, from FinBERT or from an LLM (lesson 6.1.1), not from an accounting figure. And there are many more events: tens of thousands of news items, with copies and wrap-ups that are not new events. ```mermaid flowchart LR P[Publication] --> S[Reaction session] S --> D[Drift] C[Pre-event control] --> P ``` The pre-event control (sessions −5 to −1) has to come out flat: if the CAR moves before publication, either there is a leak, or the alignment is wrong, or the news item reports something that had already happened. The data are those of level 6: 39,570 synthetic news items with their publication instant to the second and the closes of 120 fictitious stocks over 2,011 NYSE sessions. Most of the news is written in Spanish and some in English; the texts are kept as they are, since the code scores them with the Spanish and English lexicon of lesson 6.1.1. The generator planted a known effect: on the reaction day, each news item moves the price in proportion to its true sentiment and to the materiality of the event, with a decaying drift over the next five sessions. The planted slope is 123 bp per unit of sentiment (level 6 README). The abnormal return is the stock's return minus the mean of the 120 in the panel: ```python import re import exchange_calendars as xcals import numpy as np import pandas as pd noticias = pd.read_parquet("noticias6_corpus.parquet") verdad = pd.read_csv("noticias6_verdad.csv") df = noticias.merge(verdad, on="id_noticia") precios = pd.read_parquet("acciones6_precios.parquet") cierre = precios.pivot(index="fecha", columns="ticker", values="cierre") r = cierre.pct_change() anormal = r.sub(r.mean(axis=1), axis=0).to_numpy() N = len(cierre) columna = {t: j for j, t in enumerate(cierre.columns)} df["j"] = df["ticker_actual"].map(columna) print(cierre.shape, df.shape) print(df["tipo_noticia"].value_counts().to_dict()) ``` ```output (2011, 120) (39570, 22) {'original': 22593, 'sindicada': 12266, 'resultados': 3360, 'mercado': 1209, 'correccion': 142} ``` The news types are original stories, syndicated copies (`sindicada`), earnings (`resultados`), market wrap-ups (`mercado`) and corrections (`correccion`). `ticker_actual` is the ticker in force today, the one used by the price table: six companies changed ticker and their old news carries the old one (lesson 6.1.2). Joining by the news ticker would lose those rows. ## From the instant to the reaction session A closing price summarizes what was known at 16:00 New York time. A news item published at instant $t$ can only enter the returns of the sessions whose close is later than $t$. With the exchange calendar (lesson 1.1.2), the reaction session is the **first session that closes after publication**: | Publication (New York time) | Reaction session | Fraction of the effect that day | |---|---|---| | Before the open (6:00-9:29) | That day's | All | | During the session (9:30-15:59) | That day's | The part of the session that remains; the rest, the next one | | After the close or overnight | The next one | All | | Weekend or holiday | The next one | All | | Half-day session, after 13:00 | The next one | All | `exchange_calendars` gives the opening and closing time of each session in UTC, with the half-day sessions (the day after Thanksgiving, Christmas Eve and some days before July 4 close at 13:00) and the daylight saving time changes included. With that table, the reaction session is a binary search: ```python cal = xcals.get_calendar("XNYS") horario = cal.schedule.loc[cierre.index[0]:cierre.index[-1]] print((horario.index.tz_localize(None) == cierre.index).all()) cierres_us = pd.DatetimeIndex(horario["close"]).as_unit("us").asi8 aperturas_us = pd.DatetimeIndex(horario["open"]).as_unit("us").asi8 hora_cierre = horario["close"].dt.tz_convert("America/New_York") medias = horario[hora_cierre.dt.hour < 16] print(len(medias), "half-day sessions") medias.loc["2019-11-29":"2019-12-24"] ``` ```output True 18 half-day sessions open break_start break_end close 2019-11-29 2019-11-29 14:30:00+00:00 NaT NaT 2019-11-29 18:00:00+00:00 2019-12-24 2019-12-24 14:30:00+00:00 NaT NaT 2019-12-24 18:00:00+00:00 ``` The 2,011 sessions of the calendar match those of the price table. On November 29, 2019 the market closes at 18:00 UTC (13:00 in New York): a news item at 14:00 that day no longer enters its return. ```python def sesion_de(ts): """First session that closes after `ts` and whether `ts` falls inside that session (between its open and its close).""" t = pd.DatetimeIndex(ts).as_unit("us").asi8 k = np.searchsorted(cierres_us, t, side="right") dentro = t > aperturas_us[np.minimum(k, N - 1)] return k, dentro df["k"], df["en_sesion"] = sesion_de(df["ts"]) calculada = cierre.index[np.minimum(df["k"], N - 1)] coincide = (calculada.strftime("%Y-%m-%d") == df["sesion_reaccion"]) con_verdad = df["sesion_reaccion"].notna() & (df["k"] < N) print(f"matches the truth: {coincide[con_verdad].mean():.1%}", f"of {con_verdad.sum()} news items") ny = df["ts"].dt.tz_convert("America/New_York") hora = ny.dt.hour + ny.dt.minute / 60 dia_con_sesion = ny.dt.tz_localize(None).dt.normalize().isin( cierre.index) df["franja"] = np.select( [~dia_con_sesion, hora < 9.5, df["en_sesion"], hora < 20], ["no session", "pre-open", "in session", "after close"], "overnight") orig = df[(df["tipo_noticia"] == "original") & (df["k"] >= 6) & (df["k"] < N - 12)].copy() orig["franja"].value_counts(normalize=True).round(3) ``` ```output matches the truth: 100.0% of 27304 news items franja in session 0.407 pre-open 0.280 after close 0.181 no session 0.088 overnight 0.044 Name: proportion, dtype: float64 ``` The computed session matches the one in the truth file for the 27,304 news items that have it (syndicated copies do not carry their own session in that file). 18% of the original news comes out after the close, 4% overnight and 9% on days without a session (weekends and holidays): almost a third of the corpus has its effect in a session different from that of its date. The news with `k` outside the sample (the first six sessions and the last twelve) is left out so that all event windows are complete. ## The error of aligning by date, measured The reaction day is the reaction session and, if the news came out during the session, also the next one (where the rest of the effect falls). The slope of its abnormal return on the true sentiment measures how much one unit of sentiment moves the price. With the correct alignment and with two naive ones: the New York date of the news item and its UTC date, each with the return of that date: ```python def pendiente_pb(x, y): x, y = np.asarray(x, float), np.asarray(y, float) b, a = np.polyfit(x, y, 1) resid = y - a - b * x se = np.sqrt(resid @ resid / (len(x) - 2) / ((x - x.mean()) ** 2).sum()) return float(round(1e4 * b, 1)), float(round(1e4 * se, 1)) def dia_reaccion(d): k, j = d["k"].to_numpy(), d["j"].to_numpy() siguiente = anormal[np.minimum(k + 1, N - 1), j] return anormal[k, j] + np.where(d["en_sesion"], siguiente, 0) def por_fecha(d, fechas): i = cierre.index.get_indexer(fechas) return np.where(i >= 0, anormal[np.maximum(i, 0), d["j"]], np.nan) orig["y_reaccion"] = dia_reaccion(orig) ny_o = orig["ts"].dt.tz_convert("America/New_York") orig["y_fecha_ny"] = por_fecha(orig, ny_o.dt.tz_localize(None) .dt.normalize()) orig["y_fecha_utc"] = por_fecha(orig, orig["ts"].dt.tz_localize(None) .dt.normalize()) s = orig["sentimiento_verdadero"] filas = {} for c in ["y_reaccion", "y_fecha_ny", "y_fecha_utc"]: ok = orig[c].notna() filas[c] = (int(ok.sum()), *pendiente_pb(s[ok], orig.loc[ok, c])) ok = orig["y_fecha_ny"].notna() mismas = pendiente_pb(s[ok], orig.loc[ok, "y_reaccion"]) filas["y_reaccion, mismas"] = (int(ok.sum()), *mismas) pd.DataFrame(filas, index=["noticias", "pb", "error_tipico"]).T ``` ```output noticias pb error_tipico y_reaccion 22518.0 118.8 3.0 y_fecha_ny 20540.0 61.4 2.7 y_fecha_utc 20341.0 68.5 2.7 y_reaccion, mismas 20540.0 117.4 3.2 ``` With the reaction session, the slope is about 119 bp per unit (column `pb`), close to the 123 planted (the difference is market noise: the standard error, `error_tipico`, is 3 bp). With the New York date it falls by half, to 61 bp, even when compared on the same news (`mismas`; 117 bp with the correct session). The UTC date (68 bp) fixes nothing: at 20:00 in New York it is already the next day in UTC, so it gets the overnight news right and the evening news wrong. Which time slots does the error come from? The two alignments, slot by slot: ```python def por_franja(g): ok = g["y_fecha_ny"].notna() fila = {"noticias": len(g), "correcta": pendiente_pb(g["sentimiento_verdadero"], g["y_reaccion"])[0]} fila["fecha_ny"] = (pendiente_pb(g.loc[ok, "sentimiento_verdadero"], g.loc[ok, "y_fecha_ny"])[0] if ok.sum() > 30 else np.nan) return pd.Series(fila) orig.groupby("franja").apply(por_franja, include_groups=False) ``` ```output noticias correcta fecha_ny franja after close 4067.0 119.2 -10.3 in session 9169.0 115.1 59.0 no session 1978.0 134.0 NaN overnight 1000.0 120.6 12.4 pre-open 6304.0 119.0 119.0 ``` Before the open the two coincide: the date of the news item is the reaction session. After the close, the date gives a negative, meaningless slope: it compares each news item with the return of a day on which it was not yet known. During the session, the date picks up only the part of the effect that arrives before the close, half of it. News from days without a session has no return for its date, and a date alignment loses it (or, worse, assigns it to the previous session with an `ffill`). *Fig. — Reaction-day slope by publication time band (bp per unit of sentiment), with the reaction session and with the New York date: after the close, the date measures noise.* Eight bars in four pairs. Pre-open: 119 bp with both alignments. Intraday: 115 with the session and 59 with the date. After close: 119 with the session and −10 with the date (highlighted). Night: 121 and 12. The header figure summarizes the three alignments. It is a bias toward zero, not upward: whoever aligns by date concludes that text “works less”, discards good signals or underestimates a strategy's capacity. The dangerous (upward) bias appears with documents that **describe** the price. ## Two traps in the corpus: copies and wrap-ups The 1,209 **market wrap-ups** in the corpus are written after the close from the day's move (“X se desploma un 8 %”, X plunges 8%). Their tone summarizes the return that has just happened: ```python cr = df[(df["tipo_noticia"] == "mercado") & (df["k"] < N - 6)].copy() ny_cr = cr["ts"].dt.tz_convert("America/New_York") mismo_dia = por_fecha(cr, ny_cr.dt.tz_localize(None).dt.normalize()) siguiente = anormal[cr["k"], cr["j"]] s_cr = cr["sentimiento_verdadero"] print("with the wrap-up's date: ", round(np.corrcoef(s_cr, mismo_dia)[0, 1], 3)) print("with the reaction session:", round(np.corrcoef(s_cr, siguiente)[0, 1], 3)) ``` ```output with the wrap-up's date: 0.933 with the reaction session: 0.047 ``` With the date, a correlation of 0.93: the perfect “signal”. With the session in which you could trade, nothing. It is reverse causality, and in a real corpus it is common: closing wrap-ups, weekly summaries, analyst notes that explain a move. A filter by news type, or the correct alignment, removes it. The 12,266 **syndicated copies** repeat a story at another agency, from 2 minutes to 6 hours later. Counting them as events: ```python copias = df[df["tipo_noticia"].isin(["original", "sindicada"]) & (df["k"] >= 6) & (df["k"] < N - 12)].copy() copias["y_reaccion"] = dia_reaccion(copias) print("originals only:", len(orig), pendiente_pb(orig["sentimiento_verdadero"], orig["y_reaccion"])) print("with copies: ", len(copias), pendiente_pb(copias["sentimiento_verdadero"], copias["y_reaccion"])) ``` ```output originals only: 22518 (118.8, 3.0) with copies: 34743 (118.4, 2.4) ``` The slope barely moves (118.4 against 118.8: copies inherit the sentiment and almost always the session of the original), but the number of events grows 54% and the standard error falls from 3.0 to 2.4 bp as if there were more information. There is not: the t-statistic is inflated. Worse still in a strategy: three copies of the same news item would be three signals and the position would triple. Group by story before anything else (lesson 6.1.2 teaches how to detect copies without the truth file) and, if two different news items about the same company fall in the same session, aggregate them. ## CAR by dictionary tone With the alignment solved, the text signal is the one from lesson 6.1.1: the tone of the course's financial lexicon, $(p - n)/(p + n + 1)$. The news items with nonzero tone are split into terciles and the abnormal return is accumulated in event time, from session −5 to +10 relative to the reaction session: ```python lex = pd.read_csv("texto6_lexico.csv").query("lista == 'financiera'") listas = {i: (set(g.loc[g["categoria"] == "positiva", "palabra"]), set(g.loc[g["categoria"] == "negativa", "palabra"])) for i, g in lex.groupby("idioma")} def tono(texto, idioma): pos, neg = listas[idioma] w = re.findall(r"[a-záéíóúüñ]+", texto.lower()) p, n = sum(x in pos for x in w), sum(x in neg for x in w) return (p - n) / (p + n + 1) orig["texto"] = orig["titular"] + ". " + orig["cuerpo"] orig["tono"] = [tono(t, i) for t, i in zip(orig["texto"], orig["idioma"])] con_tono = orig[orig["tono"] != 0] tercil = pd.qcut(con_tono["tono"], 3, labels=["negative", "middle", "positive"]) dias = np.arange(-5, 11) k, j = con_tono["k"].to_numpy(), con_tono["j"].to_numpy() ar = pd.DataFrame(anormal[k[:, None] + dias, j[:, None]], columns=dias) ar_t = ar.groupby(tercil.to_numpy(), observed=True).mean() car = 100 * ar_t.cumsum(axis=1) resumen = pd.DataFrame({ "noticias": tercil.value_counts(), "antes_-5_-1": 100 * ar_t.loc[:, -5:-1].sum(axis=1), "dia_0": 100 * ar_t[0], "deriva_1_10": 100 * ar_t.loc[:, 1:10].sum(axis=1), }) resumen.round(2) ``` ```output noticias antes_-5_-1 dia_0 deriva_1_10 middle 3014 0.10 0.20 0.16 negative 6451 0.04 -0.36 -0.19 positive 3913 -0.01 0.44 0.30 ``` *Fig. — Cumulative abnormal return around the reaction session by dictionary-tone tercile: jump on day 0, drift afterwards and a flat pre-event control.* Three CAR curves from session −5 to +10. Before the event all three stay near zero. On day 0 the positive tercile (highlighted) jumps to +0.43% and ends at +0.73%; the middle one reaches +0.46%; the negative one falls to −0.32% and ends at −0.51%. The positive tercile rises 0.44% on the reaction day and a further 0.30% over the next ten sessions (column `deriva_1_10`, drift); the negative one falls 0.36% and a further 0.19%. The middle tercile also rises (0.20% on day 0): the tone takes few values (with a single tone word it is ±0.5) and the ties leave the terciles unequal; the middle one gathers mostly news with tone +0.5, with a mean tone of +0.37 and a mean true sentiment of +0.23. The five previous sessions (`antes_-5_-1`) are the control: between −0.01% and 0.10% in the three terciles, without the pattern of a leak or of a botched alignment. The drift from day +1 onward includes the rest of the effect of the news published during the session (which is completed on day +1) and the planted drift. As in 2.3.5, a mean CAR is not a strategy: it remains to check that the drift survives costs (it is tenths of a point over ten sessions), the capacity of small companies, where the effect is larger (exercise 1), and latency: the day 0 reaction to a 7:00 news item is only captured by buying at the open, and that of a news item during the session, in seconds. ## Topic modeling: what the news is about A **topic model** groups documents by their vocabulary without labels. **LDA** (*latent Dirichlet allocation*, Blei, Ng and Jordan, 2003) assumes that each document is a mixture of topics and each topic a distribution over words, and estimates both from the document-word count matrix. In an event study it serves two purposes: knowing what type each news item is when the vendor does not say, and separating the topics that move the price from the routine ones. With the Spanish news, without stop words or company names (otherwise each company would be a topic), 20 topics and a sample of 8,000 news items to fit (the online mini-batch fit takes a few seconds; then every item is assigned a topic). The stop word list is Spanish because the texts are: ```python from sklearn.decomposition import LatentDirichletAllocation from sklearn.feature_extraction.text import CountVectorizer from sklearn.metrics import normalized_mutual_info_score es = orig[orig["idioma"] == "es"].copy() empresas = pd.read_csv("acciones6_empresas.csv") nombres = {w for x in empresas["nombre"] for w in re.findall(r"[a-záéíóúüñ]+", x.lower())} vacias = set("""de la el en los las del un una que con por se su sus al lo y a es para ha como más este esta sobre tras ante sin según hasta entre dólares millones acciones sesión última cerraron compañía grupo empresa""".split()) | nombres vec = CountVectorizer(stop_words=sorted(vacias), min_df=15, max_df=0.3, token_pattern=r"(?u)\b[^\W\d_]{3,}\b") muestra = es.sample(8000, random_state=0) lda = LatentDirichletAllocation(n_components=20, random_state=0, learning_method="online", batch_size=1024, max_iter=10) lda.fit(vec.fit_transform(muestra["texto"])) es["tema"] = lda.transform(vec.transform(es["texto"])).argmax(axis=1) vocab = vec.get_feature_names_out() print("vocabulary:", len(vocab)) print("normalized mutual information with the event:", round(normalized_mutual_info_score(es["evento"], es["tema"]), 3)) ``` ```output vocabulary: 341 normalized mutual information with the event: 0.825 ``` Normalized mutual information measures, between 0 and 1, how similar two partitions are: here, the topics LDA finds without labels and the 20 event types with which the generator wrote the news. A value of 0.8 is high agreement: LDA recovers almost all the event types. With template texts it is easier than with real news, in which a topic mixes vocabularies and the number of topics is a debatable choice. The most probable words of some topics, with the most frequent true event: ```python def resumen_tema(t): g = es[es["tema"] == t] pal = vocab[np.argsort(lda.components_[t])[::-1][:5]] ev = g["evento"].value_counts() return pd.Series({"noticias": len(g), "palabras": " ".join(pal), "evento": ev.index[0], "pureza": round(ev.iloc[0] / len(g), 2)}) temas = pd.DataFrame({t: resumen_tema(t) for t in range(20)}).T temas.sort_values("noticias", ascending=False).head(8) ``` ```output noticias palabras evento pureza 8 1813 contrato próximos años firmado usd contrato 0.8 15 1606 junta accionistas general dentro celebrará junta 0.72 13 1453 precio objetivo ficticios valores recomendación analista_baja 0.58 4 1418 inversores foro presentará conferencia sector conferencia 1.0 3 1334 plataforma lanza impulso fuentes favorable producto 0.88 5 951 inversión banco sintético precio objetivo analista_baja 0.52 6 927 director nuevo nombra responsable jubila nombramiento 1.0 17 924 mayor pedido reciente adjudicado historia contrato 0.95 ``` The words and the events are Spanish data: topic 8, for example, is contracts (`contrato`: “contrato próximos años firmado”, contract, coming years, signed), topic 15 shareholder meetings (`junta`), topic 13 analyst downgrades (`analista_baja`: “precio objetivo”, price target), topic 4 investor conferences (`conferencia`) and topic 6 appointments (`nombramiento`). The `pureza` column (purity) is the share of the topic's news that belongs to its most frequent event. Do all topics move the price equally? A direct measure, which is that of a strategy that buys on positive tone and sells on negative tone: the mean abnormal return of the reaction day in the direction of the tone, $\overline{y \cdot \operatorname{sign}(\text{tone})}$, in bp per news item. Topic by topic, against the mean materiality that the generator gave its news (0 for shareholder meetings, conferences and appointments; up to 2 for a takeover bid): ```python def por_tema(g): signo = np.sign(g["tono"]) return pd.Series({ "noticias": len(g), "con_tono": (signo != 0).mean(), "pb_en_direccion": 1e4 * (g["y_reaccion"] * signo).mean(), "materialidad": g["materialidad"].mean(), }) pt = es.groupby("tema").apply(por_tema, include_groups=False) pt["evento"] = temas["evento"] corr = pt["pb_en_direccion"].corr(pt["materialidad"]) print("correlation:", round(corr, 2)) pt.sort_values("pb_en_direccion").round(2).iloc[[0, 1, 2, -3, -2, -1]] ``` ```output correlation: 0.86 noticias con_tono pb_en_direccion materialidad evento tema 4 1418.0 0.21 -0.53 0.00 conferencia 15 1606.0 0.21 -0.51 0.00 junta 6 927.0 0.19 2.77 0.00 nombramiento 1 461.0 0.69 53.91 0.80 dimision 0 437.0 0.77 75.06 0.99 retirada_producto 16 661.0 0.89 80.71 1.18 previsiones ``` *Fig. — Twenty LDA topics on the Spanish news: the mean reaction in the direction of tone grows with the materiality the generator gave to their events (correlation 0.86).* Scatter of 20 points. The three zero-materiality topics (meetings, conferences and appointments) sit around 0 bp per item. The others rise with materiality up to the guidance topic (highlighted), with materiality 1.18 and 81 bp, and the product-recall one, with 0.99 and 75 bp. The routine topics (conferences, shareholder meetings, appointments) do not move the price in the direction of their tone (between −0.5 and 2.8 bp per news item) even though one in five of their news items has tone: their text carries tone phrases, but the event is not material. The topics of guidance (`previsiones`), product recalls (`retirada_producto`) or resignations (`dimision`) give between 54 and 81 bp per news item. Across the 20 topics, the correlation with true materiality is 0.86. A text signal improves if it is weighted by topic or if the routine ones are filtered out. With a few hundred news items per topic, each figure has a standard error of several bp: the ranking matters more than each value. ### BERTopic: topics with embeddings LDA works with word counts. **BERTopic** (Grootendorst, 2022) starts from sentence embeddings (those of lesson 6.1.1), reduces their dimension with UMAP, clusters with HDBSCAN and describes each cluster with the most characteristic words of its class (c-TF-IDF)[^1]: ```mermaid flowchart LR E[Embeddings] --> U[UMAP] U --> H[HDBSCAN] H --> C[c-TF-IDF] ``` Unlike LDA, it assigns a single topic per document, decides the number of topics (HDBSCAN leaves noise without a topic) and captures synonyms that share no words. It is installed in the course's `ia` environment, not in this lesson's; the code, with the seeds that fix the result: ```python skip from bertopic import BERTopic from hdbscan import HDBSCAN from sentence_transformers import SentenceTransformer from umap import UMAP textos = es["texto"].tolist() st = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2", device="cpu", local_files_only=True) emb = st.encode(textos, normalize_embeddings=True, show_progress_bar=False) modelo = BERTopic( umap_model=UMAP(n_neighbors=15, n_components=5, random_state=0), hdbscan_model=HDBSCAN(min_cluster_size=50), vectorizer_model=CountVectorizer(stop_words=sorted(vacias)), ) temas_bt, _ = modelo.fit_transform(textos, emb) modelo.get_topic_info().head(10) ``` Run in the `ia` environment on a sample of 2,000 Spanish news items (with the minimal stop words, without removing the company names), the result is instructive: BERTopic with MiniLM clusters mostly by company and sector (the topics are named after the companies: *petroleum*, *steel*, *pharmaceuticals*) and leaves a third of the news without a topic; its mutual information with the event type is 0.10, against 0.82 for LDA. MiniLM is an English model (lesson 6.1.1): with Spanish text it focuses on what it understands, the proper names. For news in Spanish you need a multilingual embedding model, to remove the company names before clustering and to check the topics against a labeled sample, as here against `evento`. ## A document signal: changes in 10-K risks Text does not only arrive in news. Cohen, Malloy and Nguyen (2020) showed that companies that change the language of their annual reports a lot subsequently underperform those that do not change it (a portfolio long the non-changers and short the changers earns, according to the article's abstract, up to 188 bp of alpha per month), and that the market does not react on the filing day: the information is absorbed slowly[^2]. The course's synthetic 10-Ks carry an *Item 1A. Risk Factors* section. In 18% of the 10-Ks from fiscal year 2019 onward, the section changes a lot (impairment risks come in and the sentences become more severe) and the generator plants a −3% drift spread over the next 60 sessions. The signal is the cosine similarity between the bag of words of the section and that of the same company's previous 10-K: ```python from sklearn.preprocessing import normalize docs = pd.read_parquet("filings6_documentos.parquet") k10 = docs[docs["tipo"] == "10-K"].sort_values(["emisor", "ts"]).copy() def riesgos(html): m = re.search(r"

Item 1A\. Risk Factors

(.*?)

", html, re.S) return re.sub(r"<[^>]+>", " ", m.group(1)) if m else "" k10["riesgos"] = k10["texto"].map(riesgos) X = normalize(CountVectorizer().fit_transform(k10["riesgos"])) misma = (k10["emisor"].to_numpy()[1:] == k10["emisor"].to_numpy()[:-1]) sim = np.asarray(X[1:].multiply(X[:-1]).sum(axis=1)).ravel() k10["similitud"] = np.r_[np.nan, np.where(misma, sim, np.nan)] k10 = k10.dropna(subset=["similitud"]) k10["año"] = k10["ts"].dt.year k10.groupby("año")["similitud"].describe()[["count", "50%", "min"]] ``` ```output count 50% min año 2020 60.0 1.000000 0.924000 2021 60.0 0.985860 0.915507 2022 60.0 1.000000 0.946799 2023 60.0 0.999342 0.933138 2024 60.0 1.000000 0.941738 2025 60.0 0.999829 0.952615 ``` The median similarity is almost 1: from one year to the next, most companies copy their risk section with touch-ups. The minimum of each year is the signal. With quintiles by year (so that a change common to all, such as the pandemic risk that enters in 2021, does not take over the quintile), the CAR of the 60 sessions starting at the reaction session. 10-Ks are filed after the close, so that session is the one after the filing: ```python riesgos_v = pd.read_csv("filings6_riesgos_verdad.csv") cols = ["id_doc", "ticker_actual", "cambio_riesgos"] k10 = k10.merge(riesgos_v[cols], on="id_doc") k10["k"], _ = sesion_de(k10["ts"]) k10["j"] = k10["ticker_actual"].map(columna) k10 = k10[k10["k"] + 60 <= N].copy() k10["quintil"] = k10.groupby("año")["similitud"].transform( lambda x: pd.qcut(x, 5, labels=False)) k10["cambia"] = k10["quintil"] == 0 ventana = np.arange(60) ar60 = anormal[k10["k"].to_numpy()[:, None] + ventana, k10["j"].to_numpy()[:, None]] k10["car60"] = 100 * ar60.sum(axis=1) print(pd.crosstab(k10["cambia"], k10["cambio_riesgos"])) global_q = pd.qcut(k10["similitud"], 5, labels=False) == 0 print("changes in the quintile without grouping by year:", int((global_q & k10["cambio_riesgos"]).sum())) k10.groupby("cambia")["car60"].agg(["count", "mean", "sem"]).round(2) ``` ```output cambio_riesgos False True cambia False 276 12 True 18 54 changes in the quintile without grouping by year: 47 count mean sem cambia False 288 0.33 0.87 True 72 -3.88 1.88 ``` *Fig. — Cumulative abnormal return after a 10-K filing: the quintile whose risk section is least similar to last year's drifts down over 60 sessions.* Two CAR curves over 60 sessions. The lowest-similarity quintile (highlighted, 72 reports) falls gradually to −3.9%; the rest (288 reports) hovers near zero and ends at +0.3%. The lowest-similarity quintile of each year (`cambia`, changes) captures 54 of the 66 planted changes (with full-sample quintiles, 47) and its 60-session CAR is −3.88%, against +0.33% for the rest: the method recovers the −3% effect, with a standard error of 1.9 points. With 72 events in the quintile, the interval is wide. The drift is slow, with no jump on the filing day: as in the article, the course's market does not read the changes all at once. ## What changes in real life - **The document's time rules.** News, 8-Ks and filings with the CNMV (the Spanish securities regulator) carry a time; a daily study assigns them to the first session that closes afterward. If there is only a date, assume the worst: the next session. - **One story, one event.** Copies, translations, corrections and wrap-ups are grouped or excluded before measuring; otherwise, the standard errors lie. - **The pre-event control is mandatory.** A CAR that moves before the event reveals a leak, bad alignment or a text that describes the past. - **Materiality varies by topic.** A text signal aggregated without distinguishing topics dilutes the news that matters among the routine news. - **The course effects are planted.** The slopes, CARs and similarities in this lesson are the known answer of a generator. In real data, the size and duration of the effect have to be measured with these same precautions, and the literature gives its own results (Tetlock, 2007, finds that the pessimism of a Wall Street Journal column predicts downward pressure followed by a reversal)[^3]; no figure in this lesson carries over to a real market. ## Exercises 1. The level 6 README says that the effect of news is larger in small companies. Compute the reaction-day slope on the true sentiment in the smallest third of companies and in the largest by first-day capitalization (close on 2018-01-02 times shares outstanding). **Solution.** ```python acciones = empresas.set_index("ticker")["acciones_millones"] capital = cierre.iloc[0] * acciones.reindex(cierre.columns) tercio = pd.qcut(capital.rank(), 3, labels=["small", "mid", "large"]) orig["tamaño"] = orig["ticker_actual"].map(tercio) orig.groupby("tamaño", observed=True).apply( lambda g: pd.Series(pendiente_pb(g["sentimiento_verdadero"], g["y_reaccion"]), index=["pb", "error_tipico"]), include_groups=False) ``` ```output pb error_tipico tamaño small 208.5 8.5 mid 132.0 5.1 large 89.2 4.1 ``` Small companies react more than twice as much per unit of sentiment as large ones: the generator scales the effect with capitalization raised to −0.25. In real data the pattern is similar (less analyst coverage, less liquidity), and so are the costs: a larger drift in small companies does not imply more net profit. 2. Measure the drift of sessions +1 to +5 after the reaction day (without counting it) with the true sentiment and with the dictionary tone. What part of the drift does the dictionary recover? **Solution.** ```python k_d = orig["k"].to_numpy() + orig["en_sesion"].to_numpy() j_d = orig["j"].to_numpy() deriva = anormal[k_d[:, None] + np.arange(1, 6), j_d[:, None]].sum(axis=1) print("truth: ", pendiente_pb(orig["sentimiento_verdadero"], deriva)) print("dictionary: ", pendiente_pb(orig["tono"], deriva)) ``` ```output truth: (34.1, 5.7) dictionary: (19.0, 5.2) ``` With the true sentiment, the slope of the drift is 34 bp (the planted one is 43; the difference is 1.5 standard errors away); with the dictionary, 19 bp, a little more than half, because the tone is a noisy measure of sentiment (correlation of 0.71, lesson 6.1.1) and the noise in the explanatory variable attenuates the slope toward zero. A better measure of sentiment (FinBERT in English) translates directly into more signal. 3. A vendor delivers news with a `fecha` (date) column without a time and another `hora_recepcion` (reception time) column in its server's time zone, without saying which one. How would you check with its own data which alignment is correct? **Solution.** With a control event study. First, find the zone: the distribution of `hora_recepcion` should have peaks at known times (press releases before the open, between 6:00 and 9:30 New York time, and after the close, from 16:00 onward); if the peaks fall at 11:00 and 21:00, the time is in UTC (or London time, depending on the time of year). Second, with news of clear tone (or with earnings releases, whose surprise is known), compute the slope of the return on the tone in the session of the date, in the previous one and in the next one, by time slot: the correct alignment concentrates the reaction in a single session and leaves the pre-event control flat. If the session before the date already reacts, the date is later than the publication (or there is a leak). When in doubt, delay. [^1]: Grootendorst (2022), “BERTopic: Neural topic modeling with a class-based TF-IDF procedure” (); documentation (, accessed on October 23, 2026): “the main steps for topic modeling with BERTopic are sentence-transformers, UMAP, HDBSCAN, and c-TF-IDF run in sequence”. [^2]: Cohen, Malloy and Nguyen (2020), “Lazy Prices”, Journal of Finance 75(3), 1371-1415 (): “a portfolio that shorts "changers" and buys "nonchangers" earns up to 188 basis points per month in alpha (over 22% per year) in the future”, with no announcement effect. [^3]: Tetlock (2007), Journal of Finance 62(3), 1139-1168 (): the pessimism of a Wall Street Journal column predicts downward pressure on prices followed by a reversal. Event study methodology: MacKinlay (1997), Journal of Economic Literature 35(1), 13-39. ## Self-assessment 1. A news item is published at 17:40 New York time on a Thursday that is a regular session. Which close-to-close return does its effect enter? - A. Thursday's, because it is the date of the news item - B. Friday's: Thursday's close had already passed when it was published - C. Wednesday's - D. The following Monday's *Answer B: Friday's: Thursday's close had already passed when it was published.* Thursday's return ends at 16:00. What is published later can only affect the price from the next session onward. In the corpus, the slope of after-close news with the news date is −10 bp per unit of sentiment: pure noise. 2. With the New York date of each news item, the reaction-day slope comes out at 61 bp per unit of sentiment; with the correct reaction session, 117 bp on the same news. Where does almost all the difference come from? - A. From the news published before the open - B. From weekends, which the date does not have - C. From rounding the times - D. From the after-close and overnight news, whose effect is in the next session, and from the news published during the session, whose effect is split between that session and the next one *Answer D: From the after-close and overnight news, whose effect is in the next session, and from the news published during the session, whose effect is split between that session and the next one.* Before the open the two alignments coincide (119 bp). After the close, the date gives −10 bp; overnight, 12; during the session, 59 against 115, because the part of the effect that arrives after the close falls in the next session. 3. Market wrap-ups (“X se desploma un 8 %”, X plunges 8%, published after the close) have a tone that correlates 0.93 with the same day's return. Is it a signal? - A. No: the text describes the move that already happened; with the correct session (the next one) the correlation is 0.05 - B. Yes, the best in the corpus - C. Yes, if you trade at that day's close - D. Only in small companies *Answer A: No: the text describes the move that already happened; with the correct session (the next one) the correlation is 0.05.* It is reverse causality: the price writes the news. With the date as alignment it looks like an almost perfect signal; with the session in which you could trade, it disappears. 4. The study is repeated counting the 12,266 syndicated copies of the news as events too. The slope barely changes. What is the problem? - A. None - B. That the copies have a different sentiment - C. That the number of observations grows 54% without new information: the standard error falls and the t-statistic is inflated - D. That the copies are published before the original *Answer C: That the number of observations grows 54% without new information: the standard error falls and the t-statistic is inflated.* A story told three times is one observation, not three. OLS standard errors assume independent observations; with copies, they overstate the precision. First group by story (lesson 6.1.2). 5. The similarity between the risk section of a 10-K and that of the previous year falls for all companies in the year the pandemic risk enters. How do you form the “changers” quintile properly? - A. With a fixed similarity threshold for the whole sample - B. By filing year: the lowest-similarity quintile of each year, so that a change common to all does not dominate the ranking - C. Only with the large companies - D. With the similarity to the next year's 10-K *Answer B: By filing year: the lowest-similarity quintile of each year, so that a change common to all does not dominate the ranking.* What predicts the price is changing more than the others. With quintiles by year, the bottom quintile captures 54 of the 66 planted changes (against 47 with full-sample quintiles) and its 60-session CAR is −3.9% against +0.3% for the rest. Similarity to the next year would be future information. ## Key points - The unit of an event study with text is the publication instant, not the date: before the open, that day's session; during the session, that session and what remains for the next one; after the close, overnight or on a day without a session, the next session. Half-day sessions close at 13:00. - Aligning by New York date reduces the reaction-day slope from 117 to 61 bp per unit of sentiment; by UTC date, to 68. It is not an upward bias: it hides the signal, and with market wrap-ups it inverts it into reverse causality. - A story is counted once: syndicated copies do not change the slope, but they inflate the number of observations and the t-statistic. - With the dictionary tone, the CAR by terciles separates reaction and drift: the positive tercile rises 0.4% on day 0 and keeps rising; the negative one falls. The control of the five previous sessions must be flat. - LDA recovers the event types without labels (normalized mutual information of 0.82) and the topic separates material news from routine news; BERTopic does the same with embeddings in the `ia` environment. - The similarity between the risk sections of consecutive 10-Ks, ranked by year, flags the planted changes and a negative drift over the next 60 sessions; all the effects are synthetic and planted. ## Resources ### Documentation - [MacKinlay (1997): Event Studies in Economics and Finance, Journal of Economic Literature 35(1)](https://ideas.repec.org/a/aea/jeclit/v35y1997i1p13-39.html) · ideas.repec.org - [Tetlock (2007): Giving Content to Investor Sentiment, Journal of Finance 62(3)](https://doi.org/10.1111/j.1540-6261.2007.01232.x) · doi.org - [Cohen, Malloy and Nguyen (2020): Lazy Prices, Journal of Finance 75(3)](https://doi.org/10.1111/jofi.12885) · doi.org - [Blei, Ng and Jordan (2003): Latent Dirichlet Allocation, JMLR 3](https://www.jmlr.org/papers/v3/blei03a.html) · jmlr.org - [scikit-learn: LatentDirichletAllocation](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.LatentDirichletAllocation.html) · scikit-learn.org - [BERTopic: documentation (embeddings, UMAP, HDBSCAN and c-TF-IDF)](https://maartengr.github.io/BERTopic/) · maartengr.github.io - [exchange_calendars: exchange calendars with opening and closing times](https://github.com/gerrymanoim/exchange_calendars) · github.com Previous lesson: [6.1.2 · Text sources: EDGAR, CNMV, transcripts and social media](https://pythonparatrading.com/en/python-trading-course/level-6/text-sources-edgar-cnmv-transcripts-social-media/) Next lesson: [6.2.1 · How an LLM works: from the next token to the API](https://pythonparatrading.com/en/python-trading-course/level-6/how-llms-work-apis-and-local-models/)