US Equities Panel

Large-scale cross-sectional prediction across 3,200 stocks with 16 walk-forward folds

Equities Daily Price Data
ETF Cross-Asset Exposures Crypto Perpetuals Funding NASDAQ-100 Microstructure S&P 500 Equity + Option Analytics US Firm Characteristics FX Spot Pairs CME Futures S&P 500 Options (Straddles)
Methodology Highlight
Demonstrates the Fundamental Law of Active Management — how breadth across thousands of stocks compensates for weak per-stock signals — with the most granular walk-forward evaluation (16 folds) in the book.

This is the broadest cross-sectional equity workflow in the book, using daily data for approximately 3,200 US stocks spanning 2000-2018. The case study tests whether individually weak per-stock signals become useful when scaled across thousands of names — the Fundamental Law of Active Management in practice.
With 16 walk-forward folds, this provides the most granular temporal evaluation of any case study. Students learn to work with large panel datasets, build 72 cross-sectional features (momentum, volatility, liquidity, value), and assess how strategy performance varies across different market regimes over nearly two decades.
The case study is the natural home for large-scale latent factor models. With sufficient cross-sectional breadth (N > 3,000), PCA and IPCA can extract meaningful statistical factors. Students learn the practical tradeoffs of daily vs weekly rebalancing, era-dependent cost modeling (pre/post-decimalization), and the challenges of short-selling costs in broad equity universes.

Strategy Summary

Daily long-short decile strategy across approximately 3,200 US stocks. Equal-weight within deciles, dollar-neutral. 72 cross-sectional factors spanning momentum, volatility, liquidity, and value. 16 walk-forward folds spanning 2000-2018. Cost model is era-dependent to reflect changing market microstructure over the evaluation period.

Data Sources

NASDAQ Data Link (equity data)

ML Techniques

Panel data prediction PCA and IPCA (latent factors) Cross-sectional feature engineering Era-dependent cost modeling

ML Pipeline

Universe & Setup
1 notebook
~3,200 US stocks from NASDAQ Data Link spanning 2000-2018. The broadest universe and most CV folds (16, with 10Y train / 1Y val) in the book. Daily long-short protocol with survivorship handling (include delisted until delist date). Liquidity filter: ADV >$1M, price >$5. Era-dependent costs: 15-30 bps pre-decimalization (before 2001), 5-15 bps post-decimalization, plus ~50 bps/yr borrow for shorts.
Universe & Protocol Setup Ch 6
Defines daily long-short protocol for a broad US stock universe with survivorship handling (include delisted until delist date, point-in-time membership), liquidity filters (ADV > $1M, price > $5), and capacity constraints. Builds walk-forward CV splits -- the most folds of any case study.
Labels & Evaluation
2 notebooks
Multi-horizon labels: 1-day (primary), 5-day, and 21-day forward returns for cross-sectional prediction. The 21-day horizon produces the strongest latent factor results (IPCA IC +0.074). 72 features (63 financial + 9 temporal) evaluated with the most statistically robust IC estimates (16 folds).
Label Engineering Ch 7
Computes multi-horizon forward return labels (1-day primary, 5-day, 21-day) with point-in-time eligibility filtering (price > $5, 21-day ADV > $1M applied at each decision date). Preserves delisted stocks until their last trading day. The multi-horizon labels enable comparison between daily, weekly, and monthly rebalancing frequencies.
Feature Evaluation Ch 7
Evaluates all features (financial + temporal) using HAC-adjusted IC across the full panel. Applies Benjamini-Hochberg FDR to control false discovery. Assesses fold-level sign consistency across walk-forward folds and triages features into PROCEED/REVISE/STOP for downstream modeling.
Feature Engineering
2 notebooks
63 cross-sectional factors across 6 families: momentum/trend (multi-horizon returns, skip-month, risk-adjusted), volatility (realized, Garman-Klass, Amihud illiquidity), volume dynamics, value proxies, mean-reversion, and cross-sectional rank features. 9 temporal features: Wasserstein Regime Distance (detects shifts by clustering windowed cross-sections), walk-forward ARIMA, and HMM on daily returns.
Feature Engineering Ch 8
Constructs cross-sectional factors across multiple families: momentum/trend (multi-horizon returns, skip-month, risk-adjusted Sharpe), mean reversion (short-horizon reversals), volatility (NATR, Yang-Zhang), liquidity (Amihud illiquidity, volume ratios), value proxies, and technical indicators (RSI, MACD, ADX, CCI, stochastic). The highest-breadth feature set in the book.
Temporal Features Ch 9
Fits three temporal models walk-forward: Wasserstein Regime Distance (detects regime shifts by clustering windowed cross-sectional median return sequences), Fractional Differencing (FFD with d=0.4 preserves long memory while achieving stationarity), and GARCH conditional volatility on the most liquid stocks. Produces temporal features.
Modeling
10 notebooks
GBM dominates with 16/16 positive folds -- the strongest consistency evidence. IPCA achieves IC +0.074 at 21 days (strongest latent factor result in book) because 3,200-stock breadth supports meaningful factor extraction. Three DL architectures (NLinear, LSTM, TSMixer) tested. TabM was OOM-killed on this universe size. PCA extracts robust statistical factors from the large cross-section. Causal DML tests whether 12-1 momentum causes daily returns.
Linear Models Ch 11
Trains Ridge, LASSO, and ElasticNet on all features across the broad stock universe. Establishes the linear baseline for a Fundamental Law test case: whether modest per-stock IC combines with large breadth to produce a tradable information ratio across walk-forward folds.
GBM Grid Search Ch 12
Trains LightGBM across leaf profiles and loss functions on all features across the broad stock universe. Compares tree-based non-linearity against the linear baseline on daily returns, where the cross-sectional relationship appears largely linear. Evaluates loss function sensitivity (MSE vs MAE vs Huber) on noisy daily returns.
Tabular DL (TabM) Ch 12
Trains TabM attention-style MLP ensemble (small/medium/large) on the same feature matrix across the broad stock universe. Compares neural network expressiveness against tree-based methods on the broadest panel in the book. Note: TabM was OOM-killed during training, leaving incomplete results.
NLinear Ch 13
Trains NLinear (last-value normalization plus a single linear map) as the minimal temporal deep learning baseline on all features across the broad stock universe. Establishes whether the simplest temporal model extracts a daily edge that becomes economically relevant through breadth across thousands of names.
LSTM Ch 13
Trains LSTM on 60-day lookback windows of all features across the broad stock universe, predicting daily returns. Evaluates whether sequential memory captures per- stock temporal dynamics that flat-feature models (linear, GBM) miss, or whether the daily signal is purely cross-sectional.
TSMixer Ch 13
Trains TSMixer alternating time-mixing (across 60-day lookback) and feature-mixing on the broad stock universe. Evaluates whether cross-feature mixing layers find exploitable structure across heterogeneous stocks, as TSMixer does on the structured ETF panel.
Latent Factor Models Ch 14
Runs PCA and IPCA latent factor extraction on the broadest equity panel in the book. Applies per-fold panel construction retaining stocks via coverage filter within each fold's date range (rather than globally), avoiding survivorship bias from requiring continuous listing across all years.
PCA Ch 14
Extracts static PCA factors from the return covariance structure of US equities via walk-forward CV. Diagnoses variance concentration via scree plots and evaluates predictions across all available horizons (1d, 5d, 21d) to assess whether slow-moving covariance structure produces stronger IC at longer horizons.
IPCA Ch 14
Fits Instrumented PCA (Kelly, Pruitt, and Su 2019) with time-varying factor loadings as functions of observable characteristics via alternating least squares. Estimates the Gamma matrix mapping characteristics to factor loadings, revealing which are most informative. Achieves the strongest latent factor result in the book at the longer horizon.
Causal DML Ch 15
Applies DML to estimate the causal effect of 12-1 momentum (past_ret_12m_skip) on daily returns across the broad stock universe, conditioning on volatility and liquidity confounders. Finds substantial confounding and a borderline p-value -- momentum's daily effect is not statistically significant after deconfounding.
Strategy Pipeline
5 notebooks
Daily rebalancing creates extreme cost sensitivity -- breakeven ~50 bps, viable operating range only 5-15 bps. Holdout Sharpe collapses 89% from +1.76 to +0.20 (most severe decay in book). Era-dependent cost analysis spans pre/post-decimalization regimes. Borrow costs for short leg add ~50 bps/yr. The tightest cost tolerance of any case study.
Model Analysis Ch 11
Compares all predictive models (linear, GBM, deep learning, latent factors, causal DML) across the broadest panel in the book. GBM dominates on the 1-day label with all positive folds. IPCA leads at the longer horizon -- exceeding GBM. Most model families have registered results; TabM pending due to GPU compute requirements.
Backtest & Signal Evaluation Ch 16
Runs plumbing test, parametric sweep, and statistical analysis (DSR, family comparison) for a daily long-short decile strategy across the broad stock universe. GBM dominates at every pipeline stage. Strong validation Sharpe provides the best statistical foundation of any case study, but holdout shows severe decay.
Portfolio Construction Ch 17
Sweeps top predictions x TOP_K concentration x 6 allocators (equal-weight through HRP) on the broad stock universe. Evaluates covariance-aware methods that rebalance less against the daily-cadence cost constraint. The dominant question is whether any Sharpe gain from smarter allocation offsets the higher turnover.
Transaction Costs Ch 18
Sweeps the cost grid on top allocation combinations, revealing steep Sharpe decay from gross to net. The tightest cost tolerance of any case study due to daily rebalancing. Includes pre/post-decimalization era analysis (tick size regime change in 2001).
Risk Management Ch 19
Sweeps position-level (stop-loss, trailing stops) and portfolio-level (drawdown breakers, daily loss limits) risk controls on the top allocation combinations. Targets the regime-sensitivity weakness: strong validation Sharpe decays severely in holdout, driven by daily rebalancing during a regime shift. The broad universe already provides inherent diversification.
Synthesis & Verdict
1 notebook
Broadest universe, strongest latent factor result, worst holdout decay. The Fundamental Law in action: tiny per-stock IC x massive breadth. Verdict: Iterate -- decompose holdout returns by sector and calendar period to diagnose the failure, extend to 21-day horizon to reduce turnover, then ensemble GBM with IPCA.
Strategy Analysis Ch 20
Synthesizes the holdout collapse: GBM champion achieved strong validation Sharpe across many folds but holdout shows severe decay. Traces the champion through signal, allocation, cost, and risk stages via BacktestExplorer. Runs FF5+MOM factor attribution with placebo benchmark. Produces the "advance" verdict with severe caveats: holdout degradation dominates any deployment decision.
Previous
S&P 500 Options (Straddles)
Quick Info
Asset Class Equities
Frequency Daily
Data Type Price Data
Notebooks 21
Chapters 14
Libraries Used
ML4T Data ML4T Engineer ML4T Diagnostic ML4T Backtest
Chapters