ML4T Diagnostic¶

Signal diagnostics, statistical validation, and backtest evaluation for quantitative trading workflows.

ml4t.diagnostic tests signals, models, and backtest results for leakage, overfitting, and multiple-testing bias within the ML4T package suite.

Start with an executable check¶

This example asks whether the best of three strategy variants remains significant after accounting for selection.

import numpy as np

from ml4t.diagnostic.evaluation.stats import deflated_sharpe_ratio

rng = np.random.default_rng(42)
returns = rng.normal(
    loc=[0.0002, 0.0005, 0.0001],
    scale=0.01,
    size=(252, 3),
)

result = deflated_sharpe_ratio(
    returns,
    frequency="daily",
    correlation_method="effective_rank",
    min_k_eff=2.0,
)

print(f"Probability of skill: {result.probability:.3f}")
print(f"Expected maximum Sharpe from noise: {result.expected_max_sharpe:.3f}")
print(f"Significant: {result.is_significant}")

With this fixed seed, the result reports Significant: False and a corrected probability near 0.39. The selected variant does not clear a 95% significance threshold after accounting for three trials. For a signal-analysis first run, continue to the synthetic quickstart.

Choose the guide for your task¶

Task	Guide
Analyze cross-sectional predictions	Quickstart
Prevent leakage in time-series validation	Cross-validation
Correct Sharpe and IC significance	Statistical tests
Control false discoveries across signals	Statistical tests
Diagnose feature quality	Feature diagnostics
Select features systematically	Feature selection
Inspect trades and recurring losses	Trade analysis
Generate HTML backtest reports	Backtest tearsheets
Find examples in the public book	Book Guide
Move from Alphalens or Pyfolio	Migration guide

Validation areas¶

The package separates four stages of analysis:

Feature diagnostics test stationarity, autocorrelation, distribution, and volatility.
Signal analysis measures information coefficient, quantile returns, spread, and turnover.
Backtest analysis applies DSR, PBO, RAS, FDR control, and trade-level diagnostics.
Portfolio analysis measures returns, drawdowns, risk, and factor attribution.

The API reference lists exact public imports. The book guide maps the library to Machine Learning for Trading, Third Edition.

Install¶

pip install ml4t-diagnostic

See the installation guide for optional visualization, dashboard, backtest, and data integrations.