Afterluck
What remains of a backtest after luck is removed. Pick a market and a trading rule — this runs it, then spends the rest of its effort trying to prove the result was luck.
Any Yahoo Finance symbol, or upload your own CSV/Parquet below.
Long when the fast simple moving average is above the slow one, short when below.
| Ticker | Strategy |
|---|
Nothing tested yet
Pick a market and a rule, then hit Run Afterluck. A full run is a few seconds.
Cross-sectional strategies rank names against each other, so they get a harder null: we keep every date's gross exposure, net exposure and position count exactly as they were and randomise only which name got which weight. A book that beats that is picking names. One that doesn't was being paid for style exposure you can buy in an ETF — which the factor regression below measures directly.
Long the past winners, short the past losers, skipping the most recent month.
Nothing tested yet
Pick a universe and a ranking rule, then hit Run portfolio check.
Upload your data: OHLCV prices (then pick a built-in rule), a pre-computed positions series, or a bare returns track record. Partial panels are graded only on the tests that can run — the yellow banner calls out what was skipped.
Nothing tested yet
Choose a mode, upload a file, then hit Run validation.
Race every strategy on the same market, ranked by evidence rather than return. Buy & hold and a coin flip stay in the field as controls.
Why most backtests are wrong
A backtest is a measurement taken with a ruler you built after seeing the thing you are measuring. Four failure modes do almost all the damage, and this Space tests for each one.
1. The market had no structure to find — permutation test
We take the real price series and shuffle it: each bar's gap, high, low, body and volume are kept intact, but their order is destroyed. The result is a market with the same volatility and the same fat tails, and no exploitable structure whatsoever. Then we re-run your exact rule on hundreds of these shuffled markets.
If your Sharpe sits inside that cloud of results, your rule found nothing that a coin-flip market would not also have handed it. The p-value is the share of shuffled markets that did as well or better.
Block mode resamples contiguous chunks instead of single bars, preserving short-horizon momentum and volatility clustering. It is a harder null, and trend strategies should be held to it.
2. You tried 200 things and reported the best — Deflated Sharpe Ratio
If you test 200 worthless strategies, the best of them will show a Sharpe near 1.0 purely by chance. The Deflated Sharpe Ratio (Bailey & López de Prado, 2014) works out what the luckiest of N skill-free variants would have scored, and asks whether yours beats that bar — with an extra penalty for negative skew and fat tails, the return shapes that flatter naive Sharpe ratios.
This Space counts the whole parameter grid as trials, because that is what a researcher would really have run.
3. The parameters were fitted to the past — PBO and walk-forward
Probability of Backtest Overfitting (CSCV) cuts the timeline into chunks, and for every way of splitting them half in-sample and half out-of-sample, checks whether the in-sample winner stayed a winner. If the winner lands in the bottom half about half the time, PBO ≈ 50% and your selection process has no skill at all.
Walk-forward re-tunes on a training window and trades the next window blind, rolling forward. Efficiency is out-of-sample Sharpe over in-sample Sharpe: 100% means the edge survived intact, 0% means it was entirely curve-fit.
4. The edge is smaller than the costs — stress test
Every result here is net of commission and slippage charged on exposure changes, plus a borrow fee on short positions. We then re-run at triple the friction. A real edge degrades; a fake one disappears.
The Reality Score
| Weight | Component | What it measures |
|---|---|---|
| 30% | Significance | How far outside the shuffled-market null the result sits |
| 25% | Selection | Deflated Sharpe — does it clear the best-of-N bar |
| 20% | Walk-forward | How much of the tuned Sharpe survived trading forward |
| 15% | Overfitting | 1 − PBO, from combinatorially symmetric cross-validation |
| 10% | Robustness | Sharpe retained when costs triple |
Grades: A ≥ 85 · B ≥ 70 · C ≥ 55 · D ≥ 40 · F below 40.
The scale is deliberately harsh. Most strategies people post online score below 40, and the honest response to that is not to soften the scale.
No look-ahead, by construction
A strategy emits a target exposure at each bar's close using only data up to that
bar. The engine holds position[t] = target[t - lag] with lag ≥ 1, so a signal
computed on Tuesday's close cannot earn Tuesday's move. That is the single line where
look-ahead could enter, and the test suite asserts it directly.
Use it from Python
from algotrader import LabConfig, run_lab
report = run_lab(LabConfig(symbol="SPY", strategy="sma_cross", params={"fast": 20, "slow": 100}))
print(report.verdict["grade"], report.verdict["score"])
print(report.permutation.p_value, report.dsr["dsr"], report.pbo["pbo"])
Or from the command line:
python -m algotrader.cli lab --symbol SPY --strategy donchian_breakout --permutations 500
python -m algotrader.cli arena --symbol BTC-USD
Research tooling, not investment advice. Nothing here is a recommendation to trade.
algotrader 2.1.0 · Apache-2.0 · Research tooling, not investment advice.