Backtest Validation in 2026: Walk-Forward, Purging and the Deflated Sharpe Ratio

1. The Problem: Most Backtests Are Rigged by Accident

A backtest should answer one question: would this rule have made money on data it had not seen? Three things quietly change the question. Information from the future leaks into decisions. The cost of trading is left out. And you try many variants and report the best one. The first two inflate every run; the third inflates the winner, even when no variant has any skill.

The fix is not a smarter model. It is three layers of discipline: a pipeline where the future cannot reach the past, an evaluation scheme that respects time (walk-forward with purge and embargo), and a statistic that knows how many times you looked, the Deflated Sharpe Ratio of Bailey and López de Prado.

2. Minimum Concepts Before Touching Code

Six ideas carry the rest of the tutorial. Keep them in mind and every code block reads as plain English.

3. Step 1 — A Market Where You Know the Answer

To test a validation pipeline you need data where the truth is known. The lab is a seeded random walk whose log-returns are independent with zero mean: no rule can predict the next bar, so every ’edge’ the code finds is a bug or a fluke. First leak: the unshifted signal.

Python — synthetic series, an unshifted signal and its honest twin

import numpy as np
import pandas as pd

ANN = np.sqrt(252)                           # per-bar Sharpe -> annualized (daily bars)

def make_close(seed, n=2520):
    """Zero-drift random walk: by construction there is NO edge to find."""
    r = np.random.default_rng(seed).normal(0.0, 0.01, n)
    idx = pd.bdate_range("2016-01-04", periods=n)
    return pd.Series(100 * np.exp(np.cumsum(r)), index=idx)

def sharpe(x):                               # per bar, NOT annualized
    return x.mean() / x.std(ddof=1)

close = make_close(3)
ret = close.pct_change()
ma = close.rolling(20).mean()
side = pd.Series(np.where(close > ma, 1.0, -1.0), index=close.index)   # +1 / -1, known at the close

leaky = (side * ret).iloc[20:]               # decided with close[t], earns ret[t]
honest = (side.shift(1) * ret).iloc[20:]     # decided with close[t-1], earns ret[t]
print(f"leaky  Sharpe: {sharpe(leaky) * ANN:5.2f}")    # 5.09
print(f"honest Sharpe: {sharpe(honest) * ANN:5.2f}")   # 0.08

With seed 3 (numpy 2.2.6, pandas 2.3.3, scipy 1.16.3) the leaky version prints an annualized Sharpe of 5.09 and the shifted one 0.08. Same rule, same data, one .shift(1) apart. Every figure in this post is synthetic and annualized with the square root of 252; digits can differ on other library versions, the conclusions should not.

Python — survivorship bias in a dozen lines

rng = np.random.default_rng(11)
panel = rng.normal(0.0, 0.02, (1000, 2000))      # 2,000 zero-drift assets x 1,000 days
final = np.cumprod(1 + panel, axis=0)[-1]        # terminal value of 1 unit invested
alive = final > 0.5                              # "delisted" if it lost more than half
print(f"whole universe : {final.mean():.2f}")                            # 1.01
print(f"survivors only : {final[alive].mean():.2f}  ({alive.mean():.0%} survive)")  # 1.19 (78%)

2,000 assets built with zero drift average a terminal value of 1.01, as designed. Keep only the 78% that never lost more than half and the average becomes 1.19. No asset has skill; the filter is the whole effect.

4. Step 2 — A Leakage-Free Pipeline in Plain pandas

Three rules, in this order: split before you fit, act one bar after you decide, and charge for every unit you trade.

Python — split first, fit on train only

cut = int(len(ret) * 0.7)
train, test = ret.iloc[1:cut], ret.iloc[cut:]    # 1) split FIRST, in time order, no shuffling

mu, sd = train.mean(), train.std()               # 2) every fitted quantity comes from train only
z_test = (test - mu) / sd                        # 3) test is only ever transformed, never fitted

# leaky contrast: (ret - ret.mean()) / ret.std() lets the test block shape its own scaler

The second rule is the .shift(1) from step 1, now inside a function that also charges costs. fee_bps and slip_bps apply to turnover, the units traded each bar; flipping from short to long is two units. The delay argument adds extra bars between decision and fill when you want to be more conservative.

Python — signal, shift, turnover, costs

def side_of(close, fast, slow):
    """+1 / -1 from a moving-average crossover, known at the close of bar t."""
    ma_f, ma_s = close.rolling(fast).mean(), close.rolling(slow).mean()
    side = pd.Series(np.where(ma_f > ma_s, 1.0, -1.0), index=close.index)
    return side.where(ma_s.notna(), 0.0)         # flat until the slow window is warm

def backtest(close, fast, slow, fee_bps=5, slip_bps=5, delay=0):
    ret = close.pct_change()
    pos = side_of(close, fast, slow).shift(1 + delay).fillna(0.0)   # act on the NEXT bar (+ delay)
    turnover = pos.diff().abs().fillna(0.0)                          # units traded per bar
    cost = turnover * (fee_bps + slip_bps) / 1e4
    return (pos * ret - cost).iloc[1:]

gross = backtest(close, 10, 50, fee_bps=0, slip_bps=0)
net = backtest(close, 10, 50)                    # 5 bps fee + 5 bps slippage per unit traded
print(f"gross Sharpe: {sharpe(gross) * ANN:5.2f}   net Sharpe: {sharpe(net) * ANN:5.2f}")  # -0.51 / -0.60

On seed 3 the 10/50 crossover prints a gross Sharpe of −0.51 and a net Sharpe of −0.60: costs only ever subtract, and a rule that flips often pays them more often.

Python — prove there is no look-ahead with a truncation test

def assert_no_lookahead(fn, close, cut=1500):
    """Positions on the truncated history must equal positions on the full history."""
    full = fn(close).iloc[:cut]
    partial = fn(close.iloc[:cut])
    assert np.allclose(full, partial, equal_nan=True), "signal at t changed when the future was removed"

assert_no_lookahead(lambda c: side_of(c, 10, 50), close)             # passes silently

centered = lambda c: np.sign(c - c.rolling(11, center=True).mean())  # peeks 5 bars ahead
try:
    assert_no_lookahead(centered, close)
except AssertionError as e:
    print("caught:", e)

Recompute the positions on a truncated history and compare them with the same rows from the full history. If removing the future changes the past, the signal peeks. The centered window fails the test and the pipeline passes it. I keep this assertion in every research repo.

5. Step 3 — Walk-Forward With Purge and Embargo, Written by Hand

Libraries hide the index logic, and the index logic is where the leaks live, so write it once. Every row i has a label that looks h bars ahead: a forward return, a trade that closes h bars later. A training row within h bars before the test block carries test-period prices in its label, so purging drops it. The embargo drops e more rows right after the test block, because neighbours of a test block are correlated with it.

Python — purged_splits: the whole idea in 15 lines

def purged_splits(n, n_folds, horizon, embargo=0, forward_only=False, min_train=1):
    """Yield (train_idx, test_idx) as positional indices.
    horizon      : bars a label looks ahead (label of row i spans rows i..i+horizon)
    embargo      : bars dropped from train right AFTER each test block
    forward_only : True = walk-forward (train strictly before the test block)"""
    pos = np.arange(n)
    bounds = np.linspace(0, n, n_folds + 1).astype(int)
    for a, b in zip(bounds[:-1], bounds[1:]):            # test block = rows a .. b-1
        overlaps = (pos + horizon >= a) & (pos < b)      # label window touches the test block
        after = (pos >= b) & (pos < b + embargo)         # serial-correlation buffer
        keep = ~(overlaps | after)
        if forward_only:
            keep &= pos < a
        if keep.sum() >= min_train:
            yield pos[keep], pos[a:b]

for tr, te in purged_splits(20, 4, horizon=2, embargo=1):
    print("test", te.tolist(), "train", tr.tolist())
# test [0, 1, 2, 3, 4]       train [6 .. 19]                       (5 embargoed)
# test [5, 6, 7, 8, 9]       train [0, 1, 2, 11 .. 19]             (3, 4 purged; 10 embargoed)
# test [10, 11, 12, 13, 14]  train [0 .. 7, 16 .. 19]              (8, 9 purged; 15 embargoed)
# test [15, 16, 17, 18, 19]  train [0 .. 12]                       (13, 14 purged)

Read the toy output. With horizon=2 two rows before each test block are purged, and with embargo=1 the row right after it is gone too. With forward_only=True training never sits after the test block, which is walk-forward: there the embargo has nothing to remove and the purge does the work. scikit-learn’s TimeSeriesSplit has a gap argument, the number of samples excluded from the end of each train set before the test set, that plays the purge role: set it to at least your label horizon.

Python — walk-forward: select on train rows, score on rows never seen

import itertools

GRID = list(itertools.product((3, 5, 8, 10, 15, 20), (30, 50, 75, 100, 150, 200)))   # 36 trials
H = 5                                                    # label horizon: forward 5-bar return

def walk_forward(close, grid=GRID, n_folds=10, horizon=H, embargo=0):
    fwd = close.shift(-horizon) / close - 1              # label: looks H bars into the future
    sides = {p: side_of(close, *p) for p in grid}
    nets = {p: backtest(close, *p).reindex(close.index) for p in grid}
    oos = []
    for train, test in purged_splits(len(close), n_folds, horizon, embargo,
                                     forward_only=True, min_train=252):
        score = {p: np.nanmean(sides[p].iloc[train] * fwd.iloc[train]) for p in grid}  # train rows only
        best = max(score, key=score.get)
        oos.append(nets[best].iloc[test])                # scored on rows it never saw
    return pd.concat(oos).dropna()

oos = walk_forward(close)
print(f"walk-forward OOS Sharpe (net): {sharpe(oos) * ANN:5.2f} over {len(oos)} bars")  # 0.68 / 2016

Each fold picks the best of the 36 crossovers by how well its signal predicted the forward 5-bar return on training rows only, then trades the following block. On seed 3 the stitched out-of-sample Sharpe is 0.68 over 2,016 bars, net of costs.

6. Step 4 — Count Every Trial, Then Deflate the Sharpe

You never run one backtest. You run a grid, move a threshold, try another asset, drop a feature. The Deflated Sharpe Ratio (Bailey and López de Prado, Journal of Portfolio Management, 2014) asks: given that this Sharpe is the best of N trials, how likely is it that the true Sharpe is above what luck alone would produce? Two ingredients do the work: SR0, the Sharpe you would expect the best of N zero-skill trials to reach, and the standard error of your Sharpe, which grows with negative skew and fat tails.

The equations, as printed in the paper

Eq. 1 with the zero-skill null E[{SR_n}] = 0:

  SR0 = sqrt( V[{SR_n}] ) · ( (1 − γ) · Z⁻¹[ 1 − 1/N ]  +  γ · Z⁻¹[ 1 − 1/(N·e) ] )

Eq. 2:

  DSR = PSR(SR0) = Z[ (SR − SR0) · sqrt(T − 1) / sqrt( 1 − γ3 · SR + ((γ4 − 1)/4) · SR² ) ]

Eq. 9 (implied independent trials from M correlated ones, ρ = average pairwise correlation):

  N_hat = ρ + (1 − ρ) · M

Python — SR0 and DSR, checked against the paper’s worked example

from scipy.stats import norm, skew, kurtosis

EULER_GAMMA = 0.5772156649015329

def expected_max_sharpe(var_sr, n_trials):
    """SR0: expected best Sharpe of n_trials independent zero-skill trials (Eq. 1, E[SR_n] = 0)."""
    return np.sqrt(var_sr) * ((1 - EULER_GAMMA) * norm.ppf(1 - 1 / n_trials)
                              + EULER_GAMMA * norm.ppf(1 - 1 / (n_trials * np.e)))

def deflated_sharpe_ratio(sr, sr0, T, g3, g4):
    """Eq. 2. sr, sr0 per bar (NOT annualized); g3 skewness; g4 NON-excess kurtosis (Normal = 3)."""
    se = np.sqrt(1 - g3 * sr + (g4 - 1) / 4 * sr ** 2)
    return norm.cdf((sr - sr0) * np.sqrt(T - 1) / se)

# sanity check: the paper's worked example (N=100, V=1/2 per year, T=1250, skew -3, kurtosis 10)
sr0 = expected_max_sharpe(1 / (2 * 250), 100)
dsr = deflated_sharpe_ratio(2.5 / np.sqrt(250), sr0, 1250, -3, 10)
print(f"SR0 = {sr0:.4f}   DSR = {dsr:.4f}")              # SR0 = 0.1132   DSR = 0.9004

Do not skip the check. The paper’s example (100 trials, annual variance of trial Sharpes 1/2, 1,250 observations, skewness −3, kurtosis 10, annualized Sharpe 2.5) yields SR0 ≈ 0.1132 and DSR ≈ 0.9004, below 0.95. If your implementation does not reproduce those numbers, it is wrong. Mine does.

Python — a research session: 36 trials, one winner, three verdicts

def effective_trials(trials):
    """Eq. 9 of the paper: N_hat = rho + (1 - rho) * M, rho = average pairwise correlation."""
    m = trials.shape[1]
    rho = (trials.corr().to_numpy().sum() - m) / (m * (m - 1))
    return rho + (1 - rho) * m

def research_session(close, grid=GRID):
    trials = pd.DataFrame({p: backtest(close, *p) for p in grid})    # EVERY trial is kept
    srs = trials.apply(sharpe)                                        # per-bar Sharpe of each trial
    best = srs.idxmax()
    r, sr, T = trials[best], srs[best], len(trials)
    g3, g4 = skew(r), kurtosis(r, fisher=False)
    n_hat = effective_trials(trials)
    sr0 = expected_max_sharpe(srs.var(ddof=1), len(grid))
    sr0_hat = expected_max_sharpe(srs.var(ddof=1), max(n_hat, 2))
    return {"best": tuple(int(x) for x in best), "sr_ann": sr * ANN,
            "psr_vs_0": deflated_sharpe_ratio(sr, 0.0, T, g3, g4),
            "sr0_ann": sr0 * ANN, "dsr": deflated_sharpe_ratio(sr, sr0, T, g3, g4),
            "n_hat": n_hat, "dsr_n_hat": deflated_sharpe_ratio(sr, sr0_hat, T, g3, g4)}

for k, v in research_session(close).items():
    print(f"{k:10s}", v if k == "best" else round(v, 3))
# best (20, 150) | sr_ann 0.772 | psr_vs_0 0.993 | sr0_ann 0.833
# dsr 0.424 | n_hat 15.53 | dsr_n_hat 0.598

Python — 200 seeds of pure noise: how often does each test cry wolf?

rows = []
for seed in range(200):
    c = make_close(seed)
    res = research_session(c)
    res["wf_sr_ann"] = sharpe(walk_forward(c)) * ANN
    rows.append(res)
mc = pd.DataFrame(rows)

print("best-of-36 Sharpe, median      :", round(mc.sr_ann.median(), 2))     # 0.3
print("naive test (PSR > 0.95)        :", (mc.psr_vs_0 > 0.95).mean())      # 0.235
print("deflated (DSR > 0.95), N = 36  :", (mc.dsr > 0.95).mean())           # 0.01
print("deflated (DSR > 0.95), N_hat   :", (mc.dsr_n_hat > 0.95).mean())     # 0.02
print("walk-forward Sharpe, median    :", round(mc.wf_sr_ann.median(), 2))  # -0.05
print("walk-forward Sharpe > 0.5      :", (mc.wf_sr_ann > 0.5).mean())      # 0.055

Across 200 seeds the median best-of-36 annualized Sharpe is 0.30. The naive test calls 23.5% of those runs significant at 95%. The DSR with N = 36 calls 1.0% and with N̂ 2.0%, under the 5% a single honest test would give. The in-sample winner is biased upward; the walk-forward median of −0.05 is not.

7. Common Errors (All of Them Fixable in an Afternoon)

Almost every inflated backtest I have debugged falls into one of these buckets. The left column is what trustworthy research does; the right one produces a beautiful equity curve and nothing else.

8. A Checklist to Run Before You Believe a Backtest

Paste it in your repo’s README and make code review fail without it. The three groups map to when each check is cheapest.

Conclusion

A trustworthy backtest is mostly subtraction: remove the future from the past, add the costs you left out, and deflate the winner by how many times you looked. You built each piece on a market that has no edge, the only place where you can be sure the pipeline is not inventing one.

Next time you like a backtest, run the pipeline on a random walk first. If it finds an edge there, you have a bug; if it finds none, run it on your data and report N next to the Sharpe. The numbers get smaller and a lot more useful.

If you want the other half of the story, building and running the bot itself, start with my guide to building trading bots and then put every backtest from it through the checklist above.

Sources

• Bailey & López de Prado — The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality (SSRN 2460551)
• Same paper, author PDF (Eq. 1, Eq. 2, Eq. 9 and the numerical example used above)
• Bailey, Borwein, López de Prado & Zhu — The Probability of Backtest Overfitting (CSCV and PBO)
• López de Prado — Advances in Financial Machine Learning, Wiley, 2018 (chapter 7: purged k-fold and embargo)
• Deep, Deep & Lamptey — Interpretable Hypothesis-Driven Trading: A Rigorous Walk-Forward Validation Framework (arXiv 2512.12924, v1)
• scikit-learn — TimeSeriesSplit (the gap argument)
• SciPy — scipy.stats.kurtosis (fisher=False gives Pearson kurtosis, Normal = 3)