Walk-forward testing Polymarket strategies

• Walk-forward chooses parameters on a train window and scores on the following test window
• The archive's 5,005 markets across 5m, 15m and 4h horizons provide many rolling windows
• Backtest AI accepts sampleSize and days, which map naturally onto test windows
• Out-of-sample degradation between windows is the signal that a strategy is overfit

Split time into consecutive windows. Fit the strategy's parameters on the first window, then score it on the next without changing anything. Move both windows forward and repeat. The result is a sequence of out-of-sample scores, not a single in-sample fit.

Because the archive spans thousands of short-horizon markets, the windows can be days rather than months, and the strategy is re-scored dozens of times. That is far more informative than one long backtest that quietly reused the same data for fitting and scoring.

Every strategy with enough parameters can be tuned to fit any finite history. The fit says nothing about the future. Walk-forward exposes this by construction: parameters chosen on window one are scored on data they never saw, and if the in-sample edge was an artifact it disappears.

The gap between in-sample and out-of-sample performance is itself the measurement. A small gap means the edge is likely real; a large one means you fit the noise. Report both, not just the flattering one.

Short windows adapt faster but are noisier; long windows are more stable but slower to react to regime change. On crypto Up/Down markets, where liquidity and spreads differ by asset, a window that is stationary for BTC may straddle a regime change for HYPE.

Set window length from the strategy's holding period and the data's stability, not from convenience. If results swing wildly with small changes to the window length, that instability is itself the finding.

Compare the in-sample and out-of-sample distributions of total_pnl and max_drawdown. A strategy that earns 30% in-sample and loses out-of-sample is not a strategy; it is a curve fit. A strategy that earns a little in-sample and slightly less out-of-sample is a candidate worth more testing.

Track degradation across the whole sequence. A single bad window can be noise; a steady trend of declining out-of-sample performance means the edge is decaying, which is a market fact worth knowing before you trade it.

import requests

BASE = "https://api.polyorderbooks.com/v1"
HEADERS = {"X-API-Key": "your-key", "Content-Type": "application/json"}

# Score the same strategy across consecutive test windows.
def score(coin, days):
    body = {"prompt": f"buy {coin} 5m up on a 0.1% reference tick",
            "sampleSize": 50, "days": days}
    r = requests.post(f"{BASE}/ai/backtest", headers=HEADERS, json=body)
    r.raise_for_status()
    return r.json()["report"]["win_rate"]

for d in (3, 5, 7, 14):
    print(f"window {d}d -> win_rate", score("btc", d))

The Starter plan is free, includes order books, prices, and metrics at 250ms resolution, with 3 days of history, 60 requests/min, 1,000/day, 1 free AI backtest, and 3 strategy backtests. No credit card required.

Paid data windows from $19/mo extend history to 30–120 days at higher throughput. Backtest AI add-on is +$19/mo or standalone at $29/mo.

• Sample size
• Reproducibility
• Backtest methodology