Evaluating Polymarket backtests: the metrics that matter

• The report returns total_pnl, win_rate, profit_factor, max_drawdown and an equity curve
• It also returns exit_reasons, fill_sources, a trades table and a skips list
• win_rate alone is meaningless without the average win and loss sizes
• max_drawdown and the equity curve show risk that a single PnL figure hides

total_pnl is the sum of realized results across sampled trades. It answers 'did the strategy make money over this sample' but says nothing about how much capital was at risk or how lumpy the outcome was. A strategy can be profitable overall and still blow up on the way there.

profit_factor is gross profit divided by gross loss - a value above one means profitable. It is more informative than PnL because it is scale-free, but it can be gamed by a single enormous winner, so check the trade count before trusting it.

win_rate is the share of trades that closed profitably. On binary markets a high win rate is easy to fake: take small profits, hold losers to expiry, and the win rate looks great while the account bleeds. Always read win_rate next to the average winner and average loser.

A strategy winning 40% at three-to-one payoff is far stronger than one winning 70% at one-to-three. The report's trade table lets you compute both averages rather than inferring them from the rate alone.

max_drawdown is the deepest peak-to-trough decline in the equity curve. It is the number that decides whether a strategy is tradeable regardless of its PnL: a high return with a 60% drawdown will be abandoned in practice before it recovers.

The equity curve itself shows the shape. A smooth climb and a jagged climb can share a final value and a max drawdown while implying completely different stress. Plot it before you believe any single statistic.

fill_sources reports where simulated fills came from, which exposes an over-optimistic fill model: if almost every fill is at the midpoint, the backtest is not realistic. The skips list records trades that could not be executed because the book lacked depth, and a long skip list means the strategy depends on liquidity that was not there.

These two fields are the honesty check on the rest of the report. A high PnL alongside many skips and mostly-mid fills is a strategy that works only in the simulation's imagination.

import requests

BASE = "https://api.polyorderbooks.com/v1"
HEADERS = {"X-API-Key": "your-key", "Content-Type": "application/json"}

body = {
    "prompt": "buy btc 5m up when the reference ticks up 0.1%",
    "sampleSize": 100,
    "days": 7,
}
r = requests.post(f"{BASE}/ai/backtest", headers=HEADERS, json=body)
r.raise_for_status()
rep = r.json()["report"]
for k in ("total_pnl", "win_rate", "profit_factor", "max_drawdown"):
    print(k, rep[k])
print("skips:", len(rep.get("skips", [])))

The Starter plan is free, includes order books, prices, and metrics at 250ms resolution, with 3 days of history, 60 requests/min, 1,000/day, 1 free AI backtest, and 3 strategy backtests. No credit card required.

Paid data windows from $19/mo extend history to 30–120 days at higher throughput. Backtest AI add-on is +$19/mo or standalone at $29/mo.

• Sample size
• Fill model
• Backtest methodology