Backtest Engine Correctness
Match an oracle equity curve to 1e-6 under a precisely specified execution model.
Why this theme exists
The agent must implement simulation mechanics exactly: decision-at-close / execute-at-next-open timing, per-fill and proportional costs, whole-share rounding with cash feasibility, dividends by ex-date shares paid on pay-date, universe changes (launches, liquidations) without look-ahead, overlays with strictly-prior information.
Capability under test
Zero-look-ahead engineering, path-dependent state handling, and disciplined adherence to a spec where every sentence matters.
Design notes
Because outputs are fully determined by the spec, verification is exact. Traps target the most common practitioner errors (same-day execution, meta-driven universes, future dividends leaking into adjusted prices, wrong EWMA lag). Each engine task shares CONVENTIONS.md so agents cannot claim ambiguity.
Tier ladder
- Tier 10
- Tier 21
- Tier 34
- Tier 40
| Task | Tier | Phase | Reward | Gates | Budget | Status |
|---|---|---|---|---|---|---|
| Fixed-weight monthly rebalance with next-open execution, costs, whole shares and dividends pf-bt-calendar-rebalance Implement a precisely specified fixed-weight monthly-rebalance backtest with next-open execution, per-fill costs, whole shares and cash-settled dividends; match the oracle equity curve to 1e-6. | T2 medium | Backtest | multi-metric | — | 40 min | ready |
| Dual momentum rotation, exactly as specified pf-bt-dual-momentum Implement a fully specified dual-momentum rotation with exact eligibility rules and reproduce oracle month-by-month holdings and equity. | T3 hard | Backtest | multi-metric | — | 30 min | ready |
| Universe changes (launch and liquidation) without look-ahead pf-bt-inception-delisting Backtest an equal-weight-of-available-ETFs rule through an ETF launch and a liquidation without look-ahead in universe construction. | T3 hard | Backtest | multi-metric | — | 40 min | ready |
| Audit and fix a look-ahead-inflated momentum backtest pf-bt-lookahead-audit Audit a provided backtest script whose reported Sharpe is inflated by several look-ahead and survivorship bugs; find, classify and fix them, then report the honest results. | T3 hard | Backtest | multi-metric | — | 40 min | ready |
| Volatility-targeting overlay with strictly-prior EWMA and financing pf-bt-vol-target-overlay Implement a daily volatility-targeting overlay with EWMA vol estimated strictly from prior data, a leverage cap, financing at the risk-free rate plus a spread, and turnover accounting. | T3 hard | Backtest | multi-metric | — | 30 min | ready |