Five-policy tournament with a lexicographic ranking rule
Backtest five specified policies with a shared engine and rank them by a lexicographic criterion, reporting a full comparison table.
instruction.md
What the agent sees (CONVENTIONS.md is appended automatically)
Strategy tournament
/app/strategies/*.yaml define five policies over the IPS targets: buy-and-hold, monthly calendar, quarterly calendar, 5/25 bands checked daily, and 20% relative bands checked monthly. Engine conventions are those of pf-bt-calendar-rebalance.
Write /app/output/tournament.csv with cagr, ann_vol, sharpe, sortino, max_drawdown, calmar, avg_annual_turnover, total_costs and n_trades per strategy, and /app/output/ranking.json ranking by: (1) sharpe rounded to 2 dp, descending; (2) max_drawdown ascending in magnitude; (3) avg_annual_turnover ascending.
Verification
reward.json metrics · weights sum to 1.00
| Metric | Weight | Check |
|---|---|---|
| table | 0.6 | All cells rel 1e-6; credit per strategy. |
| ranking | 0.4 | Exact order. |
Harbor scaffold
Generated from this record — task.toml, Dockerfile, verifier, oracle stub
schema_version = "1.4" [task] name = "portfolio-agent-evals/pf-eval-strategy-tournament" version = "1.0.0" description = "Backtest five specified policies with a shared engine and rank them by a lexicographic criterion, reporting a full comparison table." keywords = ["etf", "portfolio", "backtest", "strategy-evaluation", "strategy-comparison", "backtest-mechanics", "ranking-rules"] [metadata] author_name = "portfolio-agent-evals" difficulty = "medium" category = "quant-finance" tags = ["strategy-evaluation", "tier-2", "backtest", "multi-metric"] theme = "Strategy Evaluation & Overfitting Discipline" tier = 2 reward_type = "multi-metric" [agent] timeout_sec = 2400.0 [verifier] timeout_sec = 300.0 [environment] # Offline by design: all data is synthetic and generated at build time. network_mode = "none" cpus = 2 memory_mb = 4096 storage_mb = 10240 build_timeout_sec = 900.0
Traps
Each must carry signal: a trap-blind solution must lose credit
- Round Sharpe before ranking (two strategies tie at 2 dp by construction).
- Band strategies check drift at close and trade next open.
- The 20% band is relative to target; the 5/25 rule is absolute 5% or relative 25%, whichever is tighter.
- Buy-and-hold still pays initial purchase costs.
Inputs
Fixtures mounted in the environment
- /app/strategies/
- /app/data/prices/
- /app/policy/ips.yaml
- /app/data/dividends.csv
- /app/data/factors.csv
- /app/CONVENTIONS.md
Outputs
What the verifier reads from /app/output
- /app/output/tournament.csvCSVComparison table.
- /app/output/ranking.jsonJSONOrdered list with tie-break reasons.
Anti-gaming
Data is generated so a 2-dp Sharpe tie exists; the tie pair varies by seed.
Oracle notes
solution/solve.sh must score 1.0 on five seeds
Reference engine with policy plug-ins.
Reviews (0)
Design review before a task is marked ready