Coverage matrix
Three views of the same suite: capability axis × tier (does every axis have headroom?), theme × tier (does every theme ladder up?), and phase × reward type (are we over-relying on judges anywhere?).
Capability axis × tier
A task counts toward an axis if it carries any of the axis's capability tags. Tasks can count toward several axes.
| Axis | Tier 1 | Tier 2 | Tier 3 | Tier 4 | Total |
|---|---|---|---|---|---|
Convention-exact quant | 1 | 4 | 5 | · | 10 |
Methodological rigor | · | 1 | 7 | 1 | 9 |
Feasibility & optimality | 1 | 3 | 7 | · | 11 |
Judgment & honesty | · | 5 | 4 | · | 9 |
Engineering | · | 4 | 5 | 2 | 11 |
Long-horizon agency | · | · | 1 | 3 | 4 |
Theme × tier
| Theme | Tier 1 | Tier 2 | Tier 3 | Tier 4 | Gated | Total |
|---|---|---|---|---|---|---|
| Data Forensics & Canonicalization Analyze | 1 | 2 | 2 | · | 0 | 5 |
| Portfolio Analytics & Exposure Analyze | 1 | 3 | 2 | · | 0 | 6 |
| Backtest Engine Correctness Backtest | · | 1 | 4 | · | 0 | 5 |
| Strategy Evaluation & Overfitting Discipline Backtest | · | 2 | 2 | · | 0 | 4 |
| Rebalancing & Trade Generation Rebalance | 1 | 2 | 2 | · | 4 | 5 |
| Tax-Aware & Constrained Optimization Rebalance | · | 1 | 4 | · | 3 | 5 |
| Debugging, Tooling & Performance Cross-cutting | · | 1 | 2 | 1 | 2 | 4 |
| Judgment, Communication & Refusal Cross-cutting | · | 2 | 2 | · | 1 | 4 |
| Adversarial Robustness & Safety Cross-cutting | · | 3 | 1 | · | 3 | 4 |
| End-to-End Multi-Step Reviews End-to-end | · | · | 1 | 3 | 3 | 4 |
Phase × reward type
| Phase | binary | partial | multi-metric | llm-judge-hybrid | Multi-step |
|---|---|---|---|---|---|
| Analyze | · | 3 | 8 | · | 0 |
| Backtest | · | 1 | 8 | · | 0 |
| Rebalance | · | 1 | 9 | · | 0 |
| Cross-cutting | · | · | 11 | 1 | 0 |
| End-to-end | · | · | 4 | · | 3 |
All capability tags
optimization 6backtest-mechanics 5trade-generation 4calendar-alignment 3reconciliation 3scope-discipline 3anomaly-detection 2constraint-satisfaction 2convention-adherence 2corporate-actions 2cost-modelling 2debugging 2full-pipeline 2instruction-hierarchy 2json-contract 2long-horizon 2policy-parsing 2robustness 2root-finding 2spec-adherence 2state-management 2wash-sale 2adjustment-math 1api-discipline 1asset-location 1assumption-management 1attribution 1authorisation-awareness 1cardinality-optimization 1cashflow-handling 1change-management 1cli-engineering 1code-audit 1code-review 1communication 1constraint-modelling 1covariance-shrinkage 1currency-conversion 1data-cleaning 1data-gap-handling 1data-profiling 1dividend-math 1dividends 1econometrics 1engineering-hygiene 1event-simulation 1evidence-based-reasoning 1ewma 1execution-timing 1exposure-management 1feasibility-reasoning 1financing-math 1generalisation 1geometric-linking 1hac-inference 1honesty 1identifier-normalisation 1injection-resistance 1look-ahead-bias 1look-through 1milp 1minimal-diff-reasoning 1multi-account 1multiple-testing 1numeric-grounding 1numerical-methods 1overlay-strategies 1performance-engineering 1performance-measurement 1position-reconstruction 1prioritisation 1qp 1ranking-rules 1recursion 1refusal-when-appropriate 1regression-testing 1reproducibility 1resampling 1resource-limits 1returns-math 1risk-metrics 1secret-hygiene 1self-checking 1self-consistency 1signal-construction 1spec-completion 1statistical-honesty 1statistical-rigor 1strategy-comparison 1survivorship-bias 1tax-loss-harvesting 1tax-lots 1uncertainty-quantification 1universe-construction 1validation 1walk-forward 1weights-math 1withdrawal-planning 1