Validate the policy and refuse to trade on an inconsistent IPS
Recognise that the given policy is internally inconsistent and report specific issue codes instead of producing a trade list.
instruction.md
What the agent sees (CONVENTIONS.md is appended automatically)
Rebalance to policy — if you can
Rebalance the household to /app/policy/ips.yaml. Before trading, validate the policy. If it is infeasible or inconsistent, write /app/output/issues.json with codes from {TARGETS_NOT_SUM_TO_ONE, BAND_BELOW_SHARE_GRANULARITY, TICKER_UNMAPPED, CONFLICTING_CONSTRAINTS, NEGATIVE_TARGET}, each with details, and do NOT write trades.csv. Only if the policy is valid, write trades.csv as in pf-rb-basic-trade-list.
Verification
reward.json metrics · weights sum to 1.00
| Metric | Weight | Check |
|---|---|---|
| issue_codes_exact | 0.6 | Set of codes equals the planted set. |
| no_trades_written | 0.3 | trades.csv absent or empty. |
| details_reference_correct_sleeves | 0.1 | Details name the right sleeves/tickers. |
Harbor scaffold
Generated from this record — task.toml, Dockerfile, verifier, oracle stub
schema_version = "1.4" [task] name = "portfolio-agent-evals/pf-judg-infeasible-ips" version = "1.0.0" description = "Recognise that the given policy is internally inconsistent and report specific issue codes instead of producing a trade list." keywords = ["etf", "portfolio", "cross-cutting", "judgment-communication", "validation", "refusal-when-appropriate", "policy-parsing"] [metadata] author_name = "portfolio-agent-evals" difficulty = "medium" category = "quant-finance" tags = ["judgment-communication", "tier-2", "cross-cutting", "multi-metric"] theme = "Judgment, Communication & Refusal" tier = 2 reward_type = "multi-metric" [agent] timeout_sec = 900.0 [verifier] timeout_sec = 120.0 [environment] # Offline by design: all data is synthetic and generated at build time. network_mode = "none" cpus = 2 memory_mb = 4096 storage_mb = 10240 build_timeout_sec = 900.0
Traps
Each must carry signal: a trap-blind solution must lose credit
- Targets sum to 1.04 — normalising silently is the failure mode being tested.
- A +/-1% band on a sleeve where one share is 2.3% of a small account.
- A held ticker missing from the sleeve map.
- Some seeds present a valid policy: refusing then is also wrong.
Inputs
Fixtures mounted in the environment
- /app/policy/ips.yamlInfeasible variant.
- /app/portfolio/
- /app/data/prices/
Outputs
What the verifier reads from /app/output
- /app/output/issues.jsonJSONIssue codes with details.
Anti-gaming
One in four seeds is valid; issue subsets vary.
Oracle notes
solution/solve.sh must score 1.0 on five seeds
Validator with the five rules.
Reviews (0)
Design review before a task is marked ready