Four quarterly cycles with reconciliation of actual fills against the prior plan
Four quarterly steps where each step delivers a new quarter of data and the agent must reconcile actual fills and dividends against its previously approved plan before rebalancing again.
instruction.md
What the agent sees (CONVENTIONS.md is appended automatically)
A year of quarterly reviews (multi-step)
Each step appends one quarter of prices and custodian transactions. The custodian applied your previous step's trades with partial fills, occasional rejections, price slippage and DRIP reinvestment on one account, all recorded in transactions.csv.
In each step: (1) reconcile expected vs actual positions and write reconciliation.json listing {ticker, expected, actual, cause}; (2) rebalance under the IPS with the basic constraints; (3) write state.json (positions, pending blackouts, assumptions) for the next step. Always rebalance from ACTUAL positions.
Steps
Harbor multi-step layout; rewards aggregate by mean
- 1q1
Reconcile initial positions and rebalance.
- 2q2
Reconcile Q1 fills (partial fill, rejection) and rebalance.
- 3q3
Reconcile DRIP and split; rebalance.
- 4q4
Reconcile and produce year-end state.
Verification
reward.json metrics · weights sum to 1.00
| Metric | Weight | Check |
|---|---|---|
| reconciliation_exact | 0.4 | Per step: every discrepancy listed with the right cause (partial_fill, rejected, slippage, drip, split). |
| trades_feasiblegate | 0.3 | Per step: constraints hold from actual positions. GATE per step at 0.2. |
| objective | 0.3 | Per step: sum |drift| <= oracle + 1e-6. |
No cross-step gating: recovering from a bad quarter by reconciling actuals is rewarded.
Gates: trades_feasible. A gate failure caps or zeroes the trial reward regardless of other metrics.
Harbor scaffold
Generated from this record — task.toml, Dockerfile, verifier, oracle stub
schema_version = "1.4" [task] name = "portfolio-agent-evals/pf-e2e-year-of-quarters-state" version = "1.0.0" description = "Four quarterly steps where each step delivers a new quarter of data and the agent must reconcile actual fills and dividends against its previously approved plan before rebalancing again." keywords = ["etf", "portfolio", "e2e", "end-to-end", "state-management", "reconciliation", "long-horizon"] # Per-step rewards roll up by mean; gates are declared per step below. multi_step_reward_strategy = "mean" [metadata] author_name = "portfolio-agent-evals" difficulty = "hard" category = "quant-finance" tags = ["end-to-end", "tier-4", "e2e", "multi-metric"] theme = "End-to-End Multi-Step Reviews" tier = 4 reward_type = "multi-metric" [agent] timeout_sec = 1500.0 [verifier] timeout_sec = 240.0 [environment] # Offline by design: all data is synthetic and generated at build time. network_mode = "none" cpus = 2 memory_mb = 4096 storage_mb = 10240 build_timeout_sec = 900.0 [[steps]] name = "q1" # Reconcile initial positions and rebalance. [steps.agent] timeout_sec = 1500.0 [steps.verifier] timeout_sec = 240.0 [[steps]] name = "q2" # Reconcile Q1 fills (partial fill, rejection) and rebalance. [steps.agent] timeout_sec = 1500.0 [steps.verifier] timeout_sec = 240.0 [[steps]] name = "q3" # Reconcile DRIP and split; rebalance. [steps.agent] timeout_sec = 1500.0 [steps.verifier] timeout_sec = 240.0 [[steps]] name = "q4" # Reconcile and produce year-end state. [steps.agent] timeout_sec = 1500.0 [steps.verifier] timeout_sec = 240.0
Traps
Each must carry signal: a trap-blind solution must lose credit
- Partial fills and a rejected order in Q1.
- DRIP on one account changes shares without an agent trade.
- A 2:1 split in Q3 doubles shares — not a discrepancy.
- Rebalancing from the planned rather than actual positions compounds errors.
Inputs
Fixtures mounted in the environment
- /app/portfolio/Appended each step via steps/<q>/workdir.
- /app/data/prices/
- /app/policy/ips.yaml
- /app/data/corporate_actions.csvA split lands in Q3.
Outputs
What the verifier reads from /app/output
- /app/output/q{n}/reconciliation.jsonJSONPer-step reconciliation.
- /app/output/q{n}/trades.csvCSVPer-step trades.
- /app/output/q{n}/state.jsonJSONCarried state.
Anti-gaming
Custodian behaviour is seeded; the oracle reconciles from transactions alone.
Oracle notes
solution/solve.sh must score 1.0 on five seeds
Oracle per step: positions from transactions, diff vs plan, reference trade list.
Reviews (0)
Design review before a task is marked ready