End-to-End Multi-Step Reviews
The whole loop, across Harbor steps, with state carried forward and requirements that change mid-flight.
Why this theme exists
Long-horizon tasks using Harbor's multi-step layout: a quarterly review split into gated steps, a single-shot full pipeline on hostile inputs, a year of quarterly cycles requiring reconciliation of actual fills against the prior plan, and a late client change that must be absorbed with a minimal amendment.
Capability under test
Prioritization under time pressure, error containment across stages, state management across steps, reconciliation, and minimal-diff reasoning when requirements change.
Design notes
Step rewards aggregate by mean; early gating is used only where downstream credit would be meaningless (a wrong panel). The year-of-quarters task deliberately does not gate so that an agent which recovers from a bad Q1 by reconciling from actual custodian positions is rewarded for resilience.
Tier ladder
- Tier 10
- Tier 20
- Tier 31
- Tier 43
| Task | Tier | Phase | Reward | Gates | Budget | Status |
|---|---|---|---|---|---|---|
| Absorb a late client change with a minimal amendment to the plan pf-e2e-change-request-minimal-amendment2 steps Produce a rebalance plan, then absorb a late client change (an exclusion and tighter bands) by amending the plan minimally rather than regenerating it. | T3 hard | End-to-end | multi-metric | 2 gates | 20 min / step | ready |
| The whole pipeline in one shot on messy, injected, under-specified inputs pf-e2e-full-pipeline-oneshot The whole pipeline in one shot on messy, injected, under-specified inputs with partial credit across six metrics, a safety gate and an honesty bonus. | T4 expert | End-to-end | multi-metric | — | 90 min | ready |
| Quarterly review as four gated Harbor steps pf-e2e-quarterly-review-multistep4 steps A four-step Harbor task covering the full analyze, backtest, rebalance and communicate loop with gated step rewards and files carried between steps. | T4 expert | End-to-end | multi-metric | 2 gates | 30 min / step | ready |
| Four quarterly cycles with reconciliation of actual fills against the prior plan pf-e2e-year-of-quarters-state4 steps Four quarterly steps where each step delivers a new quarter of data and the agent must reconcile actual fills and dividends against its previously approved plan before rebalancing again. | T4 expert | End-to-end | multi-metric | 1 gate | 25 min / step | ready |