Skip to main content
Do your subagents improve the final result, or only add work? Evaluate the whole task, including supervision, handoffs, retries, and final synthesis. Ten successful children do not count as ten successful tasks.
This local workflow requires a build containing the delegation evidence extension. The fixture is synthetic and makes no model calls. It does not demonstrate that any strategy improves a real model, automatically observe private subagents in a chat, or certify a provider’s native fork behavior.

​
Choose the question before the strategy

Keep roles separate from context delivery and experimental treatments: Compare the same frozen tasks across four treatments:
  1. No delegation: the root does the work and explicitly closes an empty fan-out.
  2. Isolated delegation: children receive complete instructions and necessary evidence.
  3. Inherited-context delegation: use only when the runtime supports and reports inheritance.
  4. Compact handoff: an isolated child receives a bounded summary and necessary artifacts.
isolated and fork are the native modes described in LangChain’s context article. Compact handoff is a treatment, not another native mode. A requested fork is not evidence that conversation inheritance occurred.

​
Run the deterministic recipe

Build and install a package containing this extension into a temporary consumer directory. The repository’s examples/evaluate-subagents/README.md gives the tarball and wheel build steps. With the TypeScript package installed there:
Use --python /path/to/venv/bin/python instead of --cli for an installed Python wheel. The generator uses Node for identical fixture inputs; actual ingestion and reporting use the selected package’s CLI. Each run creates frame.json, metrics-only observations, CLI receipts, and report.json. The complete fixture has 12 expected root trials and nine child observations. The missing-child case omits three children without shrinking the expected frame. Other scenarios exercise failed roots despite passing children, lost constraints, biased reviews and stale handoffs. For an explicit metadata hook, submit one record on stdin and analyze it against the independently frozen frame:
The outer metrics schema stays at version 1; the optional delegation extension has its own version 1. Existing flat sessions remain valid but do not become proven root tasks. Inspect delegationAnalysis.rootTrials, treatments, comparisons, and decisionBindingHash. A complete descriptive comparison still has eligible: false; it does not fabricate an inferential interval. For the synthetic complete control, the report shows: These are fixed fixture counters, not measured model economics. Isolated and compact comparisons contain three matched root trials across three tasks; the inherited comparison is inconclusive because native context attribution is unverified. Every comparison has eligible: false and interval: null. The missing-child fixture retains all 12 expected roots, marks three trees incomplete, and withholds their total-cost and savings claims. The separate synthetic-evaluations.json file records synthetic protected-constraint retention, reviewer defect detection/false positives and current-artifact checks.

​
Preserve the task boundary

​
Minimal parent and child

The runnable fixture contains a parent with a closed fan-out issuing one required delegationId, and a child with that same edge ID and parentSessionId. Both bind the same task version, root execution, variant and logical trial. The child has its own session and attempt IDs. To inspect the smallest complete tree, run this in the repository root:
This yields one root trial and two physical attempts, with no matched baseline comparison. Keep the complete four-treatment frame for comparing strategies. Freeze the task versions and intended root executions before collecting observations. Record issued child work before waiting for its result. Close each parent’s fan-out explicitly when no more children will be issued. Otherwise a missing child—or an entirely missing root—could disappear from the comparison. The root’s final evaluated outcome determines task quality. Required child work must be resolved for the evidence to be complete. A cancelled or missing child is different from a completed child with an attributable behavioral failure. Optional work must be declared by the issuer; a missing result cannot retroactively declare itself optional. Record physical retries separately and deduplicate retransmitted observations. Never sum repeated cumulative snapshots. Repetitions of one task and child attempts do not increase the number of independent tasks.

​
Interpret cost and latency

Exclusive session usage can be added across the task tree. Inclusive root totals already contain descendant usage and must not be added to it again. Include unsuccessful attempts, tools, context preparation and final synthesis in the reported coverage. If the exporter cannot account for a component, mark it unknown rather than zero. Known subtotal and complete total are different results. Cached input tokens are not automatically free. Root elapsed time is distinct from the sum of parallel child durations. Cost per successful task includes unsuccessful work in the intended cohort and is undefined when no verified root succeeds.

​
Capture only what your runtime exposes

Use an explicit hook or reviewed export. Do not read private session stores or scrape transcripts to fill missing fields. Keep IDs bounded and opaque; never put prompts, answers, tool arguments, source snapshots, credentials or private paths in metrics JSONL. Codex, Claude Code, Cursor and CoWork hook names do not guarantee that any of them exports every delegation field. A generic metadata export can exercise the contract without pretending to be a native provider integration. Conversation isolation also does not isolate shared files, tools, caches or external state. An isolated reviewer can still encounter a biased shared note. The existing harness record hook also accepts a bounded, hook-only traceLineage: { traceId, spanId, parentSpanId? } projection from an existing WorkflowTracer or OpenTelemetry span. It maps to rootExecutionId, sessionId and parentSessionId respectively and rejects conflicting supplied identities. Only those IDs are accepted: do not pass the whole span or its metadata. The exporter must separately supply the required delegation contract, issued fan-out, attempts, actual context observations and metrics. This mapping does not infer outcome, inheritance, role, or cost coverage from a span. If a session ID is reused across supervisor retries, provide parentAttemptId when the same delegation edge is present in more than one parent attempt. Ambiguous parentage is invalid evidence. Multiple inclusive root attempts yield an observed resource lower bound, not a verified total; their end-to-end elapsed time remains unknown unless an execution-wide measurement is available.

​
Design a fair live experiment

Keep models/settings, tools/permissions, evaluator identity, task versions and budget policy fixed. Record intended scheduling and fan-out differences. Interleave or randomize treatments and describe warm/cold cache conditions. Do not give one arm privileged requirements or leak held-out answers into any arm’s inputs. Score protected-constraint retention, final correctness, reviewer true and false positives, and evidence completeness before efficiency. Agreement with the supervisor is not reviewer quality. Measure duplicate investigation only when the runtime directly exposes it. A compact handoff should include the objective, protected constraints, verified facts, artifact versions, unresolved questions and completion criteria. Keep its actual text in your own permitted workspace. The metrics channel carries only its permitted reference and observed counters. A stale reference or a dropped requirement still needs an actual task assertion; hash equality does not certify semantic correctness. Local descriptive comparisons are exploratory. They do not grant approval, publish a winner, or change your delegation configuration. Missing evidence, unknown resource coverage, hard failures and unverified context claims must remain visible even when an observed subtotal looks cheaper. For existing instrumentation, see TypeScript workflows, Python workflows, and multi-agent trajectories.