agents-swarm-orchestration

Swarm Orchestration

Advanced execution layer for multi-worker runs after agent or team selection.

Coordinate multiple workers without polluting the main thread. Use this skill after agents-subagents has already selected the right agent, member, team, or debate pattern. This skill is for choosing the orchestration surface, freezing task ownership before fan-out, and requiring structured outputs that the lead agent can validate and merge safely.

Terminology (Aug 2026)

"Swarm" is community vocabulary — it appears nowhere in Anthropic documentation. Use the official primitive names when writing configs, prompts, or docs; keep "swarm" only as informal shorthand for the whole category.

Quick Reference

Navigation

• references/loop-orchestration.md - Bounded iteration vs retry, loop-until-dry, convergence detection, termination predicates, dedup-target rule
• ../ai-coding-agents-tasks/references/loop-and-graph-runtime-surfaces.md - Loop Engineering and Graph Engineering runtime comparison: task queues, cyclic graphs, and workflows
• references/scripted-workflows.md - Script-held deterministic control flow (Claude Code Workflows): agent/parallel/pipeline, barrier-vs-pipeline, resume and caching
• references/platform-patterns.md - Platform guidance for Claude Code subagents, Codex subagents, Codex multi-agents, and OpenAI Agents SDK
• references/output-contracts.md - Task schema, worker report schema, and merge contract
• references/operational-guardrails.md - Safety, stop conditions, observability, and verification gates
• references/cost-discipline.md - Fan-out cost patterns, session lifecycle, loops/schedules audit, orchestration-layer config
• references/orchestration-maintenance-runbook.md - How to audit, maintain, and refresh swarm discipline over time
• references/runtime-smoke-tests.md - 3 shell-runnable checks for wave dispatch, per-worker budget breach, and wave-boundary checkpoints
• references/execution-surfaces.md - Single thread, worker fan-out, agent team, manager, and handoff selection
• references/noninteractive-and-blueprints.md - CI-safe dispatch patterns and deterministic-plus-agentic blueprint flows
• references/recipe-wave-dispatch.md - Self-contained 3-worker shell example: copy, paste, run, verify
• references/typical-scenarios.md - Scenario library: common jobs mapped to surface, pattern, worker shape, and the trap to avoid
• agents-subagents - Subagent design, tool scoping, and interruption recovery
• ../agents-hooks/SKILL.md - Hook guardrails and verification automation
• ../agents-mcp/SKILL.md - MCP server scoping for workers
• ../agents-skills/SKILL.md - Skill packaging for worker preloads
• ../agents-memory/SKILL.md - Project memory for shared conventions
• ../ai-coding-agents-permissions/SKILL.md - Approval routing, allow or ask modes, and worker permission handoff
• ../ai-coding-agents-tasks/SKILL.md - Background task runtimes, teammate queues, and task ownership
• ../dev-workflow-planning/SKILL.md - Create the plan before fan-out
• ../ai-agents/references/autonomous-loop-patterns.md - Shape C autonomous loops: PRD-driven drivers, circuit breakers, drift detection (framework-neutral)
• ../ai-agents/references/context-graph-patterns.md - Graph-structured agent state: node/edge schema, traversal, conflict resolution
• data/sources.json - Curated official docs, research, and secondary references

Maintainer note: eight URLs here are intentionally duplicated from ../agents-subagents/data/sources.json (Claude Code subagents, Agent Teams, Codex Multi-Agents, Codex Subagents, both OpenAI Agents SDK pages, OpenAI prompt-caching guide, Karpathy coding notes). Each skill frames those sources for a different reader. When a URL rotates, update both files in the same commit.

When To Use / Not To Use

Relationship To Agents-Subagents

agents-subagents is the entry point. It selects the mode and prepares the first launch prompt. This skill takes over when the plan needs multi-wave execution, worker dependencies, verifier passes, or merge/conflict control.

Operating Principles

• Lead owns requirements, decisions, approvals, and final synthesis — not execution.
• Default to read-heavy parallelism; parallel writes are higher-risk.
• Freeze shared interfaces before dispatching edit-capable workers.
• Give every worker exclusive owned_files and explicit do_not_touch boundaries.
• Pass distilled dependency outputs, not raw logs or long transcripts.
• Require structured worker reports — the lead validates and merges deterministically.
• Re-plan when conflict resolution costs more than the fan-out saved.
• Fresh context per worker: each worker brief contains only its task, plan section, file ownership, and interface contracts — not the lead's full history. Prevents context rot; gives each worker a full window.
• State in files: task graph, progress, decisions, and dependency outputs live in structured files (frontmatter MD / JSON / YAML). Any new lead session resumes by reading files, not memory.
• Checkpoint long runs: snapshot task state, reports, and decisions to checkpoints/ at each wave boundary.
• Budget per worker: explicit token/time/tool caps at dispatch. Budget-conservation invariant: child budgets are strict subsets of the parent's remaining budget. Workers that breach their budget stop and escalate — they do not continue. (Ye & Tan, Agent Contracts: A Formal Framework for Resource-Bounded Autonomous AI Systems, arXiv:2601.08815, 2026)
• Telemetry per worker: assign a run id or span id; log inputs, outputs, status, tokens, and duration to one structured location.
• Durable approval channels: route approvals through mailbox/poller with request IDs, not ephemeral callbacks.
• Minimum toolset per worker: use tools, disallowedTools, and skills fields to give each worker only what it needs.
• Memory opt-in: prefer clean-context workers + file-backed checkpoints. Enable memory only when the role genuinely benefits from cross-run priors; never default it for verifiers or reviewers. Prefer file tools over schema-constrained memory APIs. (Lance Martin, 2026-04-24; ../ai-context-layer/references/filesystem-as-memory.md)

For context rotation and state handoff patterns, see ../ai-agents/references/context-rotation-and-state.md.

Explicit Fan-Out Is The Durable Default

Claude Opus 4.7 (GA 2026-04-16) shipped a lasting behavior change: it spawns fewer subagents by default than 4.6, favoring single-response completion over implicit parallelism. Fan-out workflows that previously worked without being asked — read-heavy scans, multi-file refactors, review waves, cross-repo audits — now silently serialize unless the lead is told to fan out explicitly. Opus 4.8 (current as of this writing) inherits the same conservative default; treat "assume no auto-parallelism" as the standing assumption for whatever frontier model is current, and re-verify against release notes each time the lead model changes.

Anthropic's source guidance is to give the model explicit fan-out instructions; it does not prescribe where the instruction must live. Our repo convention is to install the canonical phrasing once in AGENTS.md / CLAUDE.md (not duplicated per launch prompt):

Spawn multiple subagents in the same turn when fanning out across items or reading multiple files. Do not spawn a subagent for work you can complete in a single response.

Full guidance and source links live in agents-subagents. Judgment call for the lead: after any model swap, run one throwaway fan-out task and watch whether it parallelizes on its own — cheaper than discovering silent serialization mid-migration.

Named Patterns

Name the pattern explicitly when proposing a design. Full detail: ../agents-subagents/references/harness-patterns.md.

Typical Scenarios

Each common job maps to one dispatch shape. Load references/typical-scenarios.md for the full table (surface + pattern, worker count/tiering, waves, Claude Code vs Codex mapping, key trap), three deep walkthroughs, and a do-not-swarm list.

Orchestration Choice

Use the simplest surface that preserves ownership and coordination:

• single thread when the work is small or the interfaces are still unstable
• isolated workers when the lead only needs results back
• Claude Code agent teams when workers must talk to each other directly
• manager vs handoff depending on whether the lead keeps user ownership

Load references/execution-surfaces.md when you need:

• the detailed single-thread vs worker vs team decision
• Claude team communication patterns
• task-list and SendMessage coordination rules
• manager vs handoff guidance for OpenAI-style systems

Framework quick-pick (June 2026):

Verify current GA status before committing to a framework — this space rotated significantly in early 2026. (uvik.net/blog/agentic-ai-frameworks, June 2026)

Pre-Dispatch: Collaborative Debate

Before fan-out on high-complexity work, run a collaborative debate step: 2–3 specialized personas (e.g., architect + developer + QA) argue tradeoffs in one session before interfaces freeze. Output is a decision log that becomes part of each worker brief. Reduces mid-execution rework from conflicting assumptions.

Use when: architecture affects multiple workers; tradeoffs are unclear; early disagreement is cheaper than late integration failure. Skip for routine parallel work with stable interfaces.

• Templates: ../agents-subagents/assets/templates/
• Full pattern (Claude Code, Codex, Agent Teams): ../agents-subagents/references/agent-patterns.md §"Pattern 5: Debate Team"
• Step-by-step setup: ../agents-subagents/references/debate-quickstart.md
• Wider method landscape: ../ai-agents/references/agent-delivery-methods.md

Dispatch Workflow

1. Build a dependency-aware task graph before launching anything.
2. Freeze interfaces, ownership, and verifier commands for each task.
3. Launch only unblocked tasks; use waves unless the work is intentionally read-heavy and low-risk.
4. Cap edit-capable workers at 3 by default. Increase fan-out only for read-only scans, review, tests, or summarization.
5. Require each worker to return a structured report instead of raw intermediate output.
6. Validate the report, verification evidence, and changed files before marking the task complete.
7. Merge one worker result at a time, then unblock the next wave.
8. Stop and re-plan when conflicts or retries show the current graph is wrong.

Minimal worker brief template (paste into subagent system prompt or TOML developer_instructions):

TASK: <one-sentence objective>
OWNED FILES: <exact paths — edit only these>
DO NOT TOUCH: <paths explicitly off-limits>
READ ONLY: <dependency outputs or context files>
DELIVERABLE: <what you return — format and path>
VERIFICATION: <command to run before reporting done>
BUDGET: tokens=<N>, time=<Ns>, tool_calls=<N>
SELF-REJECT IF: <named negative criterion>

ASCII Flow

Multi-agent work
  -> Build task graph
  -> Freeze interfaces, ownership, and verifier commands
  -> Dispatch wave
     +-- unblocked read-only tasks -> broad fan-out allowed
     +-- edit-capable tasks        -> cap at 3 by default
     +-- blocked tasks             -> wait for dependency output
  -> Require structured worker reports
  -> Validate evidence and merge one result at a time
  -> Re-plan when conflicts or retries show the graph is wrong

For CI-safe dispatch, batch fan-out, and blueprint-style deterministic-plus-agentic flows, load references/noninteractive-and-blueprints.md.

Lead Agent Responsibilities

• Maintain task state: pending, in_progress, completed, blocked, failed.
• Own approvals, permissions, and escalation for risky operations.
• Keep the canonical task graph and dependency outputs.
• Reject reports that do not match the expected schema or ownership.
• Run integration verification after merging worker outputs.
• Synthesize the final answer only after the merged state passes validation.

Model Guidance

Tiering saves ~40% vs all-Opus teams with minimal capability loss on worker tasks. (cloudzero.com/blog/claude-code-agents, 2026)

3 edit-capable worker cap applies to agents sharing a branch — tracks context-window contention and super-linear merge cost. Worktree isolation relaxes this for read-only workers but does not remove coordination overhead.

Background mode (Claude Code): Subagents run in the background by default since ~July 2026 (v2.1.195+) — background is no longer the opt-in. Pin edit-capable workers to the foreground (background: false) so permission prompts pass through; leave read-only scans and audits on the default. Mix: edit wave foreground, read-only wave background.

Use exact model names from references/platform-patterns.md — catalogs change faster than orchestration patterns.

Escalation Over Retry

On task failure, escalate structurally — do not loop:

1. Self-fix — worker re-plans and retries once with a different approach.
2. Escalate to lead — worker reports failure + diagnosis; lead reassigns, re-scopes, or continues.
3. Escalate to human — lead flags as outside agent authority (safety issue, ambiguous requirements, destructive operation).

Retry the same approach at most once. Recurring failure is structural, not transient.

This governs failure handling, not iteration. Bounded iteration — where each pass succeeds but surfaces the next pass's input — is a separate, legitimate regime with its own termination discipline. The test: if a second pass would consume different input than the first, it is iteration, not retry. Unknown-extent discovery (bug hunts, dead-code sweeps, dependency chasing) should loop until convergence, not stop after one pass. See references/loop-orchestration.md for loop shapes, termination predicates, and the dedup-target rule.

Progressive tool loading: Start workers with a minimal tools list; expand only when the worker signals it needs more. Pass tools explicitly in the dispatch contract.

Worker Self-Rejection Rules

Most worker failures are plausible-but-wrong output reaching the lead unchallenged. Embed a self-rejection clause in the worker's system prompt to pre-filter before the lead sees it:

"Reject your own draft if <specific named condition>."

The condition must be named and observable — not "if the draft is bad."

Strong examples:

• "Reject if the success metric is a vanity metric instead of an action."
• "Reject if no buying signal has a dated source."
• "Reject any 'edge case' that is just 'what if input is null' without a specific scenario."

Rules:

• Clause belongs in the system prompt (worker invariant), not the dispatch brief.
• Cap at 2–3 clauses per worker — compliance drops past that.
• Self-rejection does not replace lead-side schema validation; it pre-filters common failures.
• A worker that rejects itself N times has the same budget-breach behavior as any other breach.

Full tradeoff discussion and example catalog: references/operational-guardrails.md §Worker Self-Rejection.

Source: Nav Toor — 30 Claude Code Sub-Agents I Actually Use (2026-05-01).

Common Anti-Patterns

Known Traps

• Swarm-before-checking: confirm one lead + one verifier is insufficient first
• Fan-out before freeze: interfaces, dependencies, and ownership must be frozen first
• Worker count outpacing checkpoint, telemetry, and merge capacity
• Background workers with no budget and no stop condition
• Shared-branch edit waves where worktree isolation is the safer default
• Context rot: worker context >70% full — quality degrades silently; force handoff or checkpoint
• Approval fatigue: many prompts train blind approval; pre-approve narrow scopes at launch
• Stale agent files after runtime upgrade: audit worker definitions after each Claude Code or Codex bump (field set drifted across 2026: effort, initialPrompt, skills, memory)
• No cumulative cost circuit-breaker: per-worker caps insufficient; define a run-level cap that halts new dispatch
• Reasoning fan-out at equal budget: message-passing loses mutual information vs. one strong model on full context (Data Processing Inequality; Tran & Kiela, Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets, arXiv:2604.02460, Apr 2026 — not peer-reviewed, scope limited to multi-hop reasoning)

Validation Checklist

• The orchestration surface matches the communication pattern: single thread, worker fan-out, agent team, manager, or handoff.
• Every task has explicit dependencies, ownership, deliverable, verification, and risk level.
• Edit-capable workers have exclusive files and clear do_not_touch boundaries.
• Dependency outputs are distilled and structured before reuse.
• Worker reports match the expected schema.
• Verification runs at both worker level and merged-system level.
• Stop conditions and escalation rules are defined before launch.
• Per-worker budgets (tokens, time, tool calls) are set at dispatch.
• Structured telemetry is in place (run id or span id per worker).
• Checkpoint cadence is defined for runs expected to span multiple waves.
• Self-rejection clause embedded in each worker's system prompt (≤3 named criteria).
• Emergency-stop and rollback path pre-declared for partial-wave failure.
• Model tiers assigned: strong reasoning for lead, balanced for edit workers, fast for read-only and verifiers.

Maintenance

• Use references/orchestration-maintenance-runbook.md when reviewing whether swarms are still justified, whether worker counts drifted up, or whether platform updates changed the right execution surface.
• Treat references/cost-discipline.md as the tactical cost note and the maintenance runbook as the durable operating guide.
• Keep execution-surface rules and maintenance rules aligned. If the team starts using a new default surface, update both.

Fact-Checking

Originally inspired by the Codex swarm playbook (am.will / LLMJunky); updated against primary platform docs. Adds per-worker budgets, structured telemetry, checkpoint/resume, and a named-patterns vocabulary. Last freshness pass: August 2026 — vocabulary aligned to the official primitives (see §Terminology), and three stale claims corrected: subagents are background by default (~2026-07, v2.1.195+), subagents can nest since June 2026 (chain cap 5), and cross-session messaging (Aug 2026) is now a distinct surface from agent teams.

Platform behavior, model names, permissions, and experimental flags change frequently — verify against official docs before final answers. Mark platform-specific guidance as unverified when web access is unavailable.

Known-stale-risk items re-checked in the July 2026 pass: the Agent Teams lead-model floor (originally Opus 4.6+; current docs describe a configurable default teammate model instead — verify before assuming a hard gate still applies), and the assumption that Opus 4.7's conservative fan-out default is model-specific rather than a durable behavior carried into later frontier models (treat it as durable and re-check on every model swap; see §Explicit Fan-Out Is The Durable Default).

Learnings Loop

Before applying this skill on a non-trivial task, read learnings.consolidated.md in this directory (and learnings.md if present).

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.