Once you have AI PR review on every pull request, the next question is not “should a bot look?” It is which loop you run: one cheap pass on every push, a fleet of specialists on the first head, or something that gets expensive because you re-fired everyone on commit four.
This is a playbook for those loops. The worked example is Mergestorm’s Vortex fleet — not a product recap. For the definition of the job, see What is AI PR code review?.
The problem a loop has to solve
A single generalist is cheap and honest. It also has one viewpoint. Specialists (security, tests, API, …) catch classes that a generalist skims. Fan them out on every push and you pay three taxes:
- Credits. Each extra lane is usually another metered pass.
- Latency. Parallel hire helps, but a slow specialist still holds “review done.”
- Noise. Eight comments that say the same thing teach the team to ignore the bot.
A good loop spends coverage on the first look, then spends less on follow-ups unless the diff still warrants it.
Strategy 1 — Fire one agent
Core only. Every PR, every push. On Mergestorm that is router off: one credit for an initial Vortex pass, no specialist lanes.
Use when the repo is small, the team is new to AI review, or you want a floor that never surprises billing. Con: you will miss domain bugs a specialist is built to want (an exploit path, a wire-format break, a missing assertion). Pair with the human checklist so those still get eyes.
Strategy 2 — Hire specialists on the first review
First published review assembles a fleet. Lanes fan out in parallel on the same head, so adding security should not mean waiting for security after Core finishes.
Mergestorm’s pinnable lanes (as of this writing):
- Security — realistic exploit in changed code
- Performance — a workload you can point at
- Architecture — a structural defect this PR made worse
- Tests — the missing assertion on the new branch
- Data — migration, nullability, persisted meaning
- API — wire-format break, not an internal rename
- Frontend — concrete UI failure (loops, hydration,
as any)
Intensity knobs: standard (Core + up to 3 specialists), max (Core + up to 5), or manual pins. Empty findings from a lane are a success. Specialists should not be paid to talk.
Pro: coverage without a serial queue. Con: first-review credits jump (Core 1 + 1 per hired lane). Do not default to max on a repo that opens twenty PRs a day until you have seen the comment quality.
Strategy 3 — Sticky hire on follow-ups
The expensive habit is “run the same five specialists on every push.” The cheaper habit is sticky hire: on the next head, re-invoke only the lanes that actually found something, so they can verify the fix. Lanes that came back clean retire. New risk that appears later still needs a generalist (Core) looking at the new slice.
That is how Vortex follow-ups are built: history remembers each lane’s last outcome; dirty lanes come back; all-clear lanes do not. You never re-hire more than you hired on the first review.
Pro: follow-up cost tracks unfinished work, not fleet size. Con: if a new class of bug shows up only on commit three, a retired specialist will not see it — Core (or an orthogonal hire) has to.
Strategy 4 — Fuzz with orthogonal specialists
“Fuzz” here does not mean random extra nits. It means asking a different question than the code lanes already asked.
Re-rolling security + tests + API on every push is not fuzzing. It is paying for the same viewpoint again. An orthogonal hire looks at trajectory: did this PR stop matching the original ask? Is it growing instead of converging?
Mergestorm’s version is Governance — not pinnable, not on the first review. It shows up on the third follow-up round and later. It does not re-review the diff. It posts one verdict on the path (converge vs continue). That is the right shape for an orthogonal pass: rare, cheap relative to another full fleet, and aimed at PRs that will not settle.
If you roll your own loop: pick one orthogonal question (scope drift, missing tests vs the description, “should this be three PRs?”) and fire it on long-lived branches — not on the first three-line typo fix.
Initial review vs follow-up
Keep the two jobs distinct:
- Initial — assemble coverage. Hire the fleet once. Synthesize so you get one review, not eight stacked comments. (A synthesizer that may merge or drop, but must not invent findings, is the difference between a fleet and a pile.)
- Follow-up — verify what was dirty, let Core watch for new classes, escalate orthogonally only when the PR is still alive several heads later.
Deep review is a third job. On Mergestorm that is Tempest (risky diffs), not another Vortex lane. Cyclone pushes mechanical patches. Maelstrom is a plan name, not an agent — do not put it in the loop diagram.
Pros and cons of multiple agents on one PR
| Loop | Pays off when | Hurts when |
|---|---|---|
| Core only | You need a floor on every push | The bug is domain-shaped |
| First-review fleet | The first head is the risky one | You open many tiny PRs |
| Sticky follow-ups | Authors iterate on the same findings | The change set mutates into a new problem |
| Orthogonal / governance | The PR will not die | You fire it on round 0 |
| Everything, every push | Almost never | Credits, latency, and muted Slack |
Multiple agents in parallel on one head usually do not add their latencies. Multiple agents required before merge do add wall-clock if you wait for the slowest. Optimize the trigger: full fleet on open (or on “ready for review”), sticky on synchronize, orthogonal only after N heads, Tempest only when the diff is actually risky.
A loop you can steal
- Default: one Core pass on every PR so nothing merges unread.
- First review on non-trivial PRs: hire 2–3 specialists that match the repo (API shop → API + tests; product UI → frontend + security).
- Follow-ups: sticky re-hire only. Do not re-roll the roster.
- Long-lived PRs: one orthogonal look at the path, not another security pass.
- Keep a human on approve — loops catch chores, not product judgment.
If you want that loop without assembling it from scratch,
Mergestorm’s Vortex settings are the same four modes
(off / standard / max /
manual), plus sticky follow-ups and a late governance
pass. We are an affiliate; the playbook still holds if you wire
specialists some other way.