Lessons: what breaks when agents brainstorm

Five failure modes we hit running multi-agent brainstorming in practice — fake consensus, hallucinated sources, template repetition, scoring drift and the originality tax — and the concrete fix for each.

TL;DR — Running multi-agent brainstorming in practice surfaces five failure modes. None is fatal, but each has a specific fix. Honest failure notes beat optimistic demos.

The lesson behind this page

These are observations from running structured, multi-round brainstorming with AI agents on real projects. The value is not the success stories — it is the failure modes, because they recur.

The five failure modes

1. Fake consensus

What happens: every participant produces a variation on the same idea, and the "convergence" looks like strong agreement.

Fix: keep round 1 blind (see two-round brainstorm) and score originality relative to the group, so copies do not score as high as the original.

2. Hallucinated sources

What happens: an idea cites a statistic, a study or a competitor that does not exist.

Fix: require a source id for every factual claim. The Overview stage rejects claims without provenance. Unsupported claims become visibly unsupported.

3. Template repetition

What happens: the same structure ("X for Y, powered by Z") fills the whole set. The ideas differ in nouns, not in kind.

Fix: score divergence as its own dimension, and explicitly penalise structural repeats.

4. Scoring drift

What happens: different reviewers apply the same scale differently, so scores are not comparable across a set.

Fix: a fixed rubric with a written justification per score, and compute what can be computed (depth, for example) instead of asking for it.

5. The originality tax

What happens: genuinely new ideas score low at first because nobody recognizes their value. Novelty and perceived quality are not the same thing.

Fix: keep originality and feasibility as separate dimensions, and let a human make the final call on the shortlist — do not let the average score bury the outlier.

The loop that operationalises these fixes is DBOW. See What is Aidbow?.