When AI Collaboration Breaks
In Part 1, I shared how a dusty screenplay and a spark of an idea led me to build an AI-powered writers’ room. The Claude-based prototype worked, so I built it into a real application, this time using Claude for probabilistic functions, and Gemini for deterministic.
That's when things got interesting.
Orchestrating two LLMs and getting multiple AI personas to collaborate productively - without stepping on each other, repeating themselves, or drowning the user in noise - was like trying to get a room of hyper kindergarteners to line up for recess.
I developed 50+ personas in my writers room: showrunners, character specialists, dialogue experts, world-builders, story structure gurus, narrative specialists. Each with distinct voices, creative philosophies, influences, and expertise.
And they would go rogue. Again and again. It made me crazy.
The Question Avalanche (When Helpful Isn’t Useful)
Early on, I'd send a message to the room and get three personas responding, each asking their own clarifying question:
"What is the inciting incident that starts the protagonist on his path?"
"How does the cold open set up the related scene in act 3?"
"The mental state of the antagonist lands weak given his trajectory - how can we spice this up?"
Many times their questions (Claude function) could be answered if they had referenced the materials I provided in the app (Gemini function). Sometimes they would each ask the same question in different ways. In concept, the room was working. But the experience? Frustrating and overwhelming. I felt like I was being interrogated by Aaron Sorkin.
The fix: I built robust persona instructions and coordination protocols. Personas generate responses sequentially, not in parallel. Each persona was required to review what the other said before responding. I established the showrunner persona as the lead to coordinate questions. And I wrote an explicit rule: if a question is already asked, don’t ask the same question. Stay silent. Not rocket science.
Silence as a valid response. That was counterintuitive. Most AI’s are trained to always be helpful, to always contribute. But a good writers room has people who know when to contribute and when to not pile on.
When Personas Drift
The personas would get confused.
I’d define a showrunner persona with a specific voice: direct, craft-focused, allergic to hand-holding. A few exchanges later and they'd start sounding like a generic helpful assistant. Encouraging. Supportive. Sycophantic. Highly annoying.
The "be helpful" training in these models fights against persona constraints. The AI wanted to please. That's exactly what you don't want in a creative collaborator.
The fix: Behavioral overlays injected into every prompt. Explicit rules: "Do not praise unless the work genuinely earns it. Push back on weak ideas. Respect the user's skill level - if they're advanced, be telegraphic, not tutorial."
I had to constrain the AI's helpfulness to make it actually helpful.
Teaching AI When to Shutup
Personas would get creative. In the wrong ways.
They’d step outside their assigned “lanes,” not following their roles in the overall process. They'd volunteer ideas I didn't ask for. The wrong persona would draft an outline in the chat instead of the designated drafting section.
This sounds minor, but it's not. The whole point of the room was human-in-the-loop collaboration. The AI proposes, the human considers and outputs. When personas start acting autonomously, the user loses control of their own creative process. I would be pulled down side-streets. I was drowning in unproductive ideas.
The fix: Explicit structural rules. "The “Drafter” drafts, the “Advisor” advises. Draft content goes in the draft section, not the chat. Don't offer creative ideas unless asked. Your job is to help the user realize their vision, not to have one of your own."
The Lesson: Constraint > Capability
Here's what building this has taught me:
The hard part of AI isn't getting it to do more. It's getting it to do less. And in many ways, it’s organizing ideas and inputs, and it’s about preventing chaos.
- Less unsolicited creativity
- Less sycophantic encouragement
- Less talking when silence is the right response
- Less "helpful" and more useful
Most AI development focuses on expanding capability. What else can it do? How much more can it generate? But in collaborative contexts (creative or otherwise) I’ve discovered the value comes from productive constraints.
A good collaborator knows their role. They know when to push and when to hold back. They know when their idea or words or challenge is a contribution, not a block. They serve the process.
Teaching AI to do that? Way harder than teaching it to write dialog.