# Multi-agent AI solution generator for a developer platform at scale
**Completed:** 2027
**Scope:** ai-agent-development, gcp-cloud-architecture, ai-powered-coding-assistant, llm-integration
---
A developer platform with millions of monthly visitors needed to compress weeks of API evaluation into minutes. We built a nine-agent AI pipeline that takes a natural language description of a use case, retrieves live documentation through MCP-connected knowledge tools, and delivers a complete working prototype (production-ready code, interactive preview, and implementation guide) in under four minutes.
## Sectors
AI Agents
## Metrics
from prompt to working prototype, live preview, and implementation guide
collapsed evaluation cycle for new developers
reduction in token cost per request
faster evaluation cycles after model tiering
## The Problem
Developer platforms with broad API portfolios face a conversion problem: millions of visitors evaluate the platform each month, but only a fraction convert to active users.
The gap isn't interest, it's friction.
Developers spend weeks reading documentation, comparing APIs, and assembling proof-of-concept code before they can even determine whether the platform meets their needs.
## Sector
API Platform
## Scopes
Multi agent architecture
LLM performance engineering
Google Cloud architecture
MCP tool integration
## Technologies
ADK (Agent Development Kit)
Model Context Protocol
Terraform
TypeScript
Python
## GCP Services
Vertex AI Agent Engine
Cloud Run
Cloud Build
Artifact Registry
BigQuery
Secret Manager
Cloud Logging
## Models
Gemini 2.5 Pro
Gemini 2.5 Flash
## What We Built
A nine-agent AI system that handles the full developer journey from prompt to working prototype.
A developer describes their intent in natural language and the system orchestrates a pipeline of specialized agents: an analysis agent validates feasibility by querying live documentation through MCP tools, a styling pipeline generates and validates visual configuration assets, a code generation pipeline writes, evaluates, and refines production-ready code using extended reasoning capabilities, and a programmatic agent compiles an implementation guide with API documentation links, setup instructions, and pricing guidance. The entire pipeline streams results back to the user in real time, completing in under four minutes.
The output (working code, live preview, and implementation guide) is designed for developers to copy, test, and integrate immediately, reducing the evaluation phase from a multi-week research project to a single conversation.
## Multi-Agent Orchestration
The core innovation is a nine-agent pipeline where each agent has a single responsibility, clear input/output contracts, and a dedicated model assignment optimized for its task.
## Dig deeper: how each agent works
### 01. Root Agent
๐ค gemini-2.5-flash
The orchestrator. Manages session state, conversation routing, and dynamic instruction composition. It delegates to specialized sub-agents based on the user's request and injects context gathered from upstream analysis into downstream agents at runtime.
### 02. Analysis Agent
๐ค gemini-2.5-flash
Validates feasibility before any code is generated. Queries live documentation through MCP tools to determine which APIs and capabilities match the developer's intent, producing a structured analysis that shapes every downstream agent's instructions.
### 03. Style Generator + Schema Validator + Style Refiner
๐ค gemini-2.5-flash + Programmatic
A three-agent styling pipeline. The generator translates visual descriptions into structured configuration (deciding whether to generate, update, preserve, or skip styling). The validator is fully programmatic: deterministic schema validation with zero LLM cost. When validation fails, the refiner receives specific error messages and produces a corrected output. This loop runs up to three iterations.
### 04. Code Generator
๐ค gemini-2.5-pro
The primary development agent and the only one running on the Pro-tier model. Uses extended thinking for complex reasoning, MCP documentation access for API reference, and dynamically injected canonical examples to ensure correct API usage patterns. Outputs are structured against an enforced schema to guarantee consistent, parseable results.
### 05. Code Evaluator + Code Refiner
๐ค gemini-2.5-flash
An adversarial evaluation pair. The evaluator reviews generated code against best practices, documentation accuracy, and security patterns, all backed by MCP documentation access. The refiner performs surgical code changes targeting only the specific issues flagged by the evaluator. This separation prevents the "grading your own homework" problem that emerges when a single agent generates and evaluates its own output.
## Design Decisions in Agent Orchestration
Why nine agents instead of a monolithic prompt
A single prompt attempting all tasks (analysis, styling, code generation, evaluation, documentation) would exceed context windows and produce unreliable outputs. Decomposition allows each agent to have focused instructions, specialized model selection, and independent iteration loops.
Why programmatic agents
The Schema Validator and Documentation Builder are implemented as programmatic agents (no LLM calls). Schema validation is deterministic, no model needed. The documentation builder detects APIs in the final code and maps them to documentation URLs from a canonical registry. This eliminates two potential points of hallucination and reduces token usage to zero for those pipeline stages.
Why separate evaluator and refiner**
**Combining evaluation and correction in one agent leads to confirmation bias: the agent "grades its own homework." Separating them creates genuine adversarial pressure: the evaluator applies strict rules and security scanning, the refiner responds to specific feedback and performs surgical code changes targeting only the flagged issues.
Why enforced output schemas**
**Every agent that produces code or configuration declares a structured output schema. The model is constrained to generate valid output matching the schema, eliminating freeform prose, reducing token waste from verbose responses, and guaranteeing that downstream agents receive parseable inputs without additional transformation or extraction steps. This also makes the pipeline testable: each agent's output can be validated independently against its schema contract.
## How MCP Connects Agents to Knowledge
The Model Context Protocol gives agents a standardized interface to live API documentation. Three agents in the pipeline (Analysis, Code Generator, and Code Evaluator) use it to query an indexed knowledge base at runtime, with results capped per query to control context window growth. The MCP server runs as an isolated service in its own project, so documentation retrieval never competes with agent LLM calls for API quota. The knowledge base can be updated or expanded without touching agent code, which keeps the system responsive to API changes without redeployment.
But MCP alone is not enough. Tool calls are agent initiated, which means the model decides whether to invoke retrieval. A confident but wrong model skips the call and hallucinates plausible API patterns. To prevent this, canonical examples are injected directly into agent instructions at runtime rather than offered as an optional tool. The analysis phase identifies which APIs the developer needs, and only matching examples flow into downstream instructions. MCP handles open ended retrieval where the agent has discretion. Direct injection handles patterns that must be present unconditionally. The combination eliminates stale documentation and a major class of API hallucinations.
## Iterative Performance Engineering
Building a nine-agent pipeline that performs reliably at scale required multiple development iterations. Each version addressed observed production behavior and introduced targeted refinements that maintained output quality while improving system stability.
Engineering Evolution: From Launch to Production Scale
### 01. Phase 1: Monolithic Pro (Launch)
The initial deployment ran the code generator, evaluator, and refiner on gemini-2.5-pro, the most capable model available. All canonical examples were embedded statically into every prompt, regardless of whether they were relevant to the user's request. The system worked, but at ~30,000 tokens per request and 15-20 Pro-tier calls per pipeline execution, cost and latency were unsustainable at scale.
### 02. Phase 2: Dynamic Instruction Injection (~73% token reduction)
Production logging revealed that most requests only used 2-3 APIs from a catalog of dozens, yet every prompt carried examples for all of them. We restructured the pipeline so the analysis phase identifies which APIs the developer needs, and only matching examples are injected into downstream agent instructions. Token consumption dropped from ~30,000 to ~8,000 per request with no impact on output quality.
### 03. Phase 3: Model Tiering (~70% eval latency reduction)
Evaluation data showed that code evaluation and refinement, while critical for quality, don't require the same reasoning depth as initial code generation. We moved all agents except the code generator to gemini-2.5-flash, reducing evaluation cycle time from ~45-60 seconds to ~12-18 seconds. The code generator retained gemini-2.5-pro with extended thinking for complex reasoning tasks.
### 04. Phase 4: Quota Resilience and Self-Healing
As user traffic scaled, API quota errors (429s) became the primary failure mode. Rather than simply increasing quotas, we implemented a three-layer retry strategy and self-healing behaviors (security auto-fix loops and graceful fallback to previous valid solutions). These changes shifted the user experience from "request failed" to "request takes slightly longer."
### 05. Phase 5: Caching and Budget Controls
The final optimization phase focused on eliminating redundant work. Static instruction content (guidelines, rules, security patterns) is now cached across requests, so only user-specific context is processed fresh per session. The code generator's extended thinking phase is explicitly bounded to prevent runaway token consumption on complex prompts, balancing reasoning depth against resource cost.
## Self-Healing Pipeline
A key differentiator of the system is its ability to detect, recover from, and prevent failures at multiple levels without surfacing errors to the end user. The self-healing behavior built into the pipeline (security auto-fix loops, graceful fallback to previous valid outputs) ensures that issues mid-pipeline don't become user-visible errors.
## What Monogram Delivered
PRODUCTION ยท SERVING THOUSANDS OF DAILY USERS
What used to take weeks of manual evaluation now happens in a single conversation. The outcomes below reflect steady state production behavior.
Sub-4-minute prototype generation including complete working code, live preview, and implementation guide, replacing a 2+ week manual evaluation process
~800-1,000 daily users served with consistent quality, achieved through five phases of iterative performance engineering
73% reduction in per-request token consumption through dynamic instruction injection, and ~70% faster evaluation cycles through model tiering, with no measured quality degradation
Self-healing execution with security auto-fix and three-layer retry so users never encounter raw errors or unsafe code
Nine-agent pipeline with clear separation of concerns, each agent independently testable, tunable, and backed by enforced output schemas
Full infrastructure-as-code with Terraform-managed environments (dev/staging/prod), automated CI/CD gates, BigQuery telemetry, and structured observability
## Contact Us
[Build what comes next](https://monogram.io/#contact) | Email: [hello+llm@monogram.io](mailto:hello+llm@monogram.io)
Provide your contact details and claim a time to meet. We'll help you do the rest.
Monogram is an Applied AI Studio based in Atlanta that designs and deploys production AI systems โ connecting models, data, workflows, and enterprise software to automate real work.
The contact form collects: name, email, company, estimated budget (USD), a project description, and how you heard about Monogram.
Not sure where to start? Tell us what you're trying to automate or build โ we'll help you figure out where AI fits.
---
**More content:** [Home](/.md) | [About](/about.md) | [Blog](/blog.md) | [Engineering](/engineering.md)
Monogram helps companies, organizations, and firms adopt AI by designing and building proof of concepts (POCs), MVPs, intelligent workflows, and production-ready AI applications.
Website designed and developed by [Monogram](https://monogram.io)