How we built a cross-domain AI agent with 24 tools
The architecture, the prompt engineering, the eval framework. Real numbers from our beta: acceptance rate, token usage, latency. What worked, what didn't.
The agent has 24 tools. The temptation was to wire them all into one model with one mega-prompt. We tried that. It failed.
What works: a coordinator agent that dispatches to specialist agents per domain (tasks, time, finance, hr). Each specialist has 4–6 tools and a focused prompt. The coordinator never executes; it only routes.
Latency p50 is 1.8s. Token cost averages $0.012 per agent action. Eval suite covers 312 scenarios and runs on every prompt version change.
Lessons: small focused prompts beat big ones, structured outputs beat free-form, and human-in-the-loop on writes is non-negotiable.