Overview
GPU lane (4 slots), CPU lane (2 slots), cloud on call. Hardware refreshes every few seconds.
Benchmark progress
Loaded models
Lane caps
Latest findings
- Waiting for the current results checkpoint.
Benchmarks
What each test checks, how much is finished, and what the results support.
What this was, and what it settled
TheAgentCompany experiment · 12–13 Oct 2026 · closedWe ran 19 "arms" (one model, or a formation of models) through the same 10 simulated office tasks: spreadsheets, PDFs, SQL, HR sorting, project tickets, graded by the benchmark's own checkers. Each arm is a way of organising the AIs, not a product.
- Cloud beats local by a wide margin: Codex alone 45/52; the best local setup 26/52.
- The orchestrator makes no difference: the same platoon scored 24/52 under our harness and under n8n. Points come from the workers, not the wiring.
- Local models fail at discovery, not at the work: every local arm scored 0 on the spreadsheet, Plane-issue and résumé tasks (logins, navigation, scanned documents). That is why the plan is cloud plans, local executes.
Still relevant as evidence; not being extended. The rows marked "running" below are old runs that stopped. The current programme is under Latest progress.
TAC score by arm
Official TheAgentCompany graders · 10 tasks · 52 checkpoints. Partial rows are incomplete saved runs, not evidence of an active job.
Points per task
Earlier exploratory 3D formation runs. Older checker versions and output selection limit comparisons. See Latest progress for the saved-scene recheck; this is not a current winner ranking.
Formation tournament · 3D modelling
Score out of 45 · same tasks for every structureEarlier unit screens. Published reference numbers used different samples and conditions; these are not matched head-to-head comparisons.
Unit qualification tests
Market
305 products mapped and checked against live pages on 13 Oct 2026.
Rainbow Regiment is the organisation layer for a local-first AI company. It turns your own GPU box and a few cloud models into ranked, benchmarked units you command from one chat.
Where we fit
| Layer | Market state | Main players | Our move |
|---|---|---|---|
| Organisation | Thin and weak | Paperclip, Multica, MetaGPT, ChatDev, Composio, CrewAI AMP, Claude Managed Agents | This is the product |
| Memory | Vendor claims unreliable | Mem0, Letta, Zep, Cognee | Logbook from checker-verified runs |
| Worker harnesses | Strong, mostly open | OpenHands, Codex CLI, Claude Code, OpenCode, Goose | Plug in; main lever on scores |
| Agent frameworks | Free | Pydantic AI, MS Agent Framework, LangGraph | Use as parts |
| Durable execution | Free, proven here | DBOS, Hatchet, Temporal, Conductor | Build on DBOS |
| Triggers | Commodity | Zapier, Make, n8n, Activepieces | Use one |
Lane drill · hands-on
8-job plan, GPU 4 / CPU 2 caps across two runs, killed mid-run| Tool | Layer | Caps across runs | Crash test | Verdict |
|---|
The gap nobody combines
- Hardware-lane awareness on a home box
- Cloud brains and local hands, measured as score per cloud dollar
- Units that earn rank on published, independent benchmarks
- One front door over many chats
- A findings register with a conflict ledger
- Local-first and open
Risks
THE SPECTRUM
Every agent and tool on the books, placed by its specialty. Diversity across the spectrum; each one excellent at its own task. Rank is earned by measured results.
Units
Each unit has one mission, a checker and a score. Rank thresholds are not set yet, so every unit is provisional.
| Role | Assigned | Kind | Mission | Benchmark | Rank |
|---|