SootBean factory · model efficacy · Jul 2026

Grok 4.5 ran the whole app factory

Every agent — planner, designers, workers, reviewer, QA — on Grok 4.5, building one full-stack phone app from a single prompt. It scored the highest quality we've measured.

7.5/10

Design (2 judges)

0.93

Spec coverage

46min

One prompt → shipped

Agents, all Grok

Quality vs every model we've run

Two independent strong judges score the finished app on design, polish, and how well it fulfils the brief. Grok 4.5 is the best result to date.

Grok 4.5 this run7.5 / 10

DeepSeek V4 Pro6.5 / 10

DeepSeek V4 Flash our default~5.2 / 10

Qwen 3.7 Max2.0 / 10

Both judges independently flagged the same strengths: a polished dark fitness tracker whose stats reconcile across screens, with all four brief capabilities working. It cleared every infra gate on the first pass — the cleanest and highest-scoring wave in the campaign.

Grok vs DeepSeek, side by side

DeepSeek V4 Flash is our default factory model. Here's the trade Grok 4.5 offers against it.

	DeepSeek V4 Flash	Grok 4.5
Design score	~5.2 / 10	7.5 / 10
Cost per wave	~$1	~$77
Wall-clock	~26 min	46 min
Tool discipline	good	9 errors / 3,072 calls
Role in the factory	every agent, by default	every agent, this experiment

Grok buys roughly +2.3 design points over our flash default. The cost gap is almost entirely cache-read pricing — agent loops resend the whole transcript every call (~134M input tokens), 96% of them cache reads, and Grok prices those far higher than DeepSeek. A smart split (Grok on the quality-setting roles, flash on the rest) is the obvious next test to keep most of the quality at a fraction of the cost.

What it actually built

Prompt: "a phone-first running tracker — choose a run type, enter distance and time, save the run, see today's summary." Grok's factory shipped this, seeded and working on web and native.

Today screen — remaining miles headline, stat cards, run list — Today — hero number, stat cards, trend

Log a run — run type, distance, time entry — Log — run type, distance, time, save

History — recent runs list — History — live list, newest first

Takeaways

01Grok 4.5 produced the highest-quality factory build we've measured — a real, premium, working app from one prompt.
02It's the most disciplined model we've run through the harness: clean decomposition, no tool loops, no wedges, recovers from every interrupt.
03The quality is a meaningful step up from our DeepSeek default (~5.2 → 7.5) at a premium price — the cost lives in cache-read pricing, not the model being slow or wasteful.
04Next: a Grok-on-quality-roles smart split to find out how much of the 7.5 holds at a fraction of the cost.