Executive Summary
All three sources point to the same thing: the enterprise asset that matters now is the ability to score AI output against ground truth. Nate Jones shows that evals already decide procurement. Several models clear the bar on routine work, so buyers trade down and usage grows while spend stays flat. Dwarkesh Patel argues the next capability jump will land hardest in domains that can be scored automatically. Simon Willison's plugin build shows generation is now cheap and verification is the bottleneck: his release has a browser demo but no tests. Clients that build evaluation infrastructure gain negotiating power today and early warning of where the next model generation will displace them. Clients without it overpay now and get caught off guard later.
What Changed
The "which model for which job" advisory conversation is closing. Jones calls it a 2025–early-2026 artifact. It is being replaced by a standing question: what is the cheapest model that passes our eval? Ramp's data shows the shift in the market. AI spend rose 1.65% in October after a flat August and September, while token volume kept climbing. Frontier models still take about 70% of dollars, but share of spend no longer tracks share of work.
The pressure also reaches sellers. Infosys customers now demand that AI productivity gains show up in pricing. Every services firm and independent advisor faces the same demand, and that includes Fred.
Cross-Expert Synthesis
Verification is the shared center, approached from three sides. Jones treats evals as a commercial tool for negotiating and switching. Dwarkesh treats verifiability as the variable that decides where capability concentrates. Willison shows the practical gap: one prompt produced a working integration, and the only quality signals are "alpha" and a screenshot. Read together, the scoring rubric a client builds to negotiate token prices is the same instrument that will show which of its functions are exposed first.
The real tension is saturation versus discontinuity. Jones says intelligence is becoming a commodity for most enterprise work. Dwarkesh says a step change is coming that makes today's frontier look like an intern. Both can be true, and the split falls along the verifiability line. Routine support, coding and document work is saturated now and will be bought on price. Verifiable high-value domains such as optimization, quant, chip and process design, and R&D are where a premium tier survives and where the 2028 scenario bites. The 70% frontier share of spend is the market already pricing that split.
Durable assets line up across sources. Jones warns against scaffolding built around current model weaknesses, because the next release makes it obsolete. Dwarkesh's discontinuity makes that warning stronger. Willison's build shows what does compound: mature reference implementations. Two existing plugins and a demo file beat a detailed spec. Organized data, system integrations, clean internal codebases and eval harnesses gain value as models improve. Prompt chains and guardrail patches that work around current flaws lose it.
Access becomes the constraint. Dwarkesh's superconductor example assumes lab access. Willison's plugin depends on one person's free tile server. Either way, the bottleneck moves from cognition to the systems, data and infrastructure the model acts through. Whoever owns those pipes captures the value.
Enterprise Implications
- Clients with no eval capability are paying frontier prices on saturated work. Moving from $5/M to $1/M average pricing funds five times the volume at constant spend.
- AI vendors lose pricing power at the commodity tier. Contracts signed before a client can benchmark alternatives should be treated as renegotiable.
- Integration-style development is now fast and cheap. The cost moves to review, hardening, testing and dependency control. Prototype-to-production gaps, such as relying on free third-party services, will multiply as non-engineers ship working code.
- Functions with tight feedback loops face displacement first. Planning that is anchored to today's models will under-prepare, and teams already treat medal-level math output as routine.
- Services margins built on AI-driven delivery savings will be competed away unless the value sits above the model.
What To Do About It
- Lead engagements with an eval build, not model selection. Use real tasks and edge cases, a baseline fixed in advance, and full cost per task including review, correction and harness changes. This is the deliverable that pays for itself in the first contract renegotiation.
- Map each client's functions on the verifiability axis. The same harness serves both jobs: trade down where work is saturated, and stress-test exposure where work is scorable and high-value. Run the 2028 scenario as a planning exercise, not a forecast.
- Audit AI spend for scaffolding versus durable assets. Cut investment in workarounds tied to model weaknesses. Fund data organization, integrations and internal reference code.
- Put a production-readiness gate on AI-generated software. Require tests, failure analysis and self-hosted or contracted dependencies before anything leaves prototype.
- Reprice your own practice before clients force it. Charge for the eval and integration assets that keep their value across model generations, not for hours AI is compressing.
Customer question worth asking: "Which three workloads could you move to a cheaper model tomorrow if you could prove quality held, and what stops you from proving it?" The answer is almost always that no eval exists, and that becomes the engagement.
Risks and Blind Spots
Evals can give false confidence. Benchmarks built on easy cases approve trade-downs that fail on the tail, and the review cost of a cheaper model can wipe out the token savings. Dwarkesh's thesis comes with no compute, cost or evidence behind its extrapolation, and the 2028 date is an assumption. Over-rotating on it wastes budget as surely as ignoring it. Ramp's sample skews toward mid-market spend and may not reflect large-enterprise contracts. Flat spend could mean temporary price competition rather than a permanent new pattern, and labs facing funding pressure may consolidate and raise prices. Finally, judgment-heavy functions that are hard to instrument are absent from all three sources. Whether they stay insulated or simply go unmeasured is unresolved.