The Model Is a Procurement Decision

Every vendor decision I have run started the same way: a spreadsheet with a column for every feature the sales deck mentioned, and a winner that scored highest on features nobody actually used. Cloud providers, database engines, message queues, payment processors – the checklist always looked rigorous, and it always missed the one line that mattered a year later: egress cost, a support SLA with more asterisks than clauses, a migration path that turned out not to exist once you actually needed it. I have sat in enough of those bake-offs to know the tell. The real decision was never on the slide. Choosing between model families is the same bake-off wearing a research paper's vocabulary, and the checklist is failing you in exactly the same way.

Stage 01 taught you to speak to a model through an API; it did not ask which model, and neither does Chapter 4, whose subject is the machine any of these families runs on, not the choice between them. The honest answer to which is best is that the field reshuffles faster than any dive can hand you a permanent winner. What this one can hand you is the checklist: the same five questions a good procurement process already asks of any vendor, aimed at this one.

The Checklist Was Never the Point

Model comparisons on the internet mostly chase one number: whoever leads this quarter's benchmark, on a test suite that may not resemble what you are actually building. That number does not hold still. Leaderboards have reshuffled within a single quarter before, and they will again, which is precisely the point: a comparison this specific has a shelf life measured in weeks, not years. A procurement process never asked which vendor scores highest on the sales deck's own test. It asked which vendor's failure mode you can live with, at a price you will actually pay, without signing away your ability to leave. Model selection deserves the same five questions, not a sixth benchmark chart.

Five Questions Instead of One

Run any model family through the same five questions you would already run past a cloud vendor, a database engine, or a payment processor, and the leaderboard stops being the whole conversation.

1. 01Context & modality

   Flagship tiers across all three closed vendors now sit at roughly a million tokens of context, with room to spare at the high end, and each ticks the vision and structured-output boxes as standard, not as an upsell. The frontier stopped being window size; it is what you can usefully fill it with, which is Stage 02's whole subject, not this one's.

   // check: does your workload ever actually use a meaningful fraction of what you are paying to keep open?
2. 02The cost model

   Every vendor now ships the same shape: a cheap-and-fast tier, a mid tier, a frontier tier, each roughly five to ten times the last on both sides of the price sheet. Caching and batching can beat the sticker price by most of an order of magnitude before you touch a single one of the six knobs G14 already named, and that lever is now standard across vendors, not a trick one of them offers.

   // check: are you pricing the cached, batched rate, or only the number on the landing page?
3. 03Deployment & openness

   The real fork was never the logo; it is which of the three doors you are being handed. A closed vendor defaults you to rent-the-answer. An open-weights family hands you rent-the-room or own-the-metal instead, provided you are willing to run the metal yourself.

   // check: who is actually holding the GPU, and what does that cost on a Tuesday afternoon when traffic is not spiking?
4. 04Ecosystem maturity

   Tool-calling, MCP support, eval tooling, fine-tuning paths: the closed vendors have converged hard on the protocol layer, which means this axis is closing, not opening. It is the wrong place to spend an evaluation budget you did not have five years ago either.

   // check: is there still a real gap here for your stack, or are you weighing a difference that stopped mattering two quarters ago?
5. 05Drift risk

   The one axis that should change your process, not your pick. Everything above ages in months. An evaluation harness run against your own workload does not, because it never claimed to be permanent in the first place.

   // check: when this shifts again, will you re-read somebody else's benchmark, or re-run your own?

Read the Licence Like a Contract

The open-weights door looks like the free door, and free is doing a lot of unpaid work in that sentence. A licence that reads permissively at a glance can still carry a usage cap keyed to how many people end up using what you build, a carve-out for entire regions, or a restriction on what field you may deploy into – exactly the fine print a procurement process already knows to hunt for in a vendor contract, now hiding in a model card instead. The cleanest of the current open licences genuinely mean it: no seat cap, no royalty, no geography excluded. Others look that way until you cross a size threshold nobody mentioned on the download page. Read it the way you would read the exit clause of any vendor contract, before you build on it, not after you are already dependent on it.

Same Discipline, New Menu

None of this is a new discipline. It is the one you have been running for two decades, pointed at a menu with new items on it.

The Question Underneath the Question

Every one of these five questions resists a permanent answer, and that is not a flaw in the dive; it is the actual shape of the field. What does not reshuffle quarterly is the process. You do not pick a vendor once forever in a real procurement either. You pick a process for re-deciding, with an exit cost you priced in before you signed anything. Put your model choice behind an interface you can swap without a rewrite, keep the evaluation harness that makes re-deciding a rerun instead of a redo, and the next reshuffle becomes an afternoon's work instead of a migration project. The model you pick today is not the one you are actually procuring. The process you build around picking it is.