Three Rungs and a No

Deep Dive · The Route, Deeper Practitioner · 9 min

I spent the better part of two decades in payments, where the interesting question was never what we could automate. It was what we deliberately would not. Plenty of that work was mechanical enough to hand to a machine, and some of it we kept in human hands on purpose, because the cost of being confidently wrong was somebody's money and a regulator's attention. That habit turns out to be the most useful thing I brought into AI work. The question is not where a model could go. It is where it belongs, and how little you can spend to put it there.

// the crux

Two decisions sit in front of every AI feature, and most roadmaps skip the first. Does this problem belong to a model at all, and if it does, what is the cheapest adaptation that survives contact with your domain? Prompt, then ground, then teach: climb only as far as the evidence forces you, and keep the right to stop at the bottom.

// in one breath
  • Four questions that disqualify most candidate use cases before anyone writes a prompt, and the one that quietly kills the majority.
  • Where the real use cases actually cluster across the nine lifecycle phases, including the two phases where AI is most often described and least often specified.
  • Why the ladder is climbed from the bottom, what each rung costs you in ways the invoice does not show, and the verdict worth writing into the roadmap: not here.

This dive sits at the front of Stage 02, before you learn to ration a context window or stand up retrieval, because both of those are answers. This is the question they answer. It assumes you have already picked a model family, which is its own procurement decision, and it does not re-teach the mechanics of grounding, which the RAG dive owns. It decides whether and how far.

the screen

Four Questions That Kill Most Candidates

Every backlog I have seen since 2024 has an AI section, and most of it is a list of places somebody noticed a model could plausibly go. That is not a use case list. It is a list of nouns. A use case survives four questions, and they are worth asking in this order, because each one is cheaper to answer than the next.

01 · Can you tell a good answer from a bad one, cheaply, at volume?
Not whether an expert could judge it over coffee. Whether you can judge it a thousand times a week without a person reading each one. If the only available grader is the same senior engineer whose time you were trying to save, the feature does not scale, it relocates the bottleneck. This is the question that quietly kills the majority, and it is why evals belong in the feasibility conversation rather than in a hardening sprint at the end. // fails when: quality is a matter of taste nobody has written down
02 · What does a confident wrong answer cost, and can you take it back?
Blast radius first, accuracy second. A wrong summary in a draft costs a reader thirty seconds. A wrong figure in a regulatory report costs something else entirely, and the difference is not the model, it is whether the action is reversible and who signs it. In regulated work this question outranks every benchmark, because the number that matters is not how often it is right, it is what happens the time it is not. // fails when: the action is irreversible and nobody is on the hook
03 · Is there a body of knowledge that makes the answer yours?
A general model gives a general answer. If the value of the feature comes from something only your organisation knows, your runbooks, your contracts, your ten years of incident write-ups, then you have something worth grounding in and the second rung is on the table. If the answer is the same one anybody could get from a public chat window, you have not found a product, you have found a shortcut your users already have. // fails when: the knowledge that would make it valuable is not written down anywhere
04 · Would a deterministic solution do the job?
A regular expression, a lookup table, a rules engine, a well-named function. If one of those solves it, it will be cheaper to run, easier to test, and it will still behave the same way next quarter. Asking this last is deliberate, because it is the question people skip once they are excited, and it is the one that saves the most money. // fails when: you are reaching for a model because it is interesting

There is a fifth move for when the four are not decisive, and it is a habit I have leaned on long before any of this: invert the problem. Stop asking what would make this a good use case and ask what would have to be true for it to be a bad one. The answers arrive faster and they are more honest, because the failure conditions of a system are usually easier to name than its virtues.

where they live

The Phases Where They Actually Cluster

Run that screen across a delivery lifecycle and the survivors are not evenly spread. The full eight-forces-by-nine-phases grid is the practitioner map, and this is not a second copy of it. It is the shorter answer to a different question: given the screen above, which phases repay the attention first.

01 Analyze
Strong. Requirements are text, ambiguity is checkable by a human in seconds, and a wrong draft costs nothing. Drafting for the three readers of a requirement is the clearest early win.
02 Design
Underused, and the subject of the next section. Trade-offs, decision records and diagram-from-code all pass the screen, yet most teams still only hand the model the finished design.
03 Develop
Obvious and already crowded. A failing test is the tightest scope you can hand an agent: machine-checkable, bounded, reversible.
04 Test
Strong on generation, weak on judgment. Cases and fixtures generate well; deciding what good looks like stays yours.
05 Build · 06 Deploy
Mixed. Pipeline config drafts well and reviews fast, but the action at the end is production, so question 02 governs.
07 Monitor
Strong. Correlation across noisy signals, first-pass triage, and a dead-letter queue that doubles as a dataset.
08 Deliver
Thin as an AI target. Acceptance is a human act by design, and dressing it up as automation mostly moves the signature.
09 Change
Quietly strong. Impact analysis across a codebase is retrieval over your own history, which is exactly what question 03 was looking for.
the thin phase

Analysis and Design, Read Properly

Design is where most AI writing, mine included, tells you what to hand the model rather than what the model can hand back. That asymmetry is worth correcting, because design work passes the screen more comfortably than almost anything else in the lifecycle. It is text. It is reviewed by people who can spot a bad answer in a minute. And nothing ships because a diagram was wrong.

Four things earn their place here. Decision records, where the model drafts the options and consequences from a design conversation and you supply the decision itself. Trade-off tables, where it is genuinely useful to be handed the axes you did not consider, including the ones you will reject. Diagram from code, where the source is ground truth and the output is checkable by anyone who knows the system. Context-map discovery, where a model reads a sprawl of services and proposes the seams, which is a proposal, not a finding.

Where it lies: a model will produce a confident architecture for a business it does not understand, complete with plausible trade-offs it has never paid for. It has read every reference architecture ever published and lived inside none of them. The judgment that survives contact with your constraints, your regulator, and your team's actual skills is the part that stays yours, and design is precisely where borrowed confidence is hardest to detect and most expensive later. The forces underneath this phase are Chapter 9; what follows here is only the adaptation question.

the ladder

Three Rungs, Climbed From the Bottom

Once a use case survives, the second decision starts, and this is where money is usually lost. Teams begin at the top of the ladder because the top sounds the most serious. A fine-tuned model is a real project with a real budget and a slide. A better prompt is a Tuesday. The ladder is ordered by what each rung costs you in maintenance and lock-in rather than in licence fees, and the rule is to climb only when the rung below has demonstrably failed.

03
Teach it
Fine-tuning or an adapter, for when the gap is behavioural rather than informational: a house format the model keeps missing, a tone, a classification scheme with no natural description. It buys consistency and cheaper inference, and it costs you a training pipeline, a dataset you now own forever, and a model that quietly ages out of step with the base you fine-tuned from. // pay when: the behaviour is stable and the volume justifies the pipeline
02
Ground it
Retrieval over your own material, so the answer is yours rather than the internet's. This is the rung most real products live on, and the right default whenever question 03 came back yes. It costs you an index to keep fresh and a new failure mode that arrives politely: the wrong passage retrieved, then reported in fluent prose. The mechanics are the RAG dive. // pay when: the value lives in knowledge you own
01
Prompt it
Instructions, examples, and a well-built payload. Free to change, trivial to revert, and far stronger than its reputation once the payload is treated as a contract with a budget rather than a text box. A surprising share of shipped features never needs to leave this rung, and everything you learn here transfers upward. // pay when: nothing, which is the point
read upward · each rung is a commitment, not an upgrade

The discipline is to make the ladder visible in the decision. Write down which rung you are on and what evidence would move you up. Without that, teams drift upward for reasons that have nothing to do with quality, and end up maintaining a fine-tuned model to solve a problem a better payload had already solved twice as cheaply.

the fourth answer

The No Is a Real Answer

The most useful outcome of this whole exercise is often a decision not to build. It is also the one nobody writes down, which means the same rejected idea returns in the next planning round with a new name and no memory of why it was dropped.

// when the answer is no

A deterministic solution already works and is cheaper to run. The output cannot be graded without the expert you were trying to free. The action is irreversible and no one will own the review. The knowledge that would make it valuable was never written down. Or the honest version: someone wants AI in the product and has worked backwards to a place to put it.

None of those are failures of the technology. They are the screen doing its job. Record the no with its reason and its expiry, because two of those conditions genuinely change over time and the other three rarely do.

what carries

The Same Judgment, Pointed Somewhere New

None of this is new engineering. Scoping work to what you can verify, sizing a solution to the problem instead of to the ambition, refusing the expensive option until the cheap one has actually failed, writing down why you said no: that is ordinary senior judgment. It is worth more in AI work than in most places, because this technology is unusually good at looking like it is working. The screen and the ladder are not a methodology. They are the questions a careful engineer already asks, made explicit so that a roomful of enthusiasm cannot skip them.

// carry forward

You have decided it belongs, and picked a rung. Now go build that rung properly. Rung 01 is the payload, and it is a larger subject than it looks: The Payload Contract is the next dive, then retrieval and the index behind it if question 03 sent you up a step. Or take the whole climb at a glance on the Route at Stage 02.