Not Run Is
Not Passing

On 30 August I pointed a ten-minute load test at a banking demo I had built to walk the whole lifecycle. The target was ten transfers a second. The transfer endpoint answered at a p95 of 29.88 milliseconds against a budget of 500. All 5,986 transfers went through. The results log still records that run as partial.

Fifteen scheduled iterations never started. The gate that catches that is the strictest one in the file: dropped iterations must equal zero, because if the tool cannot inject ten requests a second, none of the other numbers describe ten a second. Every latency figure in that report was true, and the run had still not proven what it set out to prove.

Four Words, and Green Is Not One of Them

The standards that project runs on, which I wrote up with an AI assistant, give every suite, run and deliverable exactly one of four words, and nothing else is accepted in a results log or a handover. They are short on purpose. Each one tells the next person what to do.

It is the courtesy a good review comment pays when it says whether it blocks: one word that tells the reader what to do next. What the list leaves out matters as much as what it holds. There is no green on it, no looks good, no should work. Those words describe a feeling about a system. The four verdicts describe what happened to it.

The lifecycle guide on this site names a passing suite as the deliverable of the Test phase. This vocabulary is what keeps that word honest.

Not Run Is the Word Nobody Wants to Write

The same project carries a quieter admission. It has five Playwright specs, one for each user journey: registration, sign-in, balance, beneficiaries and transfers. The runbook's expected column says all five pass. None of them has been run yet, and the project's own README says exactly that, next to the capacity, spike and soak scripts that are in the same state: written, not run.

That is the word teams avoid, and the reason is practical rather than moral. Writing it is a promise of work. Leaving the row blank, or letting a suite nobody executed sit beside a green build, costs nothing today. Nothing failed, so nothing looks wrong, and a reader fills the silence with the most flattering reading available.

A test that was never executed has no result yet. Reporting it as passing borrows a result it has not earned, the same survivorship trap behind the planes that didn't come back, moved into a test report.

Check the Instruments Before the Verdict

A verdict is only as honest as the instruments behind it, and instruments are wrong in both directions. In the same build, a container reported itself unhealthy while the service answered perfectly from the host: the healthcheck asked for localhost, which resolved to the IPv6 address inside the container, and the server was listening on IPv4 only. A false red, fixed with one address, 127.0.0.1. The failures that go the other way are quieter, which is why the project's definition of done checks the instruments before anyone is allowed to say the word.

1. 01
   Every container healthcheck passes, at the right address.
   A service that is up and reported down, or down and never asked. What a health check should exercise beyond that, the path a user actually pays for, is the argument of Good Swimmers Drown Quietly.
2. 02
   A freshly registered user can load every first screen.
   The empty-state path, the one a seeded test account with a full history never walks.
3. 03
   The status codes the tests assert are the ones in the contract.
   Tests and API that agree with each other and disagree with the contract. The contract settles it, and then the test gets fixed, not the API.

Two more checks on the same list are about dashboards, and they belong to the Monitor phase: why a failure ratio has to read zero rather than no data is the closing argument of Every Error Has an Address. Every check on that list has broken a real build, which is the only reason it is there. A green summary hides all of them, because the summary is assembled from the same instruments that are wrong.

Partial Is the Most Useful Word on the List

Back to the run on 30 August. The shape of that failure was specific. Almost every request finished in tens of milliseconds, and one stalled for 5.87 seconds. Saturation would have slowed everything together; this was one stall. The project's performance notes name the suspects in order: a cold JVM with its connection pool still filling in the opening seconds, then a garbage-collection pause, then a database checkpoint. They also name the experiment that would tell them apart.

A pass would have ended the conversation there. Partial kept it open, with a hypothesis attached and a cheap experiment to settle it. That second run has not happened yet, and the log says so. Anyone who reads that row later knows exactly how far to trust it.