In The Deadline Nobody Watched I wrote that a regulator's question is almost never "can you do this". It is "show me". I stand by that, and I want to take it one step further, because there is a harder version of the same question.
"Show me" is asked about a system. The harder one is asked about a single decision. One person, one day, one refusal, and a letter that says: explain this.
An audit is a conversation about your architecture. This one is a conversation with the person it happened to.
Three rulebooks on two continents now ask the same thing of automated decisions: say why, about this one, in words the person can use. A model that is accurate on average can still be unable to do it. Explanation is a capability you build at decision time, or you reconstruct later and call it a reason.
Three Rulebooks, One Sentence
I am not a lawyer, and I am describing what the texts ask, not when each one bites. The AI Act's dates moved in 2026, and which rule binds you depends on where your users live and what you decide about them. Read the dates for your case. The sentences themselves are plain.
The AI Act, Article 86. A person subject to a decision that a deployer takes on the output of a high-risk system listed in Annex III, and that significantly affects them, has the right to "clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken."
The Court of Justice, 27 February 2025 (case C-203/22, Dun & Bradstreet Austria). A customer was refused a mobile contract worth ten euros a month because an automated credit assessment found her standing insufficient. The Court said the controller must describe "the procedure and principles actually applied" so the person can understand which of their data were used and how. It added that it could be appropriate to tell them how a variation in that data would have changed the result. And it said the mere communication of an algorithm is not a sufficiently concise and intelligible explanation. Trade secrets do not close the door: the protected material goes to the supervisory authority or the court, which does the balancing.
The United States, CFPB Circular 2022-03 (26 May 2022). A creditor using a complex algorithm must still give applicants an accurate statement of the specific reasons for an adverse decision. Asked whether complex algorithms excuse a creditor from that duty, the Circular answers that ECOA and Regulation B "require creditors to provide statements of specific reasons" regardless.
Three different legal systems, written for three different problems, converge on one demand. Not "publish your model". Tell this person what happened to them.
What the Algorithm Cannot Say
Notice what the Court ruled out. Handing over the algorithm is not an explanation. That is the sentence most teams are not ready for, because their plan for "explainability" is a model card and a diagram.
A model card describes the system. The question was about a decision. The explanation has to say which of this person's data entered, how it was used, what the outcome was, and what would have moved it. Those are facts about one run, not about the model.
What a Decline Code Already Knew
Payments solved a small version of this long before anyone said "AI". Every declined transaction has always carried a reason, a short code produced at the moment of the decision and travelling with it. Some of those codes say little, and anyone who has stood at a till knows how that feels. But the mechanism was right: the reason is produced by the same path as the decision, at decision time, and stored beside it. I wrote about the human cost in A Declined Card Is a Person: a declined card is not an error code, it is a person.
The lesson is not the code. It is where the reason comes from. A reason written at decision time is a record. A reason reconstructed three months later, by re-running a model that has since changed, explains a different decision and calls it the same one.
The Decision Record
If I had to give the capability a shape, it is a record written with every consequential automated decision:
- the inputs actually used, not the inputs available;
- the model and version, the threshold, and the policy rule in force;
- the output, and any human step between the output and the decision;
- the reason, in plain words the affected person could read;
- what would have changed the result, where that can be said honestly.
Three things make it more than a log.
It is written at decision time, by the decision path, so it cannot drift from what happened. It is findable by the person's own identifier, because the question will arrive as a person and not as a request id. And it is tested the way an incident process is tested: pick a decision at random from three months ago and see whether support can explain it to the person in ten minutes, without an engineer.
That test is cheap, and it fails in an instructive way. Most teams find they can show the system and cannot explain the decision.
When the Decision Came From a Language Model
Everything above is easier when the logic is a scorecard and harder when a model with free text made the call. I would hold two lines.
Do not store the model's own account of its reasoning as the audit record. It is another output, produced the same way as the first, and it can read as convincing without being the cause. Treat it as something to evaluate, with the discipline in Tests, With a Probabilistic Twist, not as evidence.
And keep the model out of the part of the decision that has to be explainable. Where a rule can decide, let a rule decide, and use the model for the parts a rule cannot do. The more of the decision that is a rule you can name, the more of the explanation is simply true.
Being Asked Is Uneventful
In the deadline essay I ended on building so that being asked is uneventful. The same holds one level down. The system-level version of the question arrives once a year, with a lawyer and a date. The decision-level version arrives on an ordinary Tuesday, from someone who just lost something, and it is routed to whoever picks up.
Build the record now, while the decision path is still a diagram you can change. After the first letter arrives, you are not building a capability. You are writing an apology that has a ticket number.
A system you can audit is not yet a system that can answer. The answer is owed to one person, about one decision, and it has to have been written down while the decision was being made.