The Algorithm Decided

"The system flagged your account." "The model recommended denial." "The algorithm decided." Listen for those sentences and you start hearing them everywhere — from banks, from platforms, from agencies, soon from courts. They all do the same quiet work. They describe a decision that visibly affected a person while removing any person who can be said to have made it.

This is the part of the AI shift I think is underrated, because it isn't technical and it isn't loud. It's the slow evaporation of accountability at exactly the spot where a human being used to stand. A loan officer could be argued with. A caseworker could be named, escalated to, held to a standard. "The model" can be none of those things. It is a fog you can gesture at — and the person in front of you, the one who deployed it, chose its threshold, and benefits from its output, gets to stand behind the fog and shrug.

Nobody set out to build it this way. It's just that a machine in the loop is an extremely convenient place to lose responsibility. Every actor in the chain gets a real reason to point somewhere else. The vendor sold a tool, not a decision. The buyer just ran the tool as sold. The operator only clicked what the screen said was recommended. The model, of course, can't be a party to anything. The decision happened, someone lives with the consequences, and the responsibility for it has been spread so evenly across the system that it weighs nothing anywhere.

Most of what I care about building is aimed at this, though I don't usually say it in these words. Provenance — being able to prove what produced an output, from what inputs, under whose configuration. Attestation — a signed claim that a specific thing ran a specific way. Receipts, in the plainest sense: a durable trail that says this happened, these were the parameters, this party stands behind it. Boring infrastructure. The kind of thing nobody asks for until they need to reconstruct exactly who did what, and discover the honest answer is that the system was built so no one ever had to.

The trap is treating this as a problem the technology will grow out of. It won't, because the ambiguity isn't a defect anyone is motivated to fix. If "the algorithm decided" is a load-bearing excuse for the people deploying algorithms, they are not going to volunteer the receipts that take the excuse away. Accountability has never been added by the people it constrains. It gets forced — by a standard, a regulator, a court, or a customer who refuses to accept a shrug — and the systems that can produce a receipt on demand are simply the ones that survive that moment.

So the question I'd put to anyone putting a model between a decision and a human is short and unfashionable: when this goes wrong, and it will, whose name is on it? If the honest answer is "the algorithm's," you haven't automated a decision. You've automated the disappearance of the person who owed you one. The point of a receipt was never to slow things down. It was to make sure that when the machine decides, there is still, somewhere, a human who has to answer for it.