The Algorithm Decided
Robodebt asserted $1.763 billion in debts against 433,000 Australians, and there is no person you can name who decided any one of them was owed. Seven years of courts, a royal commission and an anti-corruption finding later, what that cost the people who built it.
Deanna Amato owed the Commonwealth $1.48. Centrelink told her she owed $2,754, added a penalty fee, and took her entire tax refund — $1,709.87 — without telling her first.
The debt came out of an arithmetic shortcut. The department took her annual income from the ATO, spread it evenly across the fortnights of 2011–12, and treated the result as what she'd earned in each one — true only if you work identical hours all year. She hadn't, so every fortnight she wasn't working showed an overpayment. The notice arrived and the job of disproving it was hers: find payslips from years ago, or the average stands. In November 2019 the Commonwealth conceded in the Federal Court that the method was unlawful.
Nobody in that sequence decided Amato owed $2,754.
That isn't a figure of speech. The ATO supplied an annual total it was never asked to vouch for fortnight by fortnight. Someone specified the averaging. Someone drafted the submission that carried it to Cabinet's Expenditure Review Committee. Someone approved a letter template with a reference number and no author. A contact centre worker read a balance off a screen to a person crying on the phone. Each act is defensible alone, which is how they totalled $1.763 billion asserted against roughly 433,000 people with no author anywhere in the stack. The diffusion wasn't a side effect. It was the useful part.
Australia then spent seven years trying to put a name back on it, using every instrument a wealthy democracy has. A class action settled for $1.8 billion, approved 11 June 2021. The Royal Commission reported on 7 July 2023 — three volumes, a crude and cruel mechanism, neither fair nor legal. Sixteen public servants went to the Public Service Commissioner, whose final report of 13 September 2024 found twelve had breached the Code on 97 occasions, sanctioned the four still employed, and recorded that the most senior had already resigned or retired, so nothing could reach them. The National Anti-Corruption Commission declined the six criminal referrals in June 2024, was found by its own Inspector to have decided that under apprehended bias, reconsidered, and published Operation Myrtleford on 11 March 2026: two of the six engaged in serious corrupt conduct, one for intentionally misleading departmental officers over the Cabinet submission in 2015, one for misleading the Ombudsman in 2017. It made no recommendations and referred no one for prosecution.
Add it up. Unlawful, $1.8 billion, a royal commission, a corruption finding against named individuals — and the total personal consequence is four sanctions ranging from a reprimand to a demotion, ten years after the scheme began.
Most of what I build is aimed at this. Provenance: proving what produced an output, from what inputs, under whose configuration. Attestation: a signed claim that a specific thing ran a specific way. Receipts, saying this happened, these were the parameters, this party stands behind it.
Robodebt is also the case that names the limit. Receipts would not have stopped it. The department had the trail: internal advice, an Ombudsman investigation, years of people saying the arithmetic was wrong. The corruption finding rests on officials misleading the bodies that held the receipts. The evidence existed. What was missing was an institution willing to act on it while the scheme was still running.
The Royal Commission's recommendation 17.1 goes at that: state plainly where automated decision-making is used, and publish the business rules and algorithms for independent expert scrutiny. Agreed in principle in November 2023. The Attorney-General's Department was still consulting on the framework at the end of 2024.
The question I'd put to anyone placing a model between a decision and a human is short: when this goes wrong, and it will, whose name is on it? If the honest answer is the algorithm's, you haven't automated a decision. You've automated the disappearance of the person who owed you one.