Deloitte Australia refund: AI-generated errors in a government report

This one isn’t about code. It’s about what happens when a supplier uses generative AI and nobody checks the result, and it’s the clearest example we know of why that matters to anyone who pays for work, software included.

What happened

Australia’s Department of Employment and Workplace Relations (DEWR) paid Deloitte around AU$440,000 for an independent review of the Targeted Compliance Framework, the system that penalises welfare recipients who miss obligations such as job search appointments. The report was published in July 2025.

Dr Christopher Rudge, an academic at the University of Sydney, read it and found references that didn’t hold up: citations to reports by real academics that those academics had never written, and a quotation attributed to a judge in Amato v Commonwealth, the Federal Court’s robodebt case, that doesn’t appear in the judgment. He suggested they looked like AI “hallucinations”.

In October a corrected version was published. The fabricated references and the invented quote were gone, and a new line in the methodology said that Deloitte had used “a generative AI large language model (Azure OpenAI GPT-4o) based tool chain” to fill “traceability and documentation gaps”. The original hadn’t mentioned it. Deloitte agreed to refund the final instalment of the contract. DEWR said that the substance of the review was unchanged, and so were its recommendations. Rudge described the new disclosure as a “confession”.

The missing step

Using AI to help write a report isn’t the failure here, any more than using AI to help write code is. The failure is that a document went to a client, and into the public record, with nobody having checked that its sources existed. That check would have taken an afternoon. Because nobody did it, the client paid for the work, then had to rely on an outside academic to find the errors.

If you commission software, the same thing can happen to you, and it’s harder to spot. A made-up citation can be looked up. A missing access check or a hard-coded key in code you’ve been handed looks exactly like working software until someone finds it.

What to ask your suppliers

None of these questions assume AI is a bad thing. They’re about making sure the checking still happens.

• Where do you use AI, and for what? A supplier that can answer plainly is one that has thought about it. You shouldn’t find out from a correction.
• Who reviews its output before it reaches us? For software, that means code review by a person who understands the change, not only automated checks, especially for anything touching logins, payments or personal data.
• What evidence will we get? Test results, review records, a list of the third-party components included. Ask for them to be part of what’s delivered.
• Do you follow the Software Security Code of Practice? Published by the government in May 2025 and developed with the NCSC, it sets out 14 principles software vendors are expected to meet, from secure development to how they tell customers about problems. It’s a reasonable baseline to ask about.
• Can we have it checked independently? For anything that matters, an outside review before you rely on it costs far less than finding out afterwards.

And put it in the contract. If a deliverable has to be checked by a qualified person, say so, and say what happens if it wasn’t.

Deloitte will be fine; one partial refund is a rounding error to a firm of its size. For a smaller business that has paid for an app, a system or a report, and relied on it, the cost of the check nobody did can be much higher.

Sources

• The Register, Deloitte refunds Australian government over AI in report (October 2025)
• AI Incident Database, Incident 1193
• GOV.UK, Software Security Code of Practice (May 2025)
• NCSC, Secure development and deployment guidance