The accounting giant has agreed to partially refund Australia’s Labor Department for “hallucinations” in its product
The Australian arm of UK ‘big four’ accounting firm Deloitte has agreed to partially refund the cost of a report it produced for the government in Canberra after the document was found to contain multiple AI-generated factual errors, the Australian Financial Review reported on Sunday.
The Department of Employment and Workplace Relations (DEWR) quietly replaced the original report, which was published in July, with a revised 237-page version last Friday, just ahead of a long weekend. Officials initially said the update added new information and corrected “some footnotes and references.”
Sydney University academic Chris Rudge had earlier flagged numerous apparent “hallucinations” typical of large language models, prompting Deloitte to launch an internal review in August.
The updated report includes a new disclosure confirming that AI – specifically Microsoft’s Azure OpenAI GPT-4o model – had been used in its preparation. It also corrects over a dozen errors, including references to a non-existent court ruling and academic papers, as well as a fabricated quote attributed to Justice Jennifer Davies (misspelled “Davis” in the first version), the deputy president of the Australian Competition Tribunal.
Rudge told the Financial Review that Deloitte’s admission transformed what was previously “a strong hypothesis” into certainty, even if its confession was “buried in the methodology section.”
A DEWR spokesperson confirmed that Deloitte had “agreed to repay the final installment under its contract,” though the amount was not disclosed. The full study on the computerized application of automated penalties in Australia’s welfare system cost 440,000 Australian dollars (about $290,000).
Rudge, a welfare expert, reportedly first noticed something was amiss when the report cited a book supposedly written by his Sydney University colleague Lisa Burton Crawford. The title seemed outside her field of expertise and turned out to not exist at all.
You can share this story on social media:
Read the full article here