The Risk Isn't The AI. It's What Nobody's Watching.
In October 2025, Deloitte Australia had to hand back part of its fee on a government contract because the report it delivered was full of things nobody had checked. The firm’s 237-page review of a welfare compliance system, commissioned by Australia’s Department of Employment and Workplace Relations for roughly AU$440,000 (about US$290,000), misquoted a real Australian Federal Court judgment, inventing a passage the ruling doesn’t contain, and cited academic references nobody wrote. A university researcher spotted the fabrications after the report was published; Deloitte confirmed parts of it had been produced with Azure OpenAI’s GPT-4o and refunded the final instalment, about AU$97,000 (US$63,000). The refund is a fraction of the contract. What it confirms is the same principle a British Columbia tribunal established the year before, when it made Air Canada honor a discount its chatbot had invented out of nothing: when AI puts something in front of a customer that no one reviewed, and it’s wrong, the company pays. Not the software vendor. The company.
Both cases are tidy and public, which is why they get cited constantly. Most shadow AI risk isn’t either of those things. It’s quieter, it’s internal, and by the time an owner-led business notices it, the bill has usually already been paid somewhere else in the org.
We’ve made the setup case twice already on this blog: AI is already running inside most companies on personal accounts nobody approved, and closing that gap is a governance problem before it’s a technology problem, the same ground we cover in detail here. This piece isn’t re-arguing that AI is already in the building. It’s answering the question those two pieces leave open: what does it actually cost when nobody’s governing it? Four categories, each with a real, dated example attached.
Shadow AI risk shows up as four kinds of cost: a breach premium when an ungoverned tool is involved in a data breach, an audit trail that doesn’t exist when a regulator or a customer asks what happened, a compliance settlement when unreviewed automation makes a decision the law doesn’t allow, and a customer-facing mistake the company owns because nobody checked the output before it went out. Different parts of the business feel each one, but all four start at the same missing decision.
Data and IP walk out the door quietly
LayerX’s State of AI Usage Report, published in May 2026, found that nearly half of all enterprise AI conversations, about 47%, run through employees’ personal accounts rather than anything the company manages, and another 14% of conversations logged under a corporate account are still running on the employee’s personal AI subscription underneath. More than 6% of all enterprise AI conversations contain sensitive data, a rate that climbs above 8% on ChatGPT specifically, the single most-used tool in the study. None of that requires malice. It’s someone reformatting a client list or summarizing a contract on the tool they already have open, the exact behavior we’ve described happening inside owner-led companies before anyone trains anyone on anything.
It also has a price attached once something goes wrong. IBM’s 2026 Cost of a Data Breach Report, a Ponemon Institute study of 602 breached organizations published in July, found that shadow AI showed up in 43% of breaches, more than double the one-in-five rate the same report found a year earlier. When it showed up, the breach cost $5.39 million on average, roughly $400,000 more than this year’s record global average of $4.99 million.
Nobody can produce an audit trail after the fact
That same IBM report found that 68% of the breached organizations it studied had no AI governance policy in place at all, up from 63% the year before, and only 32% had any AI policy in place, down from 37%. That’s not a security footnote, it’s the actual mechanism behind most of these stories. When there’s no record of what data went into a prompt, which tool produced which output, or who approved what, there’s nothing to audit when a customer complains, a regulator asks, or a decision needs to be explained after the fact. You don’t find out you have a governance gap when you go looking for one. You find out when something breaks and you go looking for the paper trail that was never there.
Compliance and liability exposure doesn’t wait for a data breach
The EEOC’s first AI-discrimination settlement had nothing to do with a hack. In August 2023, iTutorGroup agreed to pay $365,000 after its hiring software was found to automatically reject female applicants 55 and older and male applicants 60 and older. Two years later, the same theory is playing out at a different scale. In May 2025, a federal judge in California let Mobley v. Workday proceed as a nationwide age-discrimination collective action, ruling that every applicant screened out by Workday’s AI hiring tools since September 2020 shares the same underlying claim: they all competed against the same automated scoring system. By February 2026, the court had authorized notice to go out to that group. Age discrimination law doesn’t have a carve-out for “the algorithm did it,” whether the company built the tool itself or bought it from a vendor. Nobody appears to have audited what either system actually did to real applicants until a plaintiff, or a regulator, went looking, which is the pattern that shows up in owner-led businesses too: a tool gets adopted to save time on a task like screening resumes or drafting a contract clause, and the review step that would have caught a problem gets skipped precisely because the tool is fast enough that nobody thinks to slow down and check it.
Unreviewed output that reaches a customer becomes the company’s word
Back to Deloitte. The misquoted judgment and invented citations weren’t malicious or even unusual as a failure mode: a large language model generating a plausible-sounding answer that happens to be false is the exact behavior every current AI tool is still capable of. What made it expensive wasn’t the mistake. It was that nothing between the model and the finished deliverable caught it before a paying client did. The Air Canada tribunal made the same point in 2024: a company doesn’t get to disclaim its own AI’s answers to its own customers. Whether the output is a chatbot’s reply or a report with the firm’s name on the cover, if nobody reviewed it before it went out, the company owns it.
Governance is the piece that connects all four
Each of these traces back to the same missing decision: nobody at the company had explicitly decided what AI is allowed to touch, what needs a human to check it before it goes out, and what gets logged along the way. That’s not a security project you bolt on later. It’s shadow AI governance: one of the things a leadership team is supposed to decide on purpose, the same way it decides who signs a contract or who approves a hire.
It’s also the piece most owner-led businesses skip when they finally sit down to take stock of where they actually stand with AI. Governance is one of the six dimensions in Sillewa’s free AI Readiness Guide, alongside Strategy, Tooling, Data, Culture, and Leadership. It’s a self-scored question set, not a submission to us and not a sales call, built so a leadership team can find out in an afternoon whether AI use across the company is governed or just happening. If you already know the answer and want to fix it, here is how to write an AI acceptable use policy when you have no IT department.
The move this week isn’t a policy binder nobody reads. It’s naming, out loud, who owns the answer to “what can and can’t go into a prompt here,” and writing that answer down somewhere other than your own head. Everything in this piece happened to a company that skipped exactly that step.