I’ve been thinking about something that came up on a client engagement recently, and I think it’s worth talking about.
I was helping a client build a risk registry for their Claude deployment. Identified the risks, defined the preventative controls, the monitoring controls — all of it. And I used Claude to do it. Which, when you think about it, is a little ironic. But the output was solid.
The client had their internal cybersecurity team review it. They came back with minor tweaks — just some adjustments around how certain controls mapped to their existing tech stack. On the debrief call, their first comment was something like: “Yeah, our starting point would have been the same. We’d have used an LLM to pull this together too.”
Two experienced teams. Same right answer. Same starting point. That’s not a failure story — that’s AI working the way it’s supposed to.
But here’s what stuck with me afterward: if both teams’ first instinct is the AI, what’s our plan for when the AI isn’t there?
We’ve Been Here Before
Think about what happened with Excel in the early nineties. Accounting got dramatically faster. And for a while, people just trusted the output. That caused real problems — spreadsheet errors contributed to actual financial restatements. Logic was buried in cells nobody was auditing. The Sarbanes-Oxley era created control requirements partly because of overdependence on unaudited spreadsheet models. The profession eventually developed norms: version control, formula auditing, independent review.
We are in the early nineties of AI right now. The power is obvious. The guardrails haven’t caught up. And unlike Excel — where you could click the cell and see the formula — AI output comes with no visible logic trail. It looks authoritative. There’s no row 47 to audit.
Three Forces Making This a Right-Now Conversation
The reliability gap. Major LLM providers have each had multi-hour outages in the last couple of years. These aren’t edge cases — they’re a regular feature of infrastructure that’s still scaling. Beyond full outages, there’s model drift: a model’s behavior quietly shifting between versions. GPT-4 users widely documented a “laziness” change in late 2023. These subtle failures are harder to catch than a full outage, because at least when the tool is down you know to stop using it.
Cost uncertainty. AI is cheap right now — remarkably cheap relative to what it can do. But it’s cheap because the major labs are burning capital to build market share. That math changes. Workflows built at subsidy prices may not survive at market prices. If your practice has embedded AI into every step of a process, you haven’t stress-tested what that looks like when the cost is ten times higher.
Skills atrophy. This one’s harder to see. When AI handles the hard part of a task repeatedly, the human judgment that used to do that work gets less practice. And here’s the dangerous part: if your judgment atrophies, you also lose the ability to evaluate whether the AI is doing the job well. That’s what researchers call pseudo-competence — you look capable, you deliver quality work, you get positive feedback. But you couldn’t recreate it without the tool. And you wouldn’t know until the tool was gone.
What Accountants Should Do Now
None of this is an argument against AI. I use it constantly and I believe in it. But I also think the professionals who navigate this transition well are the ones treating AI like the operational dependency it actually is.
Map your AI dependencies. Which steps in your regular workflows require AI to complete? Could you do them without it, and how long would it take? No good financial controller runs without understanding what happens if the ERP goes down. AI is a critical vendor. Treat it like one.
Keep the planning phase manual. Before you ask AI to build the risk registry, write the first draft of key risks yourself. Before you ask AI to structure the memo, write the outline. Then use AI to pressure-test, expand, and refine. This keeps your judgment exercised — and means you’re comparing AI output against your own thinking instead of just accepting it.
Run the 48-hour simulation. If your AI tool went down tonight — full outage, 12 to 48 hours — which client commitments are at risk? Which deliverables can’t ship? Which steps can no one on your team do manually? Most professionals have never run this scenario. The answer tells you exactly where your real exposure is.
Key Takeaways
- AI is probably the next Excel — but we’re in the messy middle. Outages, model drift, and subsidized pricing haven’t resolved yet.
- Professional AI dependency is an operational risk. Map it the same way you’d map any critical vendor dependency.
- Cognitive atrophy is real. If you never do the hard parts manually, you lose the ability to evaluate whether AI is doing them well.
- Keep the planning phase manual. Your judgment about what questions to ask is the most important input — don’t offload that.
- Run the 48-hour simulation. If your AI tool went down tonight, which client commitments are at risk?
Want the CPE credit? Take the full lesson on EverydayCPE and earn 0.2 CPE credits: [lesson link]
Leave a Reply