Who’s Accountable When AI Agents Fail?

The agent acts in milliseconds. Compliance reviews take hours. We are always playing catch-up.

That’s not my line. It’s a quote from a CFO in a survey Avalara released a couple weeks ago, and it’s stuck with me since I read it.

The tension every finance leader is living with

A recent survey of over 1,500 CFOs and senior finance leaders across the US, UK, India, and Australia asked one question underneath all the AI-agent hype: who’s actually accountable when one of these things gets something wrong? The answer, for most organizations, is: nobody’s really sure yet.

92% of finance leaders feel personal career pressure to prove their AI agent investment is paying off. 71% say that pressure is specifically about deployment speed. Only 7% say their organization puts governance first. That’s not a subtle gap — that’s almost everyone being pushed to move fast, and almost nobody being told to slow down and build the guardrails first.

It’s worth naming why that’s a problem specific to agents, not AI generally. A lot of governance in finance has historically leaned on friction. The fact that things took time gave you a natural checkpoint to catch problems. Agents remove that friction on purpose. That’s the whole value proposition. But it also means the compliance process built around “things take time” doesn’t have anywhere to attach anymore.

Governance hasn’t caught up

30% of finance leaders haven’t updated their internal control framework in the past year to account for AI-driven actions. Think about how much has changed in agentic AI capabilities in the last twelve months, if your framework hasn’t moved in that window, it’s almost certainly out of date. On top of that, 46% have AI incident response plans that are either untested or still in development. And 76% say they don’t have dedicated in-house expertise to actually understand how their own AI agents work. They’re leaning on IT, on the vendor, or in some cases, on nobody at all.

Vendor vetting is just as thin. A third of organizations require no formal safeguards before a vendor’s AI agent is allowed to operate. Only 31% require documented human review or escalation thresholds. Only 28% require documented audit logs showing how the agent actually reached its decision. Most of this comes down to the same root cause: enormous pressure to deploy, minimal pressure to govern.

Nobody’s sure who’s actually on the hook

When something goes wrong, accountability is genuinely unclear at a lot of organizations: it might sit with the vendor, the person who deployed the agent, the team managing it, or the executive who approved the investment in the first place. Just over half of finance leaders say they’d be very confident explaining an agent’s actions to an auditor. I’d push back on that number. It’s one thing to feel confident at a high level. It’s another thing to actually walk a regulator through the decision logic when they start asking specific questions.

There is a more measured instinct showing up too. When you ask CFOs where general-purpose AI tools belong in regulated workflows, the largest group (29%) say it’s fine, but only if every output gets reviewed by a human. That’s the right posture for where this technology is right now.

Key Takeaways

• Get the actual vendor liability language in writing — don’t assume who’s on the hook for an error based on the vendor’s reputation.
• Confirm audit trails exist for every agent action, not just the final output.
• Test your incident response plan before you need it, not after.
• Match agent autonomy to risk: the least autonomy in tax and compliance, more room in FP&A and go-to-market.

The question worth asking before your next deployment isn’t “can it do this.” It’s “who answers for it when it doesn’t.”

Want the CPE credit? Take the full lesson on EverydayCPE and earn 0.2 CPE credits: Who’s Accountable When AI Agents Fail?