The pattern is recurring across the industry now. A finance team announces an AI pilot. The first month is exciting — early demos look great, the team gets quoted in trade press, leadership is happy. By month three, the energy has dimmed. By month six, the pilot has either been quietly shelved or extended indefinitely with shrinking actual usage. By month twelve, no one mentions it.
This is happening in finance functions across industries, at firms of all sizes. The pilots are not failing because AI doesn't work. They're failing because the people designing them understand AI but not the actual work, and the people who understand the actual work weren't in the room when the pilot was scoped.
The diagnosis is rarely "the model is bad." The diagnosis is almost always "the model doesn't know something that any analyst on the team would know within their first week."
The two-day float problem.
Here's a real example that recurs in different forms across many failed pilots. A team builds an AI agent to do daily cash reconciliation. The agent pulls the bank balance, pulls the GL balance, calculates the difference, and reports it. On day one, it works. On day three, it starts flagging exceptions that aren't actually exceptions.
The reason: there is a two-day float between when a transaction posts to the GL and when it clears the bank. Every analyst on the team knows this — they account for it implicitly when they reconcile. The agent doesn't, because no one wrote it into the prompt or the data feed.
The team's options at this point are essentially:
- Add the float logic to the prompt or the agent's reasoning, which requires someone who understands both the float dynamics and the agent's architecture.
- Have the agent escalate every flagged exception to a human, which defeats the time-savings purpose of the agent.
- Continue with a high false-positive rate, which destroys trust in the agent and ensures it gets shelved.
The actual problem was that nobody wrote down the float logic, because the float logic was tacit knowledge that lived in the analysts' heads.
The accrual calendar problem.
Another common pattern. A team uses AI to draft variance commentary for management reporting. The output reads well, the language is professional, the analysis is plausible — except in the months where there's a calendar quirk that affects the numbers.
Quarter-end true-ups. Reversing accruals. Bonus accrual catch-ups. The lease modification that hit unevenly. The vendor accrual that was reset because the contract was renegotiated. None of these things are visible in the pure number movement, and none of them are documented anywhere the model can read. They live in someone's head, in a footnote in last quarter's package, or in a conversation that happened in a hallway last Wednesday.
Without that context, the AI's variance commentary is structurally wrong. It will confidently explain a variance that exists for a reason the model doesn't know about, which means the explanation it generates is plausible nonsense. The reader either notices and loses trust, or doesn't notice and the wrong story makes its way to the CFO.
This is the most dangerous failure mode of AI in finance: the answer that looks right but isn't. It is more dangerous than an obvious failure because no one catches it.
The diagnostic framework.
There's a fairly clean way to predict whether an AI use case will work, and most teams skip it because it requires being honest about what the work actually is. The diagnostic has three questions:
1. Is the context fully expressible in the inputs?
Some tasks are bounded by their inputs. A transaction match: you have a list of transactions and a list of bank items, and the question is whether they pair up. The context required to do the task is in the data. AI can do this, and will do it well.
Other tasks require context that lives outside the data. Variance commentary on a P&L line that was affected by a contract renegotiation requires knowing about the renegotiation. Reserve adequacy on a portfolio requires knowing what the auditor said last quarter. These tasks are not bounded by the inputs, and AI will fail at them unless the context is systematically fed in — which is harder than it sounds.
The diagnostic question: can you write down everything someone would need to know to do this task, and is that information available in a structured form? If yes, AI is a fit. If no, AI is a partial fit at best.
2. Is the failure mode tolerable?
Some failures are absorbed by the workflow. An AI agent that flags 100 exceptions when 80 are real and 20 are noise is fine, because the human reviewer can quickly clear the false positives. The downside of being wrong is bounded.
Other failures are not absorbed. An AI agent that drafts a memo for the audit committee that contains a confident-but-wrong assertion has unbounded downside. The human reviewer might not catch it, the audit committee might rely on it, and the consequences land in places that are hard to unwind.
The diagnostic question: when the agent gets it wrong, what is the cost, and who catches it? If the cost is low and the catch is fast, AI is a fit. If the cost is high or the catch is slow, AI is at best an assistive tool, not a primary tool.
3. Does the workflow have a feedback loop?
This is the question most often missed. AI agents improve through feedback. A workflow where the human reviewer corrects errors, and those corrections flow back into the prompt or the model, gets better over time. A workflow where errors are silently absorbed — or where the reviewer is too busy to push corrections back — gets worse over time, because failures compound and trust degrades.
A concrete version of this. Two teams build identical agents to draft journal entry descriptions. Team A reviews the agent's output, edits the descriptions in place, and at end of week, an analyst pulls the diff between agent draft and final and feeds the patterns back into the prompt. Team B reviews the output, edits in place, and never closes the loop. After three months, Team A's agent is producing descriptions that need almost no edits. Team B's agent is producing the same quality it did on day one. Same model, same task, completely different trajectory — entirely because of whether feedback was structured.
The diagnostic question: is there a defined way for the human to correct the agent, and is the agent actually learning from those corrections? If yes, the pilot can succeed. If no, the pilot will plateau and decay regardless of how good the model is.
What a successful pilot looks like.
Strip away the noise and a successful AI pilot in finance has roughly the following characteristics:
- The use case is bounded by its inputs. Reconciliation, exception detection, format conversion, basic categorization. The data needed to do the task lives in the data.
- The failure mode is tolerable. Errors can be caught quickly by a human reviewer, and the cost of an uncaught error is low.
- The feedback loop is structured. The human reviewer's corrections feed back into the system in a defined way.
- The pilot has an honest baseline. The team measured how long the task took before the pilot, and is measuring whether the pilot actually saves time net of review and exception handling.
- The pilot has someone who actually does the work involved. Not a tech-side product manager, not a consulting partner, but a person who has done this task in production and knows where the surprises are.
That last point is the one most often missing. A successful AI-in-finance pilot is not led by the AI team or the consulting firm. It is led by — or at minimum heavily directed by — someone who has actually done the underlying finance work. They are the ones who know about the two-day float, the Q4 true-up, and the vendor accrual that resets every March.
What to do instead of a flashy pilot.
If you're starting fresh, the path that consistently works:
- Pick three small use cases instead of one big one. Each should take a single analyst less than a quarter to scope and build. Bigger pilots fail more spectacularly because the scope creep is faster than the learning.
- Pick use cases where you have the context written down. If you can't articulate the rules and the gotchas in writing, the AI can't learn them. Use cases where the rules are implicit are not yet ready.
- Define what success looks like in measurable terms. Not "this is faster" — "this saves 12 hours per close cycle, net of review time, with no increase in errors caught downstream." If you can't measure it, you can't defend it when finance leadership asks.
- Plan for the second-order effects. If the agent does the reconciliation, who reviews it? What controls cover the review? What happens when the analyst who used to do it leaves and the new analyst doesn't know how to spot when the agent is wrong? These are not hypotheticals.
The teams that get this right tend to be quiet about it. They don't issue press releases. They don't have a "Director of AI Transformation." They have a couple of senior analysts and a thoughtful manager who picked the right three use cases and built them carefully, and over two years they've shaved meaningful time off the close while improving accuracy. Their pilots succeed because they were designed by people who knew the work.
The teams that fail tend to be loud about it. They have a strategy. They have a roadmap. They have slides. What they don't have is anyone in the room who can answer the float question.
Deeper coverage of agentic finance, control framework design for AI-assisted workflows, and the broader future of the controller function lives at theagenticcontroller.com.