Why AI pilots stall at the trust gate
Most failed AI initiatives fail three weeks after launch, when the person meant to use the output quietly goes back to doing it the old way — accuracy was never the problem.
We've now sat in on the post-mortem for enough stalled AI pilots to notice a pattern that doesn't show up in the project retrospective: the model almost never gets blamed. What gets blamed, usually in softer language, is that "people didn't really adopt it." That phrase is doing a lot of work, and it's worth unpacking.
Adoption fails at a specific, identifiable moment — the trust gate — where someone with real accountability has to decide whether to act on a system's output without independently redoing the work themselves, and resistance to new tools has little to do with it. If the system hasn't earned that trust, they redo the work. Quietly, without telling anyone, because admitting it feels like criticizing the project. The pilot metrics still look fine. The forecasting tool still gets cited in the town hall. Nobody's actually using it.
We've watched this pattern repeat across otherwise well-built forecasting tools: a model that's genuinely, measurably accurate, producing a number with no visible reasoning behind it. The person accountable for acting on that number can't explain it to their own manager if it turns out to be wrong — so they keep a shadow spreadsheet going in parallel. The AI system runs in production. The actual decisions still run on the spreadsheet.
The fix is designing the system around the moment of accountability from the start, before the system goes anywhere near production — building in the reasoning trail, the override point, and the review step that the accountable person actually needs. That's a design decision as much as a data science one, and it has to be made by people who've sat with the team that will be accountable for the output, alongside the team that built the pipeline.
This is also why the trust gate is a useful diagnostic for evaluating any AI initiative before you fund it: ask who becomes accountable for this system's output, and what they'd need to see before they'd act on it without redoing the work themselves. If nobody in the room can answer that specifically, the pilot is going to stall in the same place the last one did — quietly, three weeks after a launch everyone called a success.