Mira Chen ยท Mar 17, 2026
Twenty pilots, twenty different companies. The technical patterns rhyme. The organizational ones rhyme harder. Here are the five things we know for sure now that we did not know at deployment one.
Not engineering. Not data. The person doing the work today is the one who knows when the agent is wrong, and they are the one whose markups become the eval suite. If that person is not in the room for the first two weeks, the pilot fails.
Every customer wants to start with "do all of AP." The right starting scope is "do AP for one entity, for invoices under $10,000." The narrower scope ships in two weeks and earns the right to widen. The wider scope ships in two months and earns nothing.
An agent that escalates 40 percent of cases burns the approver out in a week. The approver stops reading the agent's reasoning and just clicks approve. You have built a worse system than the one you had. Aim for under 10 percent escalation by week three, or pull the agent back.