Our AI pilot stalled after the demo, who helps get it into production?

The stall is almost never technical. In our experience there are four causes, and a stalled pilot usually has at least three of them at once. None of them get fixed by a better model.

The first is that the pilot had a champion but no owner. A champion is the person who was excited enough to get it built. An owner is the person whose week gets worse when it breaks. When the pilot ends, the champion goes back to their actual job, and there is nobody whose name is on the thing, which means there is nobody who can say yes to production either. Approval work with no owner drifts, and drift looks exactly like a stall.

The second is that nobody wrote down, in advance, what working means in numbers. Without a threshold agreed before the demo, the review meeting becomes a taste test: one person says it felt unreliable, another says it looked great, and with no agreed line to clear, the safest decision available to the room is to wait and see. Waiting is a decision that never has to be defended, so it wins by default.

The third is demo data versus real data. Demos run on clean examples picked because they work. Your real inputs include the misspelled surname, the duplicate record, the PDF that is a scan of a fax, the customer who answers a question with a different question. A pilot that looked right on twenty hand-chosen cases can fall apart on the long tail, and nobody sized that tail because the demo never touched it.

The fourth, and the one that quietly kills the most pilots, is that nobody decided what happens when it is wrong. Every production system fails sometimes; the only question is whether the failure is contained. If no one has answered who gets told, how fast, and what the customer sees in the meantime, then signing off on production means personally absorbing an unbounded risk. Sensible people do not sign that, so it sits.

Production is a checklist, not a launch date. Start with a named owner: one person, on the org chart, who is accountable for the output and has the authority to turn it off. Not a committee, not the vendor. If you cannot name that person in one breath, nothing further on this list will hold.

Then a baseline. What does the process cost today, in time, errors, or missed work, measured over a real week rather than remembered? Without a before, there is no after, and no vendor including us can be held to a payback claim. Two weeks of tally marks in a spreadsheet is enough to turn a guess into a number you can argue with.

Then an acceptance test written before the next build hour: the specific rate the system must hit, on a sample of your real records rather than the demo set, for the owner to accept it. Run it as a shadow first, where the system processes live work in parallel but a human still does the real one, and compare the two. That comparison is the only honest evidence that a demo transfers, and it usually surfaces the exceptions nobody mentioned in the kickoff.

Then the failure path, in writing. What the system does when it is unsure, who it hands to, how fast the handoff happens, what the customer experiences during it, and where the log lives that a human reads on a schedule. After that, go live on a narrow slice, one location or one queue or one document type, hold it there long enough to see a full cycle of exceptions, and widen only when the exceptions stop being surprises. When we scope a build, it ships in 14 to 21 business days, but the shadow run and the slice are calendar time you cannot compress by paying more.

Before hiring anyone to finish the pilot, work out which of the two actually broke, because the fixes are not related and the wrong one is expensive. Three questions settle it most of the time.

First: could two experienced people in your business do this task the same way, without asking each other? If not, the process is undefined, and you were asking software to make judgment calls your own team has never agreed on. That is a process problem wearing an AI costume, and no vendor can solve it for you. Write the rules down first, argue about the ten hardest cases in a room, then automate.

Second: how often is this a real exception? Count a week of them honestly. If four in ten items need a human to decide something unusual, automation is not the lever; you are buying a machine that hands most of the work back and adds a review step on top. Fix the causes of the exceptions and the same process often stops hurting enough to be worth automating at all.

Third: has this process changed in the last three months, and is anyone arguing about changing it now? Automation freezes a process in place. Automating one that is still moving buys rework, not savings. If the answer to both questions is no and the exception rate is low, the process is sound and the pilot is what failed, which is the good case: it usually means missing acceptance criteria, missing failure handling, and a demo built on data that flattered it.

Ask five questions in the first call, and score the answers rather than the enthusiasm. What would make you tell me to kill this? A vendor with no answer sells builds, not judgment, and a stalled pilot is precisely the moment you need judgment. What acceptance test would you put in writing before starting, and against what sample of my real data? If the answer is vague, you are buying a second demo. Who is the named owner on my side, and what does their week look like after go-live? Any vendor who does not push for that name is planning to hand you the same orphan.

Then two on evidence and exit. Ask them to label every number they show you as measured, client-reported, or a projection, and watch whether the labels survive the follow-up question; blurring those three is marketing dressed as diagnosis. Ask whose accounts and credentials the finished system runs on, and what breaks if you revoke their access and stop paying. If the honest answer is that it stops, you are renting the thing you already paid to build twice.

Be suspicious of a firm quote before anyone has looked at your pilot, and equally suspicious of a rescue proposal that starts with a rebuild. Sometimes rebuilding is right, but it is also the most profitable recommendation available to a vendor, so make them show their reasoning against what already exists. And if the process touches patient or health data, do not accept a compliance logo on a website as an answer: ask for a signed business associate agreement, ask for the breach notification window in writing, and ask exactly where your data is stored and which subprocessors see it.

Going back to the vendor who built the pilot is often the cheapest correct move, and we will say so on a call rather than pretend otherwise. They already know your systems. Give them the owner, the baseline, and the acceptance test they were never given, and a fair number of stalled pilots restart on that alone.

Sometimes the pilot did its job by failing. It answered the question, and the answer was that this is not worth doing. Killing it is a result, not a loss, and the money already spent is gone either way. Continuing because you have already spent it is the single worst reason on the list, and it is the reason we hear most often.

If the process is undocumented or the exception rate is high, the cheaper answer is to fix the process by hand first. Write the rules, resolve the disagreements, remove the causes of the exceptions. That work is unglamorous, costs no software, and frequently removes enough of the pain that the automation slides down your priority list where it belongs. It also makes any future build cheaper, because most of what makes a build expensive is ambiguity.

If volume is genuinely low, a checklist and a calendar reminder beat any system. A task that runs twice a month can be stressful and still not be worth automating; stress and cost are different measurements and only one of them pays back.

And if you already have an internal owner, a clear acceptance test, and someone who can read a log, you may not need to hire anyone. The last mile of a stalled pilot is often just logging, a human handoff, and a two week shadow run against real data. Do it yourself and keep the fee. Bring in an outside pair of hands when the knowledge is spread across departments, when nobody trusts anyone else's account of what happened, or when the numbers you need to decide simply do not exist and someone has to go count them.