Why AI pilots stall before production, and which blockers an engineer can actually fix
Most AI pilots do not stall because the model is weak. They stall because the pilot ran on clean sample data, with no real permissions, no real users and no real cost, and production has all four. Some of those blockers an engineer can fix in weeks. Others need a decision from the business that no engineer can make.
Blockers an engineer can fix
These are engineering problems. A forward deployed engineer working inside your environment can usually fix them.
| Blocker | The symptom you will see | What the fix looks like |
|---|---|---|
| Real data is messier than the demo data | Accuracy drops sharply on live inputs | A test set built from real failures, then fixes to extraction, retrieval or prompts until the set passes |
| Permissions | Security will not approve go-live, or the system shows people data they should not see | Retrieval and tool access scoped to the user's own permissions, using your existing identity system |
| No way to measure quality | Every review turns into an argument about examples | An agreed evaluation set and pass bar, run automatically on every change |
| Cost at volume | The pilot costs pennies, the business case says thousands a month | Measuring cost per task, then caching, smaller models for easy cases and fewer retries |
| Latency | Users wait and give up | Streaming, parallel calls, and moving slow steps out of the user's path |
| Edge cases | The system is confidently wrong on unusual inputs | A confidence check and a human review queue for the cases it should not decide alone |
| It lives outside the workflow | People have to open another tab to use it | Putting the output where people already work: the ticket, the CRM record, the inbox |
Blockers an engineer cannot fix
These look like technical problems. They are not, and no amount of engineering will move them.
| Blocker | Why engineering cannot fix it | Who has to |
|---|---|---|
| Nobody owns the outcome | Decisions stall, and the engineer builds toward guesses | A business owner with authority over the workflow |
| The data does not exist, or cannot be used | You cannot retrieve what was never recorded, or what legal will not release | Data owners, and legal or compliance |
| No measurable goal | Without a target, every result is arguable | The sponsor, before work starts |
| The process itself is disputed | The AI automates a workflow two teams do differently | Operations leadership |
| Compliance review never started | Go-live waits on a review nobody booked | Your risk or compliance function, from week one |
A two-minute self-diagnosis
Answer yes or no.
- Does the pilot work on clean sample data?
- Is there one named person who owns the outcome?
- Can you say, in one number, what production success looks like?
- Is the data the system needs available and approved for this use?
- Has compliance or security seen the design?
Yes to all five: your blockers are probably on the first list. That is engineering work, and a forward deployed engineer is the fastest way through it.
No to question 2, 3, 4 or 5: fix that first. Bringing in engineers before it is fixed will cost money and change little.
No to question 1: the system does not work yet. You need AI development before deployment work.
Need help with your answers? Contact us and tell us what works in the demo and what breaks in production.
What to do next
- If your blockers are engineering ones, read how we take stalled pilots into production
- If you want one engineer embedded in your team, see what a forward deployed engineer does and the five-question test
- If you want AMT to run the deployment end to end, see our forward deployed engineering approach
FAQ
Why do AI pilots fail to reach production?
How long does it take to get a stalled pilot into production?
Should we switch AI models if the pilot is not working?
Who should own an AI deployment?
Tell us what works in the demo and what breaks in production.