Editor's Note: This blog post is adapted from a keynote speech delivered at thegeekconf mini 2026 by Pushkar, Principal Technical Consultant at GeekyAnts. Drawing from a problem he has spent the past six months solving across client projects, Pushkar looks at why AI agents that perform well during demos often become unreliable in production. The talk explores failure handling, retries, context management, step counts, cost predictability, and the architectural changes required to build agents that can recover when things go wrong.
Pushkar works as a Principal Technical Consultant at GeekyAnts, where much of his recent work has involved taking agentic systems beyond prototypes and into real business workflows. The problem repeatedly appears after deployment: an agent works against predictable test cases in development, but production introduces changing inputs, external systems, network calls, tool failures, and longer execution chains. This talk focuses on that gap and on three areas that can help improve it: making failures visible, keeping the agent's context clean, and watching how many steps it takes to complete a task.
Watch the full talk, straight from thegeekconf mini 2026:
A Good Demo Does Not Mean a Reliable Production Agent
During development, an agent may be tested ten or fifty times against a relatively fixed set of inputs. The output looks predictable, the workflow works, and the system appears ready.
Production changes the conditions.
The same agent may now execute hundreds of times against inputs that were never part of the original test set. It may depend on APIs, MCP calls, external applications, authentication, network requests, or other tools that can fail independently.
This creates a compounding problem.
Suppose an individual agent step is 95% reliable. That sounds strong. But if completing one business task requires 29 successful steps, the reliability of the complete workflow is no longer 95%.
The 95% figure is only a representation, but the point matters: a workflow is affected by every step that has to succeed before the final result appears.
The longer the chain becomes, the more opportunities the agent has to move away from the expected outcome.
Why Coding Agents Appear More Reliable
Coding agents provide a useful comparison.
Modern coding systems can take dozens or even hundreds of steps to solve one task. They may inspect files, invoke tools, write code, run commands, check results, identify errors, and try again.
They are not reliable because every individual step succeeds.
They are reliable because the system is designed to repair itself.
If an action fails, the agent can see the failure, reason about what happened, change its approach, and continue.
That repair loop is often missing when teams build their own production agents. Considerable effort goes into the prompt and expected output, while less attention is given to what the agent should do when the workflow breaks halfway through.
That changes the engineering problem.
The goal is not simply to create an agent that gives the correct output on its first attempt. The system also needs to help the agent recognise failure and recover from it.
Three Things to Watch in Production
There are three areas that become especially important once an agent moves into a real production workflow:
- Make failures loud
- Keep the context clean
- Watch the step count
These are not replacements for good prompts or capable models. They are the surrounding controls that help those models perform consistently.
Make Failures Loud
One of the most damaging problems in an agentic system is a failure the agent cannot see.
Distributed systems already have ways to expose failures between services. Agentic workflows need the same principle.
A system should not fail quietly and allow the agent to continue as though the previous action worked.
Give the Agent an Error It Can Use
A generic error is often not enough.
Consider the difference:
Bad Request
versus
Date must follow the YYYY-MM-DD format
The first tells the agent that something went wrong. The second tells it what needs to change.
Large language models respond better when the failure is expressed clearly enough for them to act on it.
If the system returns an authentication error, formatting problem, missing field, invalid date, or unsupported value, that information should be explicit.
A failure should be loud, but it should also be useful.
That allows the agent to correct its input and retry instead of repeatedly making the same mistake.
Use Deterministic Gates Around Agent Actions
Not every decision in an agent workflow requires another model call.
Simple deterministic checks can be placed around important actions to inspect the current state before allowing the agent to continue.
These gates can verify whether an output matches the expected format, whether required data exists, or whether an action should be allowed at all.
The approach is straightforward: reason less where deterministic verification is enough, and verify more before allowing the workflow to continue.
Guardrails around individual actions can prevent small errors from moving deeper into the execution chain.
Not Every Failure Should Be Retried
Failures also need to be classified.
Traditional systems commonly deal with two broad categories:
- Transient errors: Temporary failures where trying again may work, such as a temporary server or network issue.
- Persistent errors: Failures where repeating the same action is unlikely to change anything, such as incorrect authentication or unavailable required data.
The distinction matters because unnecessary retries have a cost.
If the error is persistent, allowing the agent to retry five more times does not make the system more intelligent. It adds more steps, increases the context, consumes more tokens, and raises the cost of an action that still cannot succeed.
Retries should happen when there is a reasonable chance of recovery.
Keep the Context Clean When Retrying
Retrying introduces another problem: context contamination.
Suppose an external service returns a temporary 500 error. The agent retries, but the failed attempt remains in the context. It fails again, and that attempt is added too.
The context window now contains several failed paths even though the agent is still trying to complete the same action.
That information may no longer be useful.
Instead of carrying every failed attempt forward, the system can preserve the information that matters and allow the agent to retry with a cleaner context.
This becomes particularly useful for transient failures, where the task itself may be completely valid and only the surrounding service temporarily fails.
The idea is simple: do not make the next attempt reason through every mistake made by the previous one.
Watch the Step Count
The number of steps an agent takes to complete a task can become an important production signal.
Suppose a workflow normally requires ten steps.
A few days later, the same workflow regularly takes fifteen.
Nothing in the product has intentionally changed, but the agent now needs more actions, retries, or tool calls to reach the same result.
That should be investigated.
A longer run does not automatically mean that the agent will fail. But rising step counts can indicate that the system is struggling.
Longer Runs Also Mean Higher Costs
The effect is not limited to reliability.
Every additional model call, tool invocation, API request, or retry costs something.
If an agent that previously completed a workflow for a predictable amount begins making repeated attempts, its operating cost becomes unpredictable as well.
That becomes difficult to justify in a production product.
An organization needs to know whether an agent normally costs fifty cents, one dollar, or another predictable amount to complete a task.
A system where the same task costs one amount today and several times that amount tomorrow becomes difficult to scale.
This makes step count and cost closely connected production metrics.
The Model Is No Longer Always the Problem
Earlier generations of agentic systems could reasonably blame poor outcomes on model capability.
That explanation has become weaker.
Powerful proprietary models are widely available, and open-source models have also improved considerably. The underlying model is therefore not always the main reason a workflow fails.
The architecture surrounding it matters more than before.
Teams traditionally spent most of their effort writing increasingly detailed prompts and expecting those prompts to produce predictable outputs.
That works better when the inputs themselves are predictable.
Production inputs rarely are.
An external accounting system may be unavailable. Authentication may expire. An MCP integration may behave unexpectedly. An invoice may arrive in a different format.
A prompt cannot predict every production condition.
The surrounding system has to deal with them.
From Prompt Engineering to Loop Engineering
This leads to a broader change in how agentic workflows can be designed.
Instead of manually creating a prompt for every action and running an agent independently against every item, the system itself can decide when reasoning is actually required.
Consider an invoice-processing workflow.
Suppose 200 invoices arrive every day. A simple implementation may send all 200 through an agent individually. Each invoice can then involve model calls, tool calls, and network operations.
The total may quickly reach hundreds of execution steps.
A different architecture can process the predictable work first.
It can extract the invoices, validate them, and identify only the unusual cases. If eight invoices contain problems, the agent can focus on those eight.
The system may then create the prompts required to understand and fix those exceptions at runtime.
Instead of using an agent to repeatedly reason through 200 predictable items, reasoning is concentrated where it is actually needed.
This is increasingly described as loop engineering or graph engineering.
The developer is no longer only writing the prompt.
The developer is designing the system that decides when to prompt, what context to provide, how to validate the result, and what to do when it fails.
Three Numbers Worth Watching
Once an agent is in production, three numbers can reveal whether the workflow is becoming less reliable.
Metric | What to Watch |
Quiet failures | How many failures happened without the agent clearly recognising what went wrong? |
Median step count | Is the same workflow gradually requiring more steps to complete? |
Cost per task | Is the amount spent completing the same successful task increasing over time? |
Observability becomes essential here.
Logs from the API gateway, model provider, orchestration platform, or infrastructure should make it possible to see exactly where the workflow is breaking.
Without those signals, teams are left with the final output but very little information about how the agent reached it.
Production AI Needs Predictability, Not Perfection
An agent failing occasionally is not necessarily a badly designed system.
Even powerful models are not 100% reliable, and expecting a complex autonomous workflow to behave perfectly on every production input is not realistic today.
The more useful goal is predictability.
A production agent should have clear boundaries around:
- how it handles failures
- when it retries
- how much context it carries forward
- how many steps it normally takes
- how much one execution should cost
That gives teams a system they can measure and improve.
Where to Begin
Three things are worth taking away.
- Make every important failure visible and useful. The agent needs enough information to understand what went wrong and, where possible, correct it.
- Keep retries controlled. Clear unnecessary context, distinguish transient failures from persistent ones, and watch when execution chains begin growing longer than expected.
- Design the loop, not just the prompt. Strong models help, but production reliability comes from the architecture around them — validation, recovery, observability, context management, and predictable cost.
An agent does not become unreliable simply because it fails.
The bigger problem is when it fails quietly, continues without understanding why, and has no reliable path back.
Building that path back means thinking beyond the model itself — about grounding, validation, integration, and what happens after an agent is deployed. GeekyAnts explores these parts of the architecture further in AI Agent Development Services.
