AI agents can interpret messy requests, choose tools, and adapt a plan while they work. That is new. Everything required to make those agents dependable in production is much more familiar: clear interfaces, controlled state, repeatable operations, tests, observability, and rollback.
The industry often talks about agents as if software engineering has been replaced by a sufficiently detailed prompt. A prototype can encourage that illusion. Give a model a goal, connect a few tools, and watch it complete a polished happy-path demo.
Production removes the illusion. Customer records are incomplete. APIs time out. A user sends the same request twice. Policies conflict. Permissions change halfway through a run. The agent selects a reasonable action that happens to be wrong for this account. At that point, the quality of the prompt matters—but the quality of the system around it matters more.
The model adds a probabilistic component to the system. It does not repeal the engineering disciplines that make the rest of the system safe to operate.
An agent is a stateful service
A useful production agent is not a chat window. It receives work, gathers context, makes decisions, calls systems, waits for approvals, handles exceptions, and leaves evidence behind. That makes it closer to a stateful service or workflow engine than a clever text generator.
Treating it that way changes the design questions. What state exists before the run? Which facts can the agent update? Where is progress recorded? Can a paused job resume safely? What happens when the process crashes after an external action succeeds but before the agent records it? How does an operator know whether the work is complete?
Those are ordinary distributed-systems questions. They need explicit answers in data models, workflow states, and recovery paths—not instructions buried in a system prompt.
Contracts matter more when judgment is involved
Every tool an agent can use should have a contract. Inputs need schemas. Outputs need known shapes. Errors need useful categories. Permissions need to be narrow enough that a mistaken decision cannot become an unlimited action.
The boundary between deterministic code and model judgment should also be deliberate. A model is well suited to classifying an ambiguous request, extracting intent from an email, or drafting a response from approved evidence. Code should enforce account permissions, validate required fields, calculate totals, constrain tool arguments, and decide whether a high-risk action requires approval.
This separation is not a lack of trust in AI. It is good system design. Use flexible reasoning where the work is genuinely ambiguous, and deterministic controls where the business rule is not.
Retries must not repeat the consequence
Agent workflows retry. Networks fail, models time out, and long-running jobs get interrupted. A retry should not create a second invoice, send a customer two replies, or open duplicate tickets. That is why idempotency—a reliable way to recognize that an action already happened—is foundational.
Give work a durable identity. Record external action IDs. Check current state before writing. Separate planning from execution when the consequence is meaningful. Design each step so it can be replayed or resumed without guessing what already happened.
The same principle applies to handoffs between agents. If one agent classifies a request and another fulfills it, the handoff needs a versioned payload, an explicit status, and a clear owner. Natural-language context can travel with the task, but it should not be the only record of the workflow.
Model uncertainty belongs inside the test strategy
Traditional unit tests still matter for connectors, policy code, transformations, and state transitions. Agents add another layer: evaluations that test whether the workflow makes acceptable decisions across representative work.
Build a set of known tasks from real operating conditions. Include ordinary requests, incomplete inputs, conflicting evidence, tool failures, prohibited actions, and cases that should escalate. Score the business outcome, the evidence used, the policy boundary, and the downstream changes—not just whether the final prose sounds convincing.
Run those evals when prompts, models, retrieval, permissions, tools, or business rules change. A model upgrade is a dependency upgrade. It deserves regression testing before it gets more authority in production.
Traces turn surprises into engineering work
Logs tell you that a request failed. Agent traces should tell you what the agent saw, which sources it retrieved, which decisions it made, which tools it called, what changed, and where a human intervened. Without that chain, teams debate the model's behavior from the final output alone.
Good traces make failure actionable. A wrong answer may point to stale retrieval. A bad tool call may expose a loose schema. An unnecessary escalation may reveal a missing business rule. A correction can become a new eval case. Observability is how uncertain behavior becomes a concrete backlog instead of folklore.
Human review is a designed system state
Human review should not be a panic button attached after launch. It is a normal workflow state with defined triggers: low confidence, conflicting records, financial consequences, sensitive communication, policy exceptions, or a new situation the eval suite does not yet cover.
The reviewer needs the proposed action, supporting evidence, relevant history, and a clear choice to approve, edit, reject, or escalate. Their correction should be captured so the system can improve. A vague notification that says the agent needs help merely moves the integration work onto a person.
Rollback has to include the work
Rolling back agent software is only part of recovery. The agent may already have sent a message, changed a record, created a task, or triggered another workflow. Production design needs compensating actions: cancel the job, restore the prior field value, revoke the draft, close the duplicate, or route the affected work for review.
That requires versioned deployments, an audit trail that connects runs to consequences, and a kill switch that stops new actions without destroying evidence. The operating question is not simply whether the old code can be redeployed. It is whether the business can recover from what the new version already did.
The durable system is the product
A prompt-led prototype proves that a model can perform a task once. A production workflow proves that the company can operate that capability repeatedly: with permissions, state, tests, traces, human judgment, and recovery all working together.
The best agent systems are impressive for a surprisingly unglamorous reason. They make new model capabilities behave like dependable software. The intelligence may be probabilistic; the accountability cannot be.
Foundation designs and operates agent workflows as production systems—not prompt demos. If you are ready to connect AI judgment to real business work with the engineering controls it requires, talk to us.