Making Agent Workflows Reliable

An agent demo usually proves that a model can complete a task once. A production workflow has to prove something harder: that the surrounding system can detect failure, preserve useful progress, recover safely, and explain what happened.

That distinction matters because an agent is not a conventional function. Its output depends on natural-language instructions, retrieved context, tool responses, model behavior, and the current state of the world. Even when every component works as designed, the agent may choose a different path on the next run.

Reliability therefore does not come from trying to eliminate uncertainty. It comes from containing uncertainty inside a deterministic control system.

Start with an explicit state machine

A long prompt is not a workflow definition. It may describe the goal, but it rarely states which transitions are legal, what gets persisted, or what should happen after a partial failure.

Model the workflow as named states instead:

type RunState =
  | { kind: "queued" }
  | { kind: "planning"; attempt: number }
  | { kind: "executing"; planId: string; step: number }
  | { kind: "awaiting_approval"; actionId: string }
  | { kind: "verifying"; artifactIds: string[] }
  | { kind: "completed"; resultId: string }
  | { kind: "failed"; code: string; retryable: boolean };

This makes operational questions answerable. Can an executing run go directly to completed, or must it pass verification? Can a failed run resume from a checkpoint? Which states allow external writes? The type does not answer those questions by itself, but it gives the team a precise place to encode the answers.

Persist state after every meaningful transition. If a process disappears after creating an artifact but before recording it, the next attempt may repeat the action. A durable checkpoint should include the input, selected plan, completed steps, external operation identifiers, and the model and prompt versions used.

Separate decisions from effects

Agents are useful decision-makers; they should not have unrestricted authority to produce side effects. Put a narrow tool layer between the model and external systems.

A good tool has a small contract, validates its arguments, returns structured output, and can be called more than once without causing accidental duplication. Compare manage_customer_account with these operations:

lookupCustomer({ email });
draftRefund({ customerId, invoiceId, amount });
submitRefund({ draftId, idempotencyKey });

The second design exposes the irreversible boundary. The agent can look up information and prepare a draft, while policy code decides whether submission requires approval. idempotencyKey lets the application safely handle an uncertain response: if the network times out after the refund is accepted, retrying does not create a second refund.

Treat tool descriptions as part of the API. State preconditions, units, limits, and the meaning of each result field. Reject unknown fields rather than quietly ignoring them. If a tool returns prose where the next step expects a typed identifier, ambiguity leaks back into the workflow.

Give retries a budget and a reason

“Retry on error” is not a strategy. Some failures are transient, some require a different plan, and some indicate that the request can never succeed.

Classify failures at the boundary where they occur:

• Transport failures may justify exponential backoff with jitter.
• Rate limits should respect the provider’s retry window.
• Validation failures should return to planning with concrete feedback.
• Permission failures should stop and request authorization.
• Failed business invariants should not be retried unchanged.

Set budgets for attempts, elapsed time, tool calls, and cost. A run that keeps producing plausible next actions can otherwise continue far beyond its value. Store the retry reason with each attempt so operators can distinguish a flaky dependency from repeated bad planning.

Fallbacks should also be explicit. A smaller model, stale cache, or simplified workflow may be acceptable for generating a summary and unacceptable for approving a payment. Reliability includes refusing to degrade when the degraded result would violate the task’s safety or quality requirements.

Make progress observable

Raw model transcripts are useful for debugging, but they are not sufficient telemetry. Emit structured events around decisions and tool calls:

{
  "event": "tool.completed",
  "runId": "run_01J...",
  "step": 4,
  "tool": "lookupCustomer",
  "attempt": 1,
  "durationMs": 183,
  "outcome": "success"
}

Correlate every event with a run identifier and, where applicable, a trace identifier from downstream services. Record prompt and tool-schema versions, token usage, latency, retry count, and terminal status. Do not put credentials, raw customer data, or hidden reasoning into logs. Capture concise decision summaries and references to protected artifacts instead.

Useful service-level indicators follow from the workflow rather than the model alone: completion rate by task type, recovery rate after transient failure, approval frequency, invalid tool-call rate, time spent in each state, and verified-result rate. An overall success number can hide a workflow that performs well on summaries and poorly on actions.

Verify outcomes, not confidence

An agent saying “done” is a claim, not evidence. Completion should be decided by a verifier that examines the resulting state.

For code changes, the verifier might run type checks, tests, and a constrained diff review. For a support workflow, it might confirm that the ticket was updated, the response contains required disclosures, and no forbidden action occurred. For data extraction, it might validate a schema and reconcile totals against the source.

Prefer deterministic checks whenever possible. When judgment is unavoidable, use a separate evaluation step with a narrow rubric and structured result. A verifier should be able to return actionable failures such as missing_citation or total_mismatch, not merely a score of 0.7.

Also verify negative guarantees. “No external message was sent before approval” can be more important than whether the draft was eloquent.

Test the workflow at several layers

Agent tests become stable when they focus on contracts and invariants rather than exact sentences.

At the unit level, test state transitions, policy rules, tool validation, and retry classification. At the integration level, run recorded tool responses through the orchestrator. At the scenario level, evaluate representative tasks plus adversarial cases: missing context, conflicting instructions, dependency timeouts, duplicate callbacks, and attempts to smuggle instructions through retrieved content.

Build a small regression set from real task shapes, with sensitive data removed. Each case should define required outcomes, forbidden actions, and acceptable variation. Pinning every generated word makes tests brittle; checking that the workflow used an approved source, stayed within a tool budget, and produced a valid artifact tests what actually matters.

Run failure injection as well. Terminate a worker between an external write and a checkpoint. Return a timeout after a tool has committed its operation. Corrupt an optional context source. A resumable design should handle these cases deliberately rather than by luck.

Design for intervention

Human approval is not a patch for a weak system. It is a workflow state with a clear contract. The reviewer needs the proposed action, relevant evidence, risk, and a concise diff from the current state. Approval should be scoped to that exact action, not treated as permanent permission for the run.

Operators also need controls to pause, cancel, resume, and replay safely. Cancellation must prevent new side effects while allowing in-flight operations to settle and be recorded. Replay should reuse captured inputs and mocks unless the purpose is specifically to test current external state.

The production mindset is simple: assume individual decisions can be wrong, calls can be duplicated, dependencies can disappear, and processes can stop at inconvenient moments. Then make each condition visible and recoverable. The most reliable agent is not the one that never fails. It is the one whose failures remain bounded, diagnosable, and safe.