Engineering

Long-Running Agents Lose the Plot

On Monday, an agent starts comparing supplier proposals. On Tuesday, the buyer changes the delivery requirement. On Wednesday, the agent returns a polished recommendation based on Monday’s instructions. Every individual step looks reasonable. The finished work answers the wrong question.

This is a hypothetical workflow, but it exposes a useful engineering test. Can an agent keep the current objective intact across interruptions, context compaction, and changes from the person directing the work? A longer context window gives it more room. It does not establish which instruction is current.

Keep the task outside the transcript

A conversation contains requests, abandoned ideas, tool output, corrections, and small status questions. Those records are useful evidence. They are a poor substitute for an explicit description of the work that remains.

Suppose the buyer asks for a comparison of four proposals for replacement equipment. The agent needs to extract pricing, check installation requirements, and produce a recommendation. The buyer has authorized analysis only. Nothing in that assignment permits placing an order or emailing a supplier.

Persist a task record before the workflow grows. Give it an ID and a revision. Store the objective, acceptance criteria, applicable constraints, authority, completed work, and unresolved questions. Keep source references beside the facts they support. Each pending step should name the evidence or decision needed to complete it.

The transcript can explain how the task reached revision 4. The task record should tell the next execution what revision 4 requires. Do not make the model reconstruct that distinction from a summary of several days of conversation.

A successful resume restores the current assignment, including what the agent is still not authorized to do.

Treat a changed instruction as a state change

On Tuesday, the buyer says: “Installation has to finish before October 1. Keep the original budget. Exclude suppliers that cannot confirm that date.” This changes the selection criteria. It does not cancel the comparison or expand the agent’s permission to contact anyone.

Record the instruction with its source and time. Advance the task revision. Mark the old delivery criterion as superseded, and invalidate any ranking that depended on it. The extracted prices may still be usable. The recommendation now needs another pass. Store that dependency explicitly so a later run does not treat every completed step as permanently settled.

Missing confirmation is not confirmation. If a proposal has no installation commitment, record it as unconfirmed and exclude it under the revised rule. If no supplier qualifies, the correct result is a documented blocker or a recommendation to revise the requirement. Quietly relaxing the date produces an answer the buyer did not authorize.

A status question has different semantics. “How far along are you?” should produce a short progress update without replacing the comparison task. “Stop the comparison” should halt further execution. Classify the incoming message against the active task, and preserve uncertainty when the meaning is genuinely ambiguous.

Compact the conversation without flattening the evidence

Context compaction reduces the material carried into a later model call. A summary might retain “Supplier B can deliver in September” while dropping the fact that the statement concerned shipping, not completed installation. That loss changes the decision.

Keep the original proposal in durable storage. Reference its version and page or section from the extracted claim. Distinguish source statements from calculations and assumptions. “Ships September 20” and “installed before October 1” must remain different fields unless evidence connects them.

Before resuming, load the current task record and retrieve the evidence relevant to the next step. Check whether source versions changed while the agent was paused. If a proposal was replaced, flag the dependent extraction for review instead of silently combining the new document with old facts.

The handoff summary can remain short: current revision, next action, blockers, and pointers to supporting records. A source pointer is only useful if the resumed worker can resolve it with its existing permissions. An expired link or missing attachment should surface as missing evidence, not become an invitation to fill in the blank.

Check revisions before committing results

Now introduce concurrency. One worker is scoring proposals while another receives the buyer’s updated deadline. The scoring worker may finish correctly against an obsolete task revision.

Attach the input task revision to each work item and result. Before accepting the result, compare that revision with current state. If relevant requirements changed, reject the stale ranking and schedule a new evaluation. Use an atomic version check when writing state so two workers cannot both overwrite it based on the same old revision.

Reuse unaffected work deliberately. A corrected installation deadline need not trigger another extraction of unchanged price tables. Track dependencies at a level that makes this practical. Too little structure wastes work or accepts stale answers; excessive structure makes a small workflow expensive to maintain.

External actions need their own records. If a workflow eventually includes sending an approved message, persist an action ID and reconcile its outcome after a timeout. A summary saying “email pending” is insufficient when the provider may already have accepted it. Task memory does not replace retry handling or duplicate prevention.

Test the interruption, not just the final paragraph

Build a fixture with four synthetic proposals and an expected eligible set. Make one proposal cheap but late. Give another a shipping date without an installation commitment. Keep the documents local and fixed so failures are attributable to the workflow, not changing supplier data.

  1. Start the comparison under the original requirements and save a checkpoint after extraction.
  2. Apply the buyer’s revised installation deadline while a ranking job is still in progress.
  3. Force compaction and restart the worker from durable state without its previous in-memory conversation.
  4. Submit the old ranking result, ask a status question, and let the current task continue.
  5. Remove one referenced document in a separate run and verify that missing evidence is reported.

Assert that the stale ranking is not accepted. Check that the budget survives, unconfirmed installation dates remain unconfirmed, and the final recommendation uses the revised eligibility rule. Confirm that the status question did not cancel the assignment and that no supplier was contacted. These checks are more useful than grading whether the final answer sounds confident.

Run the fixture at multiple interruption points. Record stale-result acceptance, unsupported claims, unauthorized actions, and completion against the latest acceptance criteria. Also measure recovery time and repeated tool calls. A workflow that eventually succeeds by starting over every time may still be too costly for its intended workload.

Start with one workflow that crosses a day boundary

A short, read-only task may need little more than a saved request and its output. Multi-day work with changing requirements deserves a versioned task record, durable evidence, and an explicit resume procedure. Choose the complexity based on the failure you need to prevent.

Take one existing workflow and pause it after a meaningful intermediate result. Change a requirement. Restart with the conversation compacted. Inspect which objective it finishes, which evidence it cites, and which actions it attempts. Keep that case in the regression suite.

Foundation AI designs and deploys agent workflows with the surrounding software they need to operate. Bring us a workflow that spans multiple days, and we can help define its task state, recovery behavior, and first interruption test.