The most consequential AI architecture decision is often made before a model is selected: should the system follow a defined workflow, or should an agent decide what to do next? Both can use the same models and tools, yet they create very different operating risks. Treating an agent as a more capable workflow usually hides those risks until production traffic exposes them.
The practical goal is not maximum autonomy. It is the minimum autonomy required to handle the variability of the job. Product and engineering leaders should choose an architecture that can be explained, measured, and constrained before adding freedom that the organization cannot yet supervise.
Workflow and agent are different control models
A workflow encodes the sequence: classify a request, retrieve data, generate a draft, validate it, then route it for approval. A model may make bounded decisions inside a step, but the application owns the control flow. This makes execution paths enumerable, tests repeatable, and cost and latency easier to forecast.
An agent owns part of that control flow. Given a goal, available tools, and constraints, it plans or selects the next action, inspects the result, and may revise its approach. That flexibility is valuable when valid paths cannot be listed in advance. It also expands the state space: the same request may trigger different tools, orders, retries, and stopping points.
Start with the shape of uncertainty
Use a deterministic workflow when the business process is stable, the permitted actions are known, and exceptions can be routed explicitly. Prefer an agent only when uncertainty lies in the path itself: research across changing sources, diagnosis that requires iterative tool use, or work where intermediate evidence must determine the next step. A variable input does not automatically require variable control flow.
Four questions expose whether autonomy will create net value:
- Can the acceptable execution paths be described and versioned without excessive branching?
- Does choosing the next action require interpreting new evidence rather than applying a fixed rule?
- Can every tool call be scoped, observed, reversed, or safely compensated?
- Is the value of handling edge cases greater than the additional testing and operating cost?
Observability must follow decisions, not just requests
Traditional request logs are insufficient once the system can choose its own path. Capture the goal, model and prompt version, retrieved context, decisions, tool inputs and outputs, policy checks, retries, token and monetary cost, elapsed time, and termination reason. Traces should reconstruct why an action occurred without relying on hidden model reasoning. Record concise decision summaries and observable evidence instead.
Define a failure budget before launch. Include more than technical errors: wrong actions, unsupported claims, unnecessary escalation, policy violations, abandoned runs, and cost or latency overruns all consume the budget. Segment these measures by task and consequence. A one percent failure rate may be tolerable for draft categorization and unacceptable for changing a customer entitlement.
Autonomy is an operating liability until its decisions are observable, bounded, and recoverable.
Put approval gates at the boundary of consequence
Approval should be driven by impact, not by whether AI was involved. Let the system read broadly but write narrowly. Low-risk, reversible actions can proceed automatically within rate, spend, and scope limits. Actions that move money, alter permissions, contact customers, publish externally, delete data, or create legal commitments should require explicit approval until evidence supports a narrower gate.
A useful gate presents the proposed action, supporting evidence, affected resources, expected result, and rollback plan. It should not ask a reviewer to reread an entire transcript. Separate permission to plan from permission to execute, and issue short-lived, least-privilege credentials only after approval. This turns human review into a control, not theater.
Migrate by transferring one decision at a time
Begin with a workflow and make its contracts explicit: typed tool interfaces, policy checks, idempotency, timeouts, evaluation cases, and complete traces. Then identify the branches creating the most maintenance burden or missed outcomes. Allow a model to recommend the branch in shadow mode, compare it with production decisions, and promote it only after it meets task-specific quality and failure budgets.
Next, delegate a bounded loop with a maximum number of steps, restricted tools, explicit stop conditions, and a fallback to the existing workflow. Expand scope only when telemetry shows a durable benefit. This migration path preserves a known-good system while building the operational discipline agency requires.
The right architecture may remain hybrid: deterministic orchestration around a few agentic decisions. That is not an interim compromise. It is often the production design that best aligns flexibility with accountability. Choose autonomy where uncertainty demands it, retain workflows where predictability creates value, and make every increase in freedom earn its place.
