هندسة وكلاء ذكاء اصطناعي موثوقين لسير العمل المؤسسي

Building an AI agent that completes a task in a demo takes days. Building one that completes the same task every weekday, on messy inputs, against live systems of record, with an audit trail your risk team will sign off on, is an engineering program. The difference is not mainly the model. It is everything around the model.

A language model is a probabilistic component. Enterprise workflows expect deterministic outcomes: an invoice is either approved or it is not, a refund is issued exactly once, a customer record is updated correctly or left untouched. Reliable agents come from wrapping the probabilistic core in a deterministic envelope — explicit limits on what the agent may do, typed contracts for every tool it calls, checkpoints where people decide, and instrumentation that shows exactly what happened and why.

This article sets out the patterns we recommend for teams moving agents from prototypes into real business processes, along with the failure modes each pattern is designed to contain.

Bound the autonomy before you expand it

The most important design decision is not which model or framework to use. It is how much the agent is allowed to do on its own. We find it useful to assign every action an explicit autonomy level and to make that level a configuration value rather than a prompt instruction, so it can be changed without a redeploy and is enforced outside the model.

• Observe: read-only access to data and systems. Safe to run broadly, but still subject to data-access controls.
• Propose: the agent drafts an output or a changeset — an email, a journal entry, a ticket update — that a person or a deterministic process commits.
• Act with approval: the agent prepares a specific tool call, and execution waits for an authorized human.
• Act within limits: the agent executes directly, constrained by amount thresholds, rate limits, allowed targets, and a per-run budget.

Budgets belong in the same envelope. Every run should have a maximum number of steps, a token and cost ceiling, and a wall-clock timeout, all enforced by the orchestrator. Agents that loop — retrying the same failing call or re-planning indefinitely — are a common failure mode, and a hard step limit turns a costly incident into a logged exception.

It is also worth asking whether a process needs an open-ended agent at all. Many enterprise workflows are well understood: classify the request, gather context, decide, act, notify. Modeling that as an explicit state machine or graph, with the model making decisions at a few defined nodes, is easier to test and reason about than a free-form planning loop. Reserve open-ended planning for the parts of the work that genuinely vary.

Treat every tool as a contract

Tools are where agents touch the real world, so they deserve the same rigor as any public API. Each tool should have a narrow purpose, a strict input schema, validated arguments, typed errors the model can recover from, and documented side effects. Model output is untrusted input: validate it exactly as you would a request from an unknown client. The example below uses Zod for validation; the shape is illustrative rather than tied to a particular agent framework.

import { z } from "zod";

export const issueRefund = {
  name: "issue_refund",
  description: "Refund one settled order line. Not for disputes or chargebacks.",
  input: z.object({
    orderId: z.string().regex(/^ORD-[0-9]{8}$/),
    lineItemId: z.string().min(1),
    amountMinor: z.number().int().positive(),
    currency: z.enum(["USD", "EUR", "GBP"]),
    reasonCode: z.enum(["damaged", "not_received", "goodwill"]),
  }),
  // Supplied by the orchestrator, never by the model
  idempotencyKey: (ctx: RunContext) => ctx.runId + ":" + ctx.stepId,
  sideEffects: "write",
  autonomy: "act_with_approval",
  limits: { maxAmountMinor: 50_000, maxCallsPerRun: 1 },
  timeoutMs: 8_000,
} as const;

A few properties in that definition carry most of the reliability weight. The schema rejects malformed identifiers before any system is touched. The idempotency key, derived by the orchestrator from the run and step identifiers rather than generated by the model, means a retry after a timeout cannot issue a second refund. The autonomy and limits fields are read by the orchestrator, not the model, so a persuasive prompt cannot raise them.

Where possible, separate proposing from committing. The agent assembles a plan or changeset; a deterministic executor validates it against policy and applies it. For multi-step writes across systems, design compensating actions in advance — the saga pattern from distributed systems applies directly — so a partially completed run can be unwound cleanly.

Place human approval where it changes outcomes

Approval gates tend to be added everywhere or nowhere, and both are mistakes. Gates everywhere train reviewers to click approve without reading; no gates at all leave irreversible actions to a probabilistic component. The goal is a small number of meaningful gates, each easy to review well.

Good approval interfaces show the proposed action in concrete terms — the exact record, amount, recipient, and diff — alongside the evidence the agent relied on. They do not ask reviewers to read a reasoning transcript. Record every approval and rejection with the reviewer's identity and feed it back into evaluation data; rejections are among the most valuable signals you will collect.

Evaluate agents the way you test software

An agent without an evaluation suite cannot be changed safely, and agents change constantly: prompts are edited, tools are added, and model providers release new versions. Build a golden dataset from real cases, sanitized as needed, and include the awkward ones — ambiguous requests, missing data, conflicting records, and adversarial inputs such as instructions embedded in an email body.

Score at several levels. Outcome checks ask whether the final state is correct. Tool-call checks ask whether the agent chose the right tool with the right arguments. Trajectory checks flag unnecessary steps, repeated calls, and policy violations along the way. Prefer deterministic assertions wherever the expected result is knowable; use a model-based grader for qualitative judgments and calibrate it periodically against human labels. Because agent behavior is stochastic, run each case several times and track pass rates rather than single results.

Run the suite in continuous integration on every change to any of the following, and treat each as a versioned release artifact:

• System prompts and instruction templates
• Model identifier and sampling parameters — a provider's model upgrade is a release, not a background event
• Tool schemas, descriptions, and limits
• Retrieval indexes and the documents they contain
• Policy configuration, including autonomy levels and approval thresholds

Trace every run and design the way down

When an agent does something unexpected, the first question is always what it saw and what it did. Instrument each run as a trace, with spans for every model call and tool call, capturing inputs and outputs (with sensitive fields redacted), token counts, latency, cost, and the versions of the model, prompt, and tool definitions in play. OpenTelemetry's emerging semantic conventions for generative AI provide a vendor-neutral starting point and let agent traces sit alongside the rest of your distributed tracing.

Security failure modes need the same attention. Prompt injection and excessive agency both appear in the OWASP Top 10 for LLM Applications, and agents are exposed to both: any content the agent reads — a web page, a support ticket, a PDF — can carry instructions. Least-privilege credentials scoped to the user on whose behalf the agent acts, strict tool limits, and approval gates contain the damage when an injection succeeds, which is a safer design assumption than hoping it never will.

Define the fallback ladder before launch:

1. Retry transient failures with backoff, reusing the same idempotency key.
2. Return structured errors to the agent so it can correct arguments or choose another path, within the step budget.
3. Fall back to a narrower deterministic workflow for the common case.
4. Escalate to a human queue with the full trace attached, so nobody has to reconstruct context.
5. Fail closed: leave systems of record unchanged rather than guessing.

Pair this with operational kill switches — feature flags that disable a single tool, lower an agent's autonomy level, or route all work to people — so responders can contain an incident in seconds without a deployment.

Reliability is an operating discipline

Governance frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 are most useful when they map onto artifacts engineering teams already produce: an inventory of agents with owners and autonomy levels, evaluation reports attached to each release, retained traces, and incident reviews. Cost control fits the same pattern. Per-run budgets, caching of stable context, and routing simple classification steps to smaller models are design decisions, not afterthoughts.

We recommend starting narrow: one workflow, clear success criteria, conservative autonomy, and thorough instrumentation, then expanding autonomy as evaluation evidence accumulates. Reliability is not something a better prompt delivers. It is designed into the system, measured continuously, and earned one workflow at a time.