A model that emits SQL has proposed an action; it has not earned production execution authority. Even a SELECT can scan terabytes, expose sensitive columns, invoke an external function, or return a plausible result after an incorrect join.
The execution layer converts model intent into a controlled operation: validate before running, constrain during execution, verify afterward, and retain evidence for every side effect.
Generation and authorization are separate acts. A model may propose “create a ticket” or “run this query.” A trusted gateway decides whether the identity, policy, parameters, and risk allow it.
Use a structured execution contract
An executor should not receive free-form prose. A minimum request includes task and step IDs, caller and delegated identity, tool and version, structured parameters, input artifacts, semantic version, allowed operation, budget, timeout, idempotency key, approval reference, and output contract.
The result needs a contract too: status, execution ID, result reference, row count and truncation, cost, schema, watermark, warnings, retry eligibility, and audit reference. The model receives only what the next decision needs. Credentials, internal stacks, and unrelated sensitive data remain in the trusted runtime.
Three gates before execution
Syntax and static validation
Parse SQL or code into an AST and reject unsupported statements, dynamic execution, unbounded targets, and dangerous libraries. Searching a string for DROP is not a safety control; writes can hide in stored procedures, multi-statement requests, external functions, and tool parameters.
Semantic and data validation
Check that tables, fields, metric version, grain, join cardinality, and time scope match the plan. The executor does not redefine business meaning, but it must verify that referenced contracts still exist, versions remain compatible, and required inputs are ready.
Policy and resource validation
Authorize resource, row and column scope, purpose, region, environment, and read/write type under the end user or a narrowly scoped service identity. Then estimate scanned data, runtime, concurrency, and external-call cost. Unknown cost is not free; it requires a tighter limit or approval.
A sandbox is a collection of controls
Different tools need different isolation:
- SQL: read-only transactions, object allowlists, timeout, scan limits, result limits, and sensitive-column policy.
- Code: ephemeral containers, pinned dependencies, CPU/memory/disk quotas, no default egress, and only task-scoped files.
- Files: a bounded working directory, type and size limits, content scanning, and path-traversal prevention.
- External APIs: allowed destinations and actions, short-lived credentials, validated parameters, rate limits, and idempotency.
The environment must also be reproducible. Record the image, dependency versions, random seed, inputs, and tool version. Otherwise “the code ran” cannot become a reviewable result.
Success at the tool is not success for the task
HTTP 200, exit code 0, or a completed query says only that the tool did not report an execution failure. Verify that the output schema matches the contract, results are not silently truncated, the watermark meets the task, invariants hold, and units and denominators are correct.
For expensive queries, use EXPLAIN or a bounded probe before the full run. For engineering changes, generate a diff, test it, perform impact analysis, seek authorization, apply the change, and read back the target state. Generated, submitted, and effective are distinct statuses.
Retry semantics depend on side effects
A deterministic read may be retried after a transient failure within a limit. A write needs a logical operation ID and proof that the downstream service supports idempotency. Timeout does not mean failure. Retrying an unknown operation can create two tickets, send two emails, or write the same record twice.
Classify failures: invalid caller input, policy denial, resource limit, retryable infrastructure fault, deterministic tool failure, result-verification failure, and unknown state. Each requires a different transition; “let the model try again” is not a recovery strategy.
Audit what matters—and avoid logging secrets
An audit event links task, user, delegated identity, plan step, tool version, parameter digest, policy decision, approval, execution ID, result reference, and final state. A high-risk action retains the exact approved parameter snapshot.
Ordinary logs should not contain secrets, full sensitive outputs, or prompts with no retention need. The audit service itself requires access control, integrity protection, and retention policy. Having logs does not mean they are sufficient for accountability.
Minimum release tests
Before production, verify that:
- unauthorized tables, columns, rows, and tools are blocked;
- comments, aliases, subqueries, and multiple statements cannot bypass static rules;
- scan, timeout, row, and concurrency limits work;
- the sandbox cannot reach unauthorized files, networks, or environment variables;
- repeated idempotency keys do not duplicate side effects;
- an unknown state is queried before retry;
- cancellation stops the execution, not only the chat;
- truncation, empty sets, missing data, and validation failure remain visible.
Execution-layer maturity is not the number of tools an agent can call. It is the ability to answer, for every action: who authorized it, where it ran, what limits applied, what it produced, and how failure was contained.