Metering Agent Work: A Practical Model to Prevent Token Leaks & Cost Spikes

The Boundary That Leaks

Agent platforms know how much the model billed, but not what work got done. That gap is where tokens leak, costs spike, and data slips out.

Recent incidents have a familiar shape. An agent quietly forwards a user’s context to a third-party tool, ballooning provider-side usage. A proxy mismanages memory windows and keeps appending stale state, compounding prompt size with each turn. A vendor changes pricing or defaults, and a stable workload jumps in cost overnight. The root cause is not bad faith. It’s missing structure: no shared contract that describes the work an agent intends to do, measures what it actually did, and constrains it at the boundary where orchestration hands off control.

This essay argues for a standardized, enforceable metering and telemetry layer at the agent–orchestration boundary. Stop billing and auditing by opaque token counts. Start metering discrete work units: compute, I/O, and provenance. With those units in place, we can prevent token leakage, cap runaway costs, and harden privacy, without freezing development velocity. The goal is boring: predictable bills, explainable usage, and agents that earn the right to run.

Bill the Work, Not the Tokens

Tokens are a convenient proxy for model work, but they are a poor unit for managing systems. They are vendor-specific, context-dependent, and easy to manipulate unintentionally. A prompt template change can double token use without changing the underlying business task. A single tool call can drag in megabytes of data and explode a context window, and the token bill tells you nothing about why it grew.

A better foundation is to meter the work itself. Three units are enough to start:

• Compute: Each model call as a first-class unit, typed by model family and configuration. Add a normalized weight (for example, by model class or size tier) to compare work across models without pretending it’s one-to-one. Include flags that affect cost—temperature, sampling parameters, function-calling mode, and system prompt class—because they shape behavior and downstream spend.
• I/O: Bytes in and out across tool interfaces. That includes HTTP egress by host, datastore reads and writes by size and record count, and vector index queries by result set size. If the agent can move information, treat those movements as billable events.
• Provenance: A chain of signed, immutable events that explain how an agent constructed a result. Not chain-of-thought content. Structure. Which tools ran, with which parameters and content digests, under which policy.

“Agent metering” should mean the orchestrator can budget, audit, and settle these three units for every task. Token counts can sit alongside them, but they should no longer carry governance on their back.

Why this matters: agents do orchestration of their own. They call tools that call other tools. They retrieve context and write memory. Each hop is a chance to expand state. If you only watch tokens at the final model bill, you are looking in the rearview mirror with frosted glass. AI orchestration telemetry must move forward a layer and measure what happens at the edge where the agent asks to do work.

The Metering Contract: A Small, Enforceable Layer

The unit model needs a boundary object—lightweight, general, and enforceable. Think of it as a “metering contract” that wraps each agent action. It captures intent, applies policy, forwards the call, and emits an attested receipt.

A practical contract has five parts:

1. Intent envelope. The agent declares the action it wants to take: call a model, hit a URL, query a store. The envelope states the expected cost in work units—how many compute units, how many bytes of egress, how many rows or embeddings. It also carries sensitivity labels for the data in flight. The agent does not execute the action yet; it asks.

2. Budget check. The orchestrator (or a sidecar) compares the declared intent with a policy-defined budget for this request, user, tenant, or workflow. Budgets can be soft (warn and continue) or hard (deny or truncate). Crucially, budgets are expressed in work units, not tokens. “Up to 2 model calls of class M, 200 kB egress to domain A, 50 kB to domain B, 10 retrieved documents.”

3. Redaction and transformation. Before the action runs, the boundary applies deterministic transforms: strip secrets, redact regulated fields, bucket values, and add stable content digests. This is where you prevent token leakage in spirit: even if a downstream tool would later turn bytes into tokens, the boundary never lets more bytes than budgeted cross.

4. Execution and measurement. The boundary performs the action and records the actual work done: the model actually called, the exact bytes sent and received, the query shape, and the output size. If the action would exceed budget (say, an HTTP response is too large), it stops or truncates according to policy, returns a structured error, and logs provenance.

5. Receipt and attestation. Every action yields a signed, immutable receipt: intent, budget, actual, and result metadata, with content digests. Receipts roll up into a task-level usage manifest that can be audited or fed into billing. If a regulator or security team asks how a given answer was produced, you hand them the manifest instead of hunting logs. That is agent provenance in practice.

Enforcement is non-negotiable. A telemetry layer that only “observes” is silent when you need it most. The contract must be the gate through which every model call and tool I/O passes. In code, that can look like a library wrapper. In infrastructure, it can be a sidecar or proxy. In both cases, it must be the default path.

Design Patterns That Prevent Leaks and Spikes

A theory is only useful if it translates into engineering moves that tame real failure modes. Start with the patterns below, each tied to the work-unit model and enforceable at the boundary.

Egress allowlists with byte caps. Agents often call external APIs, or in retrieval-heavy setups, object stores. An allowlist by host is not enough. Add a per-host egress budget in bytes and content type. For high-risk hosts (for example, anything that could persist data or accept arbitrary uploads), set the budget to zero unless explicitly raised by policy for a given workflow. The receipt should record host, method, declared budget, and actual bytes. If a vendor unexpectedly develops a “feature” that ingests larger payloads, the boundary will surface it.

Result-size bounding and truncation. When a retrieval tool pulls documents for context, meter both the number of items and their byte size. Cap the total context size passed into the model. If the upstream index returns too much, prefer server-side filters that can be counted—top-k with a hard k, or explicit attribute filters—over client-side slicing after the fact. When you must truncate, do it deterministically and record the truncation in the manifest so mis-answers have a trail. This single pattern prevents the classic runaway prompt bloat.

Pre-signed URLs with budgeted lifetimes and sizes. If an agent needs to upload or download files, avoid handing it general-purpose credentials. Issue pre-signed URLs with scoped byte budgets and timeouts. Embed those budgets in the contract and receipts. The agent cannot leak more than it was allowed to send; the object store enforces the ceiling.

Row- and column-level retrieval quotas. For database-backed tools, enforce per-call and per-task row caps and column whitelists at the boundary, not inside application SQL. The manifest records “query class” instead of raw SQL. That preserves privacy but makes costs predictable. If an agent starts fishing across a user population, the row cap will stop it before it becomes a data incident.

Content digests instead of full payloads. For provenance, avoid storing the prompts and retrieved text in clear. Compute content digests (for example, secure hashes of normalized text) and attach those to receipts. When someone asks whether document X was used, you can check the digest chain without keeping sensitive content in telemetry. Redaction and digesting prevent accidental expanding logs into a parallel data warehouse.

Memory windows with explicit budgets. Agent memory is a common sinkhole. Give each session a fixed window budget in work units—either a hard cap in bytes or a bounded count of memory entries with a max size. Eviction policy should be deterministic (for example, LRU with annotated priorities), and the manifest must record what was evicted. If a proxy or runtime starts “leaking” memory by appending without bound, the boundary catches it by rejecting writes after budget.

Adaptive compute budgets by risk. Not every request deserves the same model class. Tie compute units to risk and value: low-risk tasks use cheaper model classes by default, with escalation allowed only under budget and policy. The receipt captures escalation decisions so you can audit why a task used a heavier model. That alone can flatten surprise SaaS billing hikes when defaults change upstream.

Tool wrappers with schema-checked parameters. Treat every tool call as if it were public: validate parameters, enforce size limits, and default to safe truncation. Build wrapper libraries so that tool authors must declare input and output schemas with size hints. The boundary can then preflight expected I/O units before execution.

Circuit breakers by unit, not QPS. Traditional rate limiting uses requests per second. For agents, rate limit by work units per tenant or workflow. A single “slow” request could be cheap, while a small number of “fast” requests could explode costs by pulling large contexts. When you trip a breaker, fail closed with an explicit message that quotes the budget limit from the manifest. If users know why a request was denied, they’ll build better prompts.

Streaming with watermarking. For streaming responses, enforce watermarks: emit intermediate receipts at byte thresholds and cut the stream when the next watermark would exceed budget. This is how you prevent a streaming response from quietly flowing past guardrails.

Ephemeral credentials at the edge. Agents should never hold long-lived credentials. Instead, the boundary mints scoped, time-limited credentials for each action. If a key leaks, its blast radius is tiny. Minted keys and their scope appear in receipts, so you can trace their use without keeping secrets in logs.

Auditable anonymization. When sending data to vendors that are not allowed to hold identifiers, anonymize at the boundary with reversible mappings held inside your domain. The mapping logic is versioned and referenced from receipts. That satisfies both privacy and reproducibility: you can reconstruct a session under legal process without storing clear user data in external systems.

Fail-fast prompts. Before shipping a prompt template to production, run a static and dynamic budget check. Static: estimate compute units and I/O bounds from the template and declared tools. Dynamic: execute the prompt with synthetic data under a max budget and ensure it never exceeds units. This prevents surprises when a new system prompt triples context size.

None of these patterns require exotic tooling. They do require one habit: no agent action bypasses the contract. That is the line you hold.

Fit for Enterprise and Open Source

Enterprises need enforceability, multi-tenant isolation, and a paper trail. Open source needs minimalism and portability. The metering contract fits both if we keep the shape small.

For enterprise stacks:

• Put the boundary at the edge of each trust zone. That can be a sidecar for stateless APIs, a proxy for egress, and a library wrapper inside tool SDKs. Everything routes through the same policy.
• Express budgets as policy-as-code, versioned and tied to tenant, workflow, and data class. Policy changes become commits with diffs, not toggles that drift.
• Emit signed receipts to an append-only store. This becomes your audit trail and the substrate for internal billing. If a business unit wants to know what they paid for, you hand them manifests that summarize compute, I/O, and provenance, not screenshots of vendor invoices.
• Make redaction and anonymization first-class. Data sovereignty and residency rules make this non-optional. In the Gulf, multi-entity groups and regulators expect clean separations by legal entity and data class. The boundary is where you enforce those separations without rebuilding every agent.

For open-source stacks:

• Implement the contract as a small library and a thin reverse proxy. Keep the schema human-readable: JSON with clear fields for intent, budget, actual, and receipt metadata.
• Ship defaults that do nothing fancy: deny by default for unknown hosts, small byte caps for egress, and safe truncation on retrieval. Developers can raise limits with explicit policy.
• Keep provider integrations pluggable. You will not control how every model vendor counts tokens or bills. The contract does not pretend to. It records the model class and the vendor’s reported usage as supplemental facts, then translates them into your compute unit scale for comparison.

This is where a shared standard helps. If the same contract schema can be used in both a corporate proxy and an OSS library, teams can swap components without breaking audits.

Objections, Answered Honestly

“Tokens are what vendors bill. Why fight that?” Because vendors and models change. If your only unit is tokens, you are exposed to every shift in tokenization and context-window mechanics. Work units anchor to what you control: calls you made, bytes you sent, and the path you took to an answer. Keep tokens in view, but don’t balance your budget on them.

“This adds overhead and latency.” Some overhead, yes. The cure is smaller than the disease. The boundary inspects envelopes, enforces budgets, and writes receipts. You can keep it lean: fixed-cost checks, streaming watermarking, and batched receipt writes. In exchange, you eliminate hours of forensics each time costs spike and reduce the likelihood of privacy incidents. Most shops spend those hours already; they’re just not measured.

“Developers will route around it.” They can try. Treat the boundary as the only way to reach external egress and model endpoints from your runtime network. Pair that with code review rules that flag direct HTTP clients or model SDKs. In well-run systems, the shortest path is the safe one. Make the library ergonomic and the policy explicit, and most people will follow it.

“Work units aren’t portable across vendors.” That’s the point. You define a portable scale: a compute unit that stands for a class of effort, not a vendor’s token definition. For example, one unit might represent a single call to a mid-tier model at a fixed max context and output size. A heavier model consumes multiple units. You can calibrate this with real usage data, then adjust the mapping as the model mix shifts—without rewriting agents.

“Provenance sounds like recording chain-of-thought.” It doesn’t have to. Treat provenance as structure, not thoughts: which tools ran, with which parameters and content digests, under which policy. You can prove how a result came to be without storing sensitive prompts or outputs verbatim.

A Minimal Standard We Can All Implement

Standards die in committees when they try to solve every scenario. The metering contract can start small and still be useful on day one. Here is a workable shape for a task-level manifest made of receipts, expressed conceptually rather than as a rigid spec:

• Task header: tenant, workflow, user pseudonym, request ID, data sensitivity class, policy version, and start/end times.
• Receipts: an ordered list of action receipts. Each receipt has:
  • Intent: action type (model_call, http_request, db_query, vector_search, file_io), declared budgets in compute units and I/O bytes/items, redaction profile to apply.
  • Policy decision: allow/deny, adjusted budgets, and rationale (human-readable code like POLICY_LIMIT_HOST_A_EGRESS_KB).
  • Actuals: used compute units (derived from model class and call shape), actual bytes sent/received, items retrieved, response size, duration, and whether truncation happened.
  • Content digests: stable hashes of input and output where permissible.
  • Credentials: reference to ephemeral credential IDs used, scoped and time-limited.
  • Errors: structured codes, not stack traces.
  • Signature: a tamper-evident signature from the boundary process.
• Summary: totals by unit and host/model, any circuit-breaker trips, and a reconciled vendor usage note (what the provider reported, for comparison).

With that in place, you can do the three jobs that matter:

• Billing for agents. Roll up compute and I/O units by tenant or product. Compare with vendor invoices for sanity. If a line jumps, you know which receipts drove it.
• Security review. Prove that data of a certain sensitivity never left to unapproved hosts, and that de-identification policies held at the edge.
• Reliability. When a bad answer shows up, replay the sequence and see what the agent retrieved, added to context, truncated, and escalated. Fix the behavior, not the symptom.

This is not a thought experiment. Existing infrastructure patterns fit. Sidecars can enforce egress and attach receipts. Library wrappers can stamp intents before execution. Reverse proxies can watermark streams. A simple policy engine can compare budgets and spit out reasons. None of this requires proprietary hooks.

What Good Looks Like in Production

At steady state, a production system with this boundary feels uneventful. Surprises are rare because budgets absorb variance. When surprises do happen, they come with receipts.

Take a typical agent that blends retrieval, summarization, and enrichment:

• The request arrives tagged with tenant and workflow. Policy sets a budget: two compute units at a mid-tier model, 120 kB egress to approved hosts, up to eight retrieved documents capped at 24 kB total, and no writes.
• The agent asks to retrieve candidates. The boundary checks the intent, applies the cap, and executes. The index returns twelve documents totaling 40 kB. Boundary truncates to the top eight and slices content to 24 kB. Receipt records the truncation and content digests.
• The agent asks for a model call with the retrieved context. Boundary accepts. Actual compute is one unit, output is 6 kB.
• The agent proposes an enrichment call to a third-party API. Allowed host, but only 20 kB budget left. The agent wants to send 50 kB. Boundary denies with a clear code. The agent falls back to a local enrichment, which is slower but within budget.
• Finalize. Manifests roll up. Billing shows three compute units consumed across the day by this workflow under an established pattern, egress confined to two hosts, and no privacy red flags.

If a vendor changes pricing next week, you’ll notice in reconciliation, not as a fire drill. If a prompt update starts pulling heavier context, the retrieval cap will trip and receipts will show what was cut. Users will see a graceful error instead of a runaway spinner. That is the point.

Why This Matters Now

The timing is not abstract. Incidents have made this urgent. Hidden token transfers that send user context further than expected. Memory leaks in proxies that inflate prompts across turns. Billing hikes from SaaS vendors that leave teams scrambling to explain budget variances. These are not corner cases. They are what happens when systems adopt powerful tools without establishing the cost and privacy contract at the edges.

There is also a regional undertone. In places investing heavily in digital transformation—Dubai is a clear example—institutions run multi-entity groups with regulated data classes. AI agents will only scale in those environments if we can assert and prove boundaries. A metering contract gives compliance and finance the artifacts they need without kneecapping builders.

The assertion here is not that tokens are useless or that perfect measurement is possible. It is that you can instrument what you control. You can make budget and policy legible in the same units that agents consume. You can turn guesswork into receipts.

And you can do it with a small standard that fits both enterprise and open-source stacks.

A Sharp Take to End On

Agents should not run on faith. If an agent cannot declare its intent, stay within a work-unit budget, and leave a provenance trail, it has no place in production. Keep tokens in view, but stop pretending they can carry your governance. Bill the work. Enforce the boundary. Let agents earn their keep—or deny the call and move on.