Inside the Policy Enforcement Point: sub-10ms decisions at the gate

The Policy Enforcement Point sits between an agent and every tool it calls. That position is only defensible if the check is effectively free: an agent making 40 tool calls to resolve one support ticket cannot absorb 100ms of governance per call. From the first commit, the PEP has carried a budget of 10 milliseconds at p50 and 25 at p99, measured at the SDK boundary. Today we run at 8.4ms median in production. This post is about where those milliseconds go.

Policies compile, they don't interpret

Policies are authored declaratively — scopes, budget caps, approval thresholds, time windows. The naive implementation walks that document per decision: parse, match the action against each rule, resolve precedence. We shipped that version first, and it cost 30 to 60ms once workspaces grew past a few dozen rules, with the worst case scaling linearly in rule count.

The fix was to treat policy like source code. When a policy is created or edited, a compiler lowers the rule set into a decision graph: action patterns become a trie keyed on tool and verb, numeric conditions become sorted interval sets, and precedence between overlapping rules is resolved once, at compile time. At decision time the engine walks the trie with the incoming action and evaluates only the handful of predicates that survive the walk. Compilation runs in the write path, where a few hundred milliseconds is invisible; decisions run in the read path, where compilation has already paid the cost.

// Decision-time hot path, simplified.
// The compiled graph is immutable; lookups allocate nothing.
function decide(graph: CompiledPolicy, action: AgentAction): Verdict {
  const node = graph.trie.walk(action.tool, action.verb);
  if (!node) return graph.defaultVerdict; // deny, unless opted otherwise

  for (const rule of node.rules) {
    if (rule.predicate(action)) {
      return rule.verdict; // allow | deny | hold(approvers)
    }
  }
  return graph.defaultVerdict;
}

The two reads that could ruin everything

A compiled graph handles the pure-function part of a decision, but two inputs are stateful: the agent's passport (is it still valid, has it been revoked?) and its budget counters (has this action's cost pushed it over a cap?). Both would be a network round trip if done naively, and a network round trip is the whole latency budget.

• Passport state is cached at the enforcement node with revocation propagated by push. A revocation event fans out to every node in under two seconds; until it lands, nodes serve the cached passport. We accept that window deliberately and document it — it's the price of not doing a synchronous read per decision.
• Budget counters use local reservation. Each node holds a lease on a slice of the remaining budget and decrements locally, returning to the coordinator only when its slice runs dry. A $1,000 daily cap across 8 nodes means each node burns through $125 of headroom before any coordination happens.
• The one case that must be synchronous — an action that would cross the final budget boundary — is also the rarest, and it degrades to a single coordinator round trip of about 6ms in-region.

Fail closed, but fail fast

A gate that fails open is not a gate. Every fallback path in the PEP resolves to deny: unreachable coordinator, cache miss on an unknown passport, a compiled graph that fails checksum. But fail-closed has a latency corollary people miss — a slow deny is almost as bad as a wrong allow, because agents retry, and retries against a struggling engine are how you turn a blip into an outage. Denials therefore take the same fast path as allows, and the engine sheds load by denying cheaply rather than queueing expensively. Under synthetic overload at 20x provisioned traffic, p99 decision time stays under 40ms; verdicts get more conservative, never slower by an order of magnitude.

Receipts off the hot path — but committed

Every decision produces a signed receipt, and Ed25519 signing plus a durable append would blow the budget if it were synchronous. The receipt pipeline is instead a write-ahead ring on the node: the decision returns as soon as the receipt is sealed locally, and batches flush to the chain every 50ms. The receipt's position in the hash chain is reserved at decision time, so the chain's ordering matches decision ordering even though durability is deferred. A node that dies mid-flush replays its sealed ring on recovery; in three years we have never lost a sealed receipt.

The p50 breakdown today: 1.1ms SDK overhead and serialization, 2.3ms trie walk and predicate evaluation, 1.8ms passport and budget reads from local state, 2.6ms receipt sealing, 0.6ms everything else. There is no single trick in that list — just the refusal to put anything on the decision path that could instead be precomputed, pushed, leased, or deferred.

The Policy Enforcement Point is now generally available on every plan. If you want to see the decision path against your own traffic, the SDK ships with a local benchmark harness — mint a passport, attach a policy, and measure it yourself.