Make the policy decision point pluggable (the rules are injectable, the evaluator is not) #350

Activity

1. Hotragn commented on Sep 14, 2026

   

   Hotragn

   on Sep 14, 2026

   Contributor

   More actions

   The diagnosis holds — I went through all of it while adding a field to PolicyContext in #526, so
   here is what the seam actually looks like from inside, and one thing in the proposal I think is a
   harder problem than it reads.

   The seam is three call sites, and two of them are free

   evaluateActionPolicy is called in exactly three places:

   computer/gateway.ts:532	inside async function govern<T>
   plugins/store.ts:2967	inside async callTool
   computer/policy-dry-run.ts:189	inside dryRunAgainstHistory, not async

   So a PolicyDecider returning Promise<PolicyDecision> costs nothing at the two that decide live —
   both are already async and already awaiting a provider or a vendor. The injection point is the same
   one options.policy() already uses, which is the part of your argument I would keep front and
   centre: the data is injectable and the evaluator is a direct import, and there is no reason for that
   asymmetry.

   Worth knowing if you pick this up: PolicyContext gained an initiator field last week (#526,
   {kind, id} — person, deployment, routine or handoff). A plug receives the context, so that is one
   more thing it gets for free, and the shape is settled rather than in motion.

   The hard part is the third call site, and it is not just async plumbing

   dryRunAgainstHistory is how #241 lets somebody test a rule against real history before saving it.
   It replays recorded actions through the same evaluator the gateway used, which is the whole reason
   the answer means anything.

   A pluggable evaluator breaks that, and not for a mechanical reason:

   • Case 2 in your list — "this bot has already submitted three orders today" — is a decision
     about state at a moment. Replaying it against last Tuesday's rows asks today's counter about last
     Tuesday's action, and the answer is confidently wrong rather than unavailable.
   • Case 3 — human in the loop — cannot be replayed at all. You cannot re-ask somebody what they
     would have approved a week ago, and a replay that waits on a human is not a replay.

   So an evaluator that serves cases 2 and 3 is exactly an evaluator whose answers are not reproducible,
   which is the property the dry-run depends on. Making the interface async is easy; deciding what
   dryRunAgainstHistory does when the deployment has plugged in a non-replayable decider is the
   design question.

   Three ways out, and I have no strong view:

   1. The interface says so. A decider declares whether it is replayable; the dry-run runs CEL for
      the ones that are, and reports "this deployment's decider cannot be replayed" for the ones that
      are not. Honest, and keeps the feature working where it can.
   2. The dry-run always runs CEL, explicitly as "what the built-in rules would have said", and
      stops claiming to predict the live decider. Simplest, and quietly weakens a feature.
   3. Two seams rather than one — the CEL evaluator stays where it is and a plug is consulted
      as well, deny-wins. Keeps replay meaningful, and means a deployment that wants OPA to be the
      only authority cannot have that.

   Whichever it is, it should be decided before the interface exists rather than after, because it
   determines whether dryRunAgainstHistory takes a decider at all.

   One smaller thing

   If the point is "a deployment that already owns a policy engine", the fail-closed contract has to
   travel with the seam and be stated in the interface, not assumed. Today a thrown CEL expression
   denies, a non-boolean result denies, and an absent policy denies — evaluateActionPolicy is careful
   about all three. A remote decider adds failure modes the current one cannot have: a timeout, a 500,
   a malformed body. If the default for those is anything other than deny, the seam has quietly made
   the boundary weaker than the thing it replaced, and that is the kind of change nobody notices until
   it matters.