Skip to content

feat(sandboxpolicy): report privilege, network and file verdicts to the exporter receivers - #1080

Draft
AlonLiwsky wants to merge 153 commits into
kubescape:feat/sandbox-policy-exec-enforcementfrom
AlonLiwsky:feat/sandbox-policy-event-export
Draft

AlonLiwsky wants to merge 153 commits into
kubescape:feat/sandbox-policy-exec-enforcementfrom
AlonLiwsky:feat/sandbox-policy-event-export

Conversation

@AlonLiwsky

Copy link
Copy Markdown
Collaborator

Overview

This reports the privilege, network and file verdicts (enforced denies and audit wouldDeny) to the same receivers that exec authorizations use, so a private exporter (for example the OTLP mapping in armosec/private-node-agent#747) can pick them up. Until now these verdicts only went to the node-agent log.

It is stacked on #1075 (file gadget, which contains #1073, #1068 and #1067). GitHub can't use a fork branch as the base, so please review only the commits that are not in #1075, and merge after it.

Draft on purpose: the OTLP side lives in the private fork, and I'd like to align with @matthyx on whether this belongs here or folded into his exporter work.

What changes

  • Typed section events (pkg/sandboxpolicy/section_verdict.go): one event per verdict for privilege, network and file, with its own dispatch key, the policy identity (kind, name, uid, generation, namespace) and revision, rule name, origins list, mode, wouldDeny, the target in policy words (for example read, not the C enum name) and the actor (process, container, pod).
  • Binding lookup: the event's cgroup is resolved to its container or pod binding, and the policy set comes from the judging instance's slot table (by generation and slot).
  • Own queue: section events have their own queue and worker and feed the same receivers. The exec exporter is unchanged (exporter.go is identical to feat(sandboxpolicy): enforce the SandboxPolicy file section with a BPF LSM gadget #1075), so a burst of section denies can never drop or delay an exec verdict. A test sends 1,000 section denies into a 4-entry section queue: 996 section events are dropped and counted, 0 exec verdicts.
  • Counters: kernel ring-buffer drops and section queue drops are exposed through Service.SectionVerdictStats() / SectionExporterStats(). A drop is never silent.
  • Debug receiver behind sandboxPolicyDebugExport (default off): logs one line per exported event, for local testing.
  • One verdict is one kernel check, not one syscall (an audited setns on a mount namespace gives two). The exporter does not dedupe; the doc says so.

Production code is about 950 lines (the section exporter ~150, the debug receiver ~70, the subscriber rewrite), tests about 820, docs about 35.

How to test

  • go test ./pkg/sandboxpolicy/...
  • Kernel export test on 6.8 and 6.17 (arm64): one enforced deny and one audit wouldDeny per section reach a test receiver with binding, policy identity, rule, origins, target and counters.
  • Local kind end-to-end run with the debug receiver on: every one of the 175 denied or audited actions produced its event with the right wouldDeny, rule and pod binding; for a pod with two policies, the event names the deciding policy and both in the set.

Open questions

  1. Does this stay here, or fold into the exporter work in the private fork (Test/improvecomponenttest #747)?
  2. The backend accepts only exec verdicts today. Privilege, network and file need backend changes (rule family per section with an authorization phase, a phase guard on the R-NET-1 / R-FILE-1 observation views, section defaults, target keys). I'm aligning that separately.

Tracking: SUB-9591.

…ent library

pkg/ebpf/gadgets/common is a small library for BPF LSM enforcement gadgets:
the bindings map (cgroup -> policy slot), policy rank resolution, verdict
base fields and counters, the arming rule, and the Go binding controller of
one node-wide instance. It is taken from the enforce_exec simple engine of
the pinned Inspektor Gadget fork.

pkg/ebpf/gadgets/enforce_privilege enforces the SandboxPolicy privilege
section with BPF LSM, no seccomp: namespaces, mounts, pivot_root/chroot,
capabilities, ptrace, modules, bpf, perf, keys, io_uring, userfaultfd,
mknod, setuid/setgid and capset. Its compiler merges several policies into
one slot per combination.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…cy controller

- CompiledPolicy carries the privilege section; the compiler accepts it
- SectionBackend splits each plan per section; plans without a privilege
  section go to the exec backend unchanged
- PrivilegeBackend owns one enforce_privilege instance per node: live
  bind/unbind; a new policy combination replaces the instance with overlap
- slots are created when a matching pod is scheduled to the node
- sandboxPolicyPrivilegeImage enables the section; empty rejects it

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- each namespace flag of a call resolves on its own (bpf_loop), so an
  allow for one flag cannot carry another flag past a default deny
- compat ptrace read-mode access is not judged
- failed arming marks are counted; a deep tree arms instead of matching a
  wrong namespace
- transient arming failures are retried and reported
- a binding whose rollback failed stays tracked for Unbind
- reject NUL in mountFsType and duplicate slot generation entries
- the gadget build loop stops on the first failure

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- each namespace flag of a call resolves on its own (bpf_loop), so an
  allow for one flag cannot carry another flag past a default deny
- compat ptrace read-mode access is not judged
- failed arming marks are counted; a deep tree arms instead of matching a
  wrong namespace
- transient arming failures are retried and reported
- a binding whose rollback failed stays tracked for Unbind
- reject NUL in mountFsType and duplicate slot generation entries
- the gadget build loop stops on the first failure

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The binding cgroup path comes from the container collection, which reads
/proc/<pid>/cgroup in the cgroup namespace of PID 1. node-agent runs in its
own cgroup namespace, where the same file shows a /../.. path, so every
lease check failed with "process cgroup changed" and no container bound.
Check now reads the file in PID 1's cgroup namespace when it differs.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…efix

Gadget eBPF parameters are named operator.oci.ebpf.<key>. The engine set
operator.ebpf.host-mntns-id, which IG ignored, so host_mntns_id stayed 0 and
the program used the global init task's mount namespace. In kind the
runtime runs in the node container's mount namespace, so runc init for
kubectl exec was armed and its setns(CLONE_NEWNS) was denied: every exec
into a bound pod failed.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…group

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…group

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- keep an unready candidate that failed to stop for a stop retry
- prepare slots for ephemeral containers too
- list the node's pods for slot preparation only after a Pending-pod event
  or every 30 s, not on every refresh

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- the gadget has its own policy document with Parse and an example file;
  the SandboxPolicy translation moves to the controller connection (kubescape#1068)
- neutral names in the gadget, BPF, gadget.yaml and README
- a guard test fails on imports outside Inspektor Gadget and the gadget
  folders, and on host-product names or outside includes
- an opt-in test runs the gadget with Inspektor Gadget alone

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- the gadget has its own policy document with Parse and an example file;
  the SandboxPolicy translation moves to the controller connection (kubescape#1068)
- neutral names in the gadget, BPF, gadget.yaml and README
- a guard test fails on imports outside Inspektor Gadget and the gadget
  folders, and on host-product names or outside includes
- an opt-in test runs the gadget with Inspektor Gadget alone

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…n the controller layer

PrivilegeDocument (pkg/sandboxpolicy/privilege.go) replaces the gadget former
FromAPI, so the gadget no longer imports the CRD types.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…active

Without "bpf" in the host's active LSM list, BPF LSM programs attach but
never run (for example default AKS Ubuntu and Azure Linux images), so a
policy could report Ready with no enforcement. The service now reads
<host>/sys/kernel/security/lsm at startup; if bpf is missing it creates no
privilege backend, privilege bindings are refused, and every policy with a
privilege section reports Ready=False with reason BPFLSMInactive.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…path

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…path

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…sted upward includes

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…sted upward includes

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…an enforcing slot

When an armed task forks and the child's task storage mark cannot be
allocated, the child escaped every rule while the instance stayed ready.
Now an enforcing slot (any enforce entry) refuses the fork with -ENOMEM
(a clone into another cgroup too), and counts a tracking fault per slot
in arming_faults. A lost mark at exec, which cannot be refused, is
counted the same way. Audit-only slots and unbound cgroups allow the
fork. A new thread of an armed group leader is covered through the
leader and is never refused. Moving a container to another slot arms
its running tasks again. The binding controller reads the count with
SlotFaults. A test-only parameter injects the failure for the new
kernel test.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…an enforcing slot

When an armed task forks and the child's task storage mark cannot be
allocated, the child escaped every rule while the instance stayed ready.
Now an enforcing slot (any enforce entry) refuses the fork with -ENOMEM
(a clone into another cgroup too), and counts a tracking fault per slot
in arming_faults. A lost mark at exec, which cannot be refused, is
counted the same way. Audit-only slots and unbound cgroups allow the
fork. A new thread of an armed group leader is covered through the
leader and is never refused. Moving a container to another slot arms
its running tasks again. The binding controller reads the count with
SlotFaults. A test-only parameter injects the failure for the new
kernel test.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… of Linux 6.15

Linux 6.15 added `bool kernel` to the bpf hook and dropped the attr
argument of perf_event_open, so the previous-verdict check refused every
6.15+ kernel (and the old perf_event_open program would not load there).
Both programs now read `ret` at the position of the running kernel: a
CO-RE type match against security_list_options picks the branch at load.
The BTF check returns a result per hook (LSMHookArgMismatches) and
accepts both signatures for these two hooks. The other 21 hooks have the
same argument count from 6.1 to 6.18 (checked in lsm_hook_defs.h).

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…roller

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
clang folded the CLONE_INTO_CGROUP check into (flags << 30 s>> 63) & -ENOMEM.
The Linux 6.17 verifier (Ubuntu 6.17.0-40) loses the range of that AND and
rejects ig_file_task_alloc: R0 not in [-4095, 0]. Use the same volatile
result as the match path. Verified: loads on 6.8, 6.12 and 6.17; the kernel
test passes on 6.17 at cgroup levels 2 and 4.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The section gadgets only logged their verdicts. Each now reports a typed
authorization event under its own dispatch key (privilegeAuthorization,
networkAuthorization, fileAuthorization) through the bounded verdict
exporter that exec already uses.

- SectionVerdict holds the shared decision metadata; the operation facts
  are in PrivilegeTarget, NetworkTarget and FileTarget.
- The binding comes from the event cgroup (container or pod cgroup).
- The policy set comes from the slot table of the gadget instance that
  judged the call (event generation and slot).
- The kernel verdict counters (denied, dropped, rate limited) are read
  and exposed by Service.SectionVerdictStats.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The gadgets' string fields are C enum names (FILE_ACCESS_READ). Report
the policy word instead (read), for hooks, syscalls and access kinds.

Add an opt-in kernel test: for privilege, network and file, one enforced
deny and one audit wouldDeny reach a receiver with binding, policy,
origins, target and kernel counters. Passed on Linux 6.8 and 6.17 arm64.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Review fixes:
- A slot table now lives until its instance stops, or its start fails.
  A fixed window of 4 generations let repeated failed starts remove the
  table of the instance that still enforces.
- ReportSection queues a copy, so the producer shares no memory with
  the queue.
- An unknown rule id gives no reason, an unknown protocol number no name,
  an unreadable capability -1, and an empty slot no policy set.
- The privilege log line has origin, wouldDeny, reason and slot, like the
  network and file lines.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
setns into a namespace checks CAP_SYS_ADMIN twice (the namespace user
namespace and the caller's). Audit lets the first check pass, so one
audited setns gives two identical verdicts; enforce stops at the first.
A per-syscall dedupe is not reliable in an LSM hook (there is no syscall
instance id, and a time window would merge real calls), so the README
documents it and a kernel test pins it down. The test fixture now records
the fields of each verdict.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…roller

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
A syscall the kernel checks twice gives two identical verdicts. Neither
the gadget nor the exporter merges them; a consumer may.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…/privilege-gadget

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…roller

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The kubescape#1054 base moved to storage v0.0.388: PrivilegeTarget has no path,
fromImage or fsType fields any more, and Rule.Origin is a pointer. The
translator follows: it rejects actor and pathClass as before, now also
rejects except (the gadget has no exceptions, and they were ignored
before), and maps a missing origin and an empty one to no origin.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… branch

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… storage types

kubescape/storage now has the network fields of the SandboxPolicy design
(sandbox-policy-crd.md section 6.1): NetworkSection.DefaultMode,
NetworkSection.DefaultOrigin and NetworkRule.Except. Use them and remove
the local NetworkExtras mirror and the pre-decode split of the raw spec.

- TranslateNetwork and validateNetwork take the spec only.
- An except entry is validated with NetworkException.Validate, so an
  entry with an empty actor and no target is rejected, as in storage.
  An except list on an allow rule is rejected also when it is empty.
- defaultOrigin is still not carried: the gadget default has no origin.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The new storage types carry the file defaults by access, their modes and
origins, and the except list of deny rules, so the file local mirror
(FileExtras, FileAccessValues, FileExcept, splitFileExtras) is removed and
every field is read from api.FileSection.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
sandboxPolicyDebugExport (default false) registers a receiver that logs
each exported exec, privilege, network and file authorization as one
JSON line. For local tests without an exporter; existing receivers stay.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… branch

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
After the kubescape#1054 merge, RuleMatch has Origins. Section matches set it, and
SectionVerdict.Origins is never nil ([] means none), like exec. The
kernel test sets api.Rule.Origin as *string (storage v0.0.388).

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Exec keeps its VerdictExporter unchanged (exporter.go is the kubescape#1075
version again). SectionExporter has its own bounded queue and worker and
delivers to the same receivers, so a section deny burst never drops an
exec verdict. Section events are not enriched. Queue loss is in
Service.SectionExporterStats.

Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Nov 5, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@matthyx

matthyx commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

I prefer this typed public section-verdict/independent-queue work to stay in node-agent #1080. The private OTLP mapping belongs in armosec/private-node-agent#747; folding public event dispatch into the private fork would blur that boundary. Please keep the authorization-vs-observation distinction and explicit loss counters; I agree that one event per kernel check is not one event per syscall.

One schema detail to fix before the exporter consumes it: NewSectionVerdict currently appends an origin only when origin != "", while verdictFields.read returns the same empty string for absent ID 0 and a present explicitly empty label. The merged design and exec path preserve explicit origin: [""] separately from omitted origin: []. Please carry origin presence/ID separately through this decoder and test both cases (including defaults), rather than inferring presence from the string. The private family/phase mapping and backend ingestion contract can be agreed in a subsequent stacked change; this draft need not implement them itself.

@matthyx matthyx added the needs-rebase PR has merge conflicts and needs to be rebased label Nov 6, 2026
@matthyx matthyx moved this from WIP to Waiting on Author in KS PRs tracking Nov 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-rebase PR has merge conflicts and needs to be rebased

Projects

Status: Waiting on Author

Development

Successfully merging this pull request may close these issues.

2 participants