Repository navigation
feat(sandboxpolicy): report privilege, network and file verdicts to the exporter receivers - #1080
Conversation
…ent library pkg/ebpf/gadgets/common is a small library for BPF LSM enforcement gadgets: the bindings map (cgroup -> policy slot), policy rank resolution, verdict base fields and counters, the arming rule, and the Go binding controller of one node-wide instance. It is taken from the enforce_exec simple engine of the pinned Inspektor Gadget fork. pkg/ebpf/gadgets/enforce_privilege enforces the SandboxPolicy privilege section with BPF LSM, no seccomp: namespaces, mounts, pivot_root/chroot, capabilities, ptrace, modules, bpf, perf, keys, io_uring, userfaultfd, mknod, setuid/setgid and capset. Its compiler merges several policies into one slot per combination. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…cy controller - CompiledPolicy carries the privilege section; the compiler accepts it - SectionBackend splits each plan per section; plans without a privilege section go to the exec backend unchanged - PrivilegeBackend owns one enforce_privilege instance per node: live bind/unbind; a new policy combination replaces the instance with overlap - slots are created when a matching pod is scheduled to the node - sandboxPolicyPrivilegeImage enables the section; empty rejects it Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- each namespace flag of a call resolves on its own (bpf_loop), so an allow for one flag cannot carry another flag past a default deny - compat ptrace read-mode access is not judged - failed arming marks are counted; a deep tree arms instead of matching a wrong namespace - transient arming failures are retried and reported - a binding whose rollback failed stays tracked for Unbind - reject NUL in mountFsType and duplicate slot generation entries - the gadget build loop stops on the first failure Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- each namespace flag of a call resolves on its own (bpf_loop), so an allow for one flag cannot carry another flag past a default deny - compat ptrace read-mode access is not judged - failed arming marks are counted; a deep tree arms instead of matching a wrong namespace - transient arming failures are retried and reported - a binding whose rollback failed stays tracked for Unbind - reject NUL in mountFsType and duplicate slot generation entries - the gadget build loop stops on the first failure Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The binding cgroup path comes from the container collection, which reads /proc/<pid>/cgroup in the cgroup namespace of PID 1. node-agent runs in its own cgroup namespace, where the same file shows a /../.. path, so every lease check failed with "process cgroup changed" and no container bound. Check now reads the file in PID 1's cgroup namespace when it differs. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…efix Gadget eBPF parameters are named operator.oci.ebpf.<key>. The engine set operator.ebpf.host-mntns-id, which IG ignored, so host_mntns_id stayed 0 and the program used the global init task's mount namespace. In kind the runtime runs in the node container's mount namespace, so runc init for kubectl exec was armed and its setns(CLONE_NEWNS) was denied: every exec into a bound pod failed. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…group Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…group Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- keep an unready candidate that failed to stop for a stop retry - prepare slots for ephemeral containers too - list the node's pods for slot preparation only after a Pending-pod event or every 30 s, not on every refresh Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- the gadget has its own policy document with Parse and an example file; the SandboxPolicy translation moves to the controller connection (kubescape#1068) - neutral names in the gadget, BPF, gadget.yaml and README - a guard test fails on imports outside Inspektor Gadget and the gadget folders, and on host-product names or outside includes - an opt-in test runs the gadget with Inspektor Gadget alone Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
- the gadget has its own policy document with Parse and an example file; the SandboxPolicy translation moves to the controller connection (kubescape#1068) - neutral names in the gadget, BPF, gadget.yaml and README - a guard test fails on imports outside Inspektor Gadget and the gadget folders, and on host-product names or outside includes - an opt-in test runs the gadget with Inspektor Gadget alone Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…n the controller layer PrivilegeDocument (pkg/sandboxpolicy/privilege.go) replaces the gadget former FromAPI, so the gadget no longer imports the CRD types. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…active Without "bpf" in the host's active LSM list, BPF LSM programs attach but never run (for example default AKS Ubuntu and Azure Linux images), so a policy could report Ready with no enforcement. The service now reads <host>/sys/kernel/security/lsm at startup; if bpf is missing it creates no privilege backend, privilege bindings are refused, and every policy with a privilege section reports Ready=False with reason BPFLSMInactive. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…path Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…path Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…sted upward includes Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…sted upward includes Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…an enforcing slot When an armed task forks and the child's task storage mark cannot be allocated, the child escaped every rule while the instance stayed ready. Now an enforcing slot (any enforce entry) refuses the fork with -ENOMEM (a clone into another cgroup too), and counts a tracking fault per slot in arming_faults. A lost mark at exec, which cannot be refused, is counted the same way. Audit-only slots and unbound cgroups allow the fork. A new thread of an armed group leader is covered through the leader and is never refused. Moving a container to another slot arms its running tasks again. The binding controller reads the count with SlotFaults. A test-only parameter injects the failure for the new kernel test. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…an enforcing slot When an armed task forks and the child's task storage mark cannot be allocated, the child escaped every rule while the instance stayed ready. Now an enforcing slot (any enforce entry) refuses the fork with -ENOMEM (a clone into another cgroup too), and counts a tracking fault per slot in arming_faults. A lost mark at exec, which cannot be refused, is counted the same way. Audit-only slots and unbound cgroups allow the fork. A new thread of an armed group leader is covered through the leader and is never refused. Moving a container to another slot arms its running tasks again. The binding controller reads the count with SlotFaults. A test-only parameter injects the failure for the new kernel test. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… of Linux 6.15 Linux 6.15 added `bool kernel` to the bpf hook and dropped the attr argument of perf_event_open, so the previous-verdict check refused every 6.15+ kernel (and the old perf_event_open program would not load there). Both programs now read `ret` at the position of the running kernel: a CO-RE type match against security_list_options picks the branch at load. The BTF check returns a result per hook (LSMHookArgMismatches) and accepts both signatures for these two hooks. The other 21 hooks have the same argument count from 6.1 to 6.18 (checked in lsm_hook_defs.h). Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…roller Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
clang folded the CLONE_INTO_CGROUP check into (flags << 30 s>> 63) & -ENOMEM. The Linux 6.17 verifier (Ubuntu 6.17.0-40) loses the range of that AND and rejects ig_file_task_alloc: R0 not in [-4095, 0]. Use the same volatile result as the match path. Verified: loads on 6.8, 6.12 and 6.17; the kernel test passes on 6.17 at cgroup levels 2 and 4. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The section gadgets only logged their verdicts. Each now reports a typed authorization event under its own dispatch key (privilegeAuthorization, networkAuthorization, fileAuthorization) through the bounded verdict exporter that exec already uses. - SectionVerdict holds the shared decision metadata; the operation facts are in PrivilegeTarget, NetworkTarget and FileTarget. - The binding comes from the event cgroup (container or pod cgroup). - The policy set comes from the slot table of the gadget instance that judged the call (event generation and slot). - The kernel verdict counters (denied, dropped, rate limited) are read and exposed by Service.SectionVerdictStats. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The gadgets' string fields are C enum names (FILE_ACCESS_READ). Report the policy word instead (read), for hooks, syscalls and access kinds. Add an opt-in kernel test: for privilege, network and file, one enforced deny and one audit wouldDeny reach a receiver with binding, policy, origins, target and kernel counters. Passed on Linux 6.8 and 6.17 arm64. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Review fixes: - A slot table now lives until its instance stops, or its start fails. A fixed window of 4 generations let repeated failed starts remove the table of the instance that still enforces. - ReportSection queues a copy, so the producer shares no memory with the queue. - An unknown rule id gives no reason, an unknown protocol number no name, an unreadable capability -1, and an empty slot no policy set. - The privilege log line has origin, wouldDeny, reason and slot, like the network and file lines. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
setns into a namespace checks CAP_SYS_ADMIN twice (the namespace user namespace and the caller's). Audit lets the first check pass, so one audited setns gives two identical verdicts; enforce stops at the first. A per-syscall dedupe is not reliable in an LSM hook (there is no syscall instance id, and a time window would merge real calls), so the README documents it and a kernel test pins it down. The test fixture now records the fields of each verdict. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…roller Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
A syscall the kernel checks twice gives two identical verdicts. Neither the gadget nor the exporter merges them; a consumer may. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…/privilege-gadget Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
…roller Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The kubescape#1054 base moved to storage v0.0.388: PrivilegeTarget has no path, fromImage or fsType fields any more, and Rule.Origin is a pointer. The translator follows: it rejects actor and pathClass as before, now also rejects except (the gadget has no exceptions, and they were ignored before), and maps a missing origin and an empty one to no origin. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… branch Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… storage types kubescape/storage now has the network fields of the SandboxPolicy design (sandbox-policy-crd.md section 6.1): NetworkSection.DefaultMode, NetworkSection.DefaultOrigin and NetworkRule.Except. Use them and remove the local NetworkExtras mirror and the pre-decode split of the raw spec. - TranslateNetwork and validateNetwork take the spec only. - An except entry is validated with NetworkException.Validate, so an entry with an empty actor and no target is rejected, as in storage. An except list on an allow rule is rejected also when it is empty. - defaultOrigin is still not carried: the gadget default has no origin. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
The new storage types carry the file defaults by access, their modes and origins, and the except list of deny rules, so the file local mirror (FileExtras, FileAccessValues, FileExcept, splitFileExtras) is removed and every field is read from api.FileSection. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
sandboxPolicyDebugExport (default false) registers a receiver that logs each exported exec, privilege, network and file authorization as one JSON line. For local tests without an exporter; existing receivers stay. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
… branch Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
After the kubescape#1054 merge, RuleMatch has Origins. Section matches set it, and SectionVerdict.Origins is never nil ([] means none), like exec. The kernel test sets api.Rule.Origin as *string (storage v0.0.388). Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
Exec keeps its VerdictExporter unchanged (exporter.go is the kubescape#1075 version again). SectionExporter has its own bounded queue and worker and delivers to the same receivers, so a section deny burst never drops an exec verdict. Section events are not enriched. Queue loss is in Service.SectionExporterStats. Signed-off-by: Alon Liwsky <40373481+AlonLiwsky@users.noreply.github.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
I prefer this typed public section-verdict/independent-queue work to stay in node-agent #1080. The private OTLP mapping belongs in armosec/private-node-agent#747; folding public event dispatch into the private fork would blur that boundary. Please keep the authorization-vs-observation distinction and explicit loss counters; I agree that one event per kernel check is not one event per syscall. One schema detail to fix before the exporter consumes it: |
Overview
This reports the privilege, network and file verdicts (enforced denies and audit wouldDeny) to the same receivers that exec authorizations use, so a private exporter (for example the OTLP mapping in armosec/private-node-agent#747) can pick them up. Until now these verdicts only went to the node-agent log.
It is stacked on #1075 (file gadget, which contains #1073, #1068 and #1067). GitHub can't use a fork branch as the base, so please review only the commits that are not in #1075, and merge after it.
Draft on purpose: the OTLP side lives in the private fork, and I'd like to align with @matthyx on whether this belongs here or folded into his exporter work.
What changes
pkg/sandboxpolicy/section_verdict.go): one event per verdict for privilege, network and file, with its own dispatch key, the policy identity (kind, name, uid, generation, namespace) and revision, rule name, origins list, mode,wouldDeny, the target in policy words (for exampleread, not the C enum name) and the actor (process, container, pod).exporter.gois identical to feat(sandboxpolicy): enforce the SandboxPolicy file section with a BPF LSM gadget #1075), so a burst of section denies can never drop or delay an exec verdict. A test sends 1,000 section denies into a 4-entry section queue: 996 section events are dropped and counted, 0 exec verdicts.Service.SectionVerdictStats()/SectionExporterStats(). A drop is never silent.sandboxPolicyDebugExport(default off): logs one line per exported event, for local testing.setnson a mount namespace gives two). The exporter does not dedupe; the doc says so.Production code is about 950 lines (the section exporter ~150, the debug receiver ~70, the subscriber rewrite), tests about 820, docs about 35.
How to test
go test ./pkg/sandboxpolicy/...wouldDeny, rule and pod binding; for a pod with two policies, the event names the deciding policy and both in the set.Open questions
Tracking: SUB-9591.