Local inference on hardware I already have, matching decompilation, and agents that do the boring parts of both. Everything here is meant to run offline, on consumer cards, with the source in plain sight and the measurements in the README.
| Repo | What it is | ★ |
|---|---|---|
decomp.anything Python |
Point an AI agent at something you own and take it apart. Agents write C/C++ against the real binary, the original compiler builds it, the result is checked byte for byte. PE and ELF binaries, apps, firmware, plain files. | |
ninfer-studio TS |
Native Linux and Windows desktop app (Tauri 2) that turns an inference server binary into a product: per-GPU presets, Hugging Face model management, streaming chat with vision, an agentic coding harness, live token and VRAM metrics. | |
KernelOPT Rust |
Dispatch-aware, agentic GPU-kernel optimization. Point it at a kernel, a directory, or a whole model; it runs a profile → plan → edit → verify → gate loop until each kernel is faster or the budget runs out. Your working tree is never touched. | |
late-sast Go |
Autonomous security auditor. One command, one repo in, one report out — graph-first taint tracing in a Docker sandbox with live exploit verification, so a finding is proven reachable instead of merely suspicious. | |
Decapybara C++ |
Open-source C++20 and CUDA reconstruction of a closed inference server, targeting Qwen3.8-27B on RTX 5090 Blackwell. "The source code will be published later." — later is now. |
The engines themselves, and the work of squeezing a model onto one GPU.
- spite
Rust— one engine, your model, your card: a single Rust host that loads any GGUF file and runs it on the hardware you have. No cloud, no subscription, nothing leaving the machine. - ninfer-ext
C++— an extended fork of NInfer (engine and artifact format by Neroued) adding Qwen3.8-Flash-Next, EXL3, deeper speculative decoding and Gemma 4 vision on one RTX 5090. Archived 2026-10-22: the tree still builds and serves everything documented, it is simply not being worked on. Its README carries the head-to-head measurements against basalt for Qwen3.8-Flash-Next NVFP4 at 262,144-token context, and recommends basalt or capy-flash over it. - llama-cpp-turboquant-hybrid
C++— a llama.cpp branch adding hybrid speculative decoding: composes MTP (multi-token prediction) and DFlash (block diffusion) draft paths into a single decoder pass, with per-token provenance so each backend's acceptance stats stay separate. Everything outside that branch is upstream llama.cpp. - OSCAR-llamacpp
C++— a llama.cpp fork adding an INT2 KV cache for Qwen3 and Gemma 4; the Metal path runs INT2+f16 KV on-GPU with fused mixed-precision flash-attention kernels. Pre-built*-rot-kv.ggufmodels on Hugging Face.
- decomp.anything
Python— the matching-decompilation workbench above, plus an optional REA MCP bridge for binaries with no original build to match. - NanoHound
Python— lightweight in-memory Active Directory attack-path explorer, built on NiceGUI and NetworkX.
- zap-extension-aitrafficanalyst
Java— OWASP ZAP add-on that feeds live request/response pairs into the official ZAP LLM add-on for OWASP-first analysis. Provider-agnostic, local or remote, no model config of its own. - SnaffleSnap
PowerShell— parses Snaffler output into colour-coded HTML reports sorted by severity.
C++/CUDA and Rust for anything that has to hold a model in VRAM. Python for reverse-engineering and analysis plumbing. Go for single-binary tools. TypeScript with Tauri when the thing needs a GUI. Linux throughout, NVIDIA throughout.
Most of what I post carries the numbers that justified it: tokens per second at a stated context depth, byte-for-byte match status, before-and-after kernel timings. When a project stops being worth the time I say so at the top of its README and leave the tree building, rather than letting it rot quietly.
41 of my 69 public repos carry GitHub's fork label, and they are there because this is where my working copies live, not because I wrote them. Read the parent repo before crediting me with any of it; a few carry a handful of my commits, most carry none.
The remaining 28 are not GitHub-flagged, but four of them still continue somebody else's project and say so in their own READMEs:
- decomp.anything builds on decomp.wtf by i12bp8 (MIT), which its owner discontinued and archived on 2026-10-15. I continued and extended it.
- ninfer-ext extends NInfer, designed and developed by Neroued, who also owns the artifact format.
- OSCAR-llamacpp and llama-cpp-turboquant-hybrid are llama.cpp, upstream by Georgi Gerganov and contributors; my work is the KV-cache quantization and the hybrid speculative-decoding path respectively.
Fully mine, from an empty repo: ninfer-studio, KernelOPT, late-sast, Decapybara, spite, NanoHound, SnaffleSnap. The other repos here are either upstream projects I run, or older tools I did not write; each repo's own history is the record.




