Skip to content
View giveen's full-sized avatar
🫠
...grinding
🫠
...grinding
  • Idaho
  • 21:46 (UTC -06:00)

Organizations

@Team-Streak

Block or report giveen

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
giveen/README.md

Engines for models you own, and tools that take binaries apart.

Local inference on hardware I already have, matching decompilation, and agents that do the boring parts of both. Everything here is meant to run offline, on consumer cards, with the source in plain sight and the measurements in the README.

Followers Public repos · 69 Originals · 28 C++ · Rust · Python · Go · TS

Things I'd point you at first

Repo What it is ★
decomp.anything Python Point an AI agent at something you own and take it apart. Agents write C/C++ against the real binary, the original compiler builds it, the result is checked byte for byte. PE and ELF binaries, apps, firmware, plain files. ★
ninfer-studio TS Native Linux and Windows desktop app (Tauri 2) that turns an inference server binary into a product: per-GPU presets, Hugging Face model management, streaming chat with vision, an agentic coding harness, live token and VRAM metrics. ★
KernelOPT Rust Dispatch-aware, agentic GPU-kernel optimization. Point it at a kernel, a directory, or a whole model; it runs a profile → plan → edit → verify → gate loop until each kernel is faster or the budget runs out. Your working tree is never touched. ★
late-sast Go Autonomous security auditor. One command, one repo in, one report out — graph-first taint tracing in a Docker sandbox with live exploit verification, so a finding is proven reachable instead of merely suspicious. ★
Decapybara C++ Open-source C++20 and CUDA reconstruction of a closed inference server, targeting Qwen3.8-27B on RTX 5090 Blackwell. "The source code will be published later." — later is now. ★

By area

Inference engines and quantization

The engines themselves, and the work of squeezing a model onto one GPU.

  • spite Rust — one engine, your model, your card: a single Rust host that loads any GGUF file and runs it on the hardware you have. No cloud, no subscription, nothing leaving the machine.
  • ninfer-ext C++ — an extended fork of NInfer (engine and artifact format by Neroued) adding Qwen3.8-Flash-Next, EXL3, deeper speculative decoding and Gemma 4 vision on one RTX 5090. Archived 2026-10-22: the tree still builds and serves everything documented, it is simply not being worked on. Its README carries the head-to-head measurements against basalt for Qwen3.8-Flash-Next NVFP4 at 262,144-token context, and recommends basalt or capy-flash over it.
  • llama-cpp-turboquant-hybrid C++ — a llama.cpp branch adding hybrid speculative decoding: composes MTP (multi-token prediction) and DFlash (block diffusion) draft paths into a single decoder pass, with per-token provenance so each backend's acceptance stats stay separate. Everything outside that branch is upstream llama.cpp.
  • OSCAR-llamacpp C++ — a llama.cpp fork adding an INT2 KV cache for Qwen3 and Gemma 4; the Metal path runs INT2+f16 KV on-GPU with fused mixed-precision flash-attention kernels. Pre-built *-rot-kv.gguf models on Hugging Face.

Reverse engineering

  • decomp.anything Python — the matching-decompilation workbench above, plus an optional REA MCP bridge for binaries with no original build to match.
  • NanoHound Python — lightweight in-memory Active Directory attack-path explorer, built on NiceGUI and NetworkX.

Security and detection

  • zap-extension-aitrafficanalyst Java — OWASP ZAP add-on that feeds live request/response pairs into the official ZAP LLM add-on for OWASP-first analysis. Provider-agnostic, local or remote, no model config of its own.
  • SnaffleSnap PowerShell — parses Snaffler output into colour-coded HTML reports sorted by severity.

What I work in

C++/CUDA and Rust for anything that has to hold a model in VRAM. Python for reverse-engineering and analysis plumbing. Go for single-binary tools. TypeScript with Tauri when the thing needs a GUI. Linux throughout, NVIDIA throughout.

Measure first, then claim

Most of what I post carries the numbers that justified it: tokens per second at a stated context depth, byte-for-byte match status, before-and-after kernel timings. When a project stops being worth the time I say so at the top of its README and leave the tree building, rather than letting it rot quietly.

About the forks

41 of my 69 public repos carry GitHub's fork label, and they are there because this is where my working copies live, not because I wrote them. Read the parent repo before crediting me with any of it; a few carry a handful of my commits, most carry none.

The remaining 28 are not GitHub-flagged, but four of them still continue somebody else's project and say so in their own READMEs:

  • decomp.anything builds on decomp.wtf by i12bp8 (MIT), which its owner discontinued and archived on 2026-10-15. I continued and extended it.
  • ninfer-ext extends NInfer, designed and developed by Neroued, who also owns the artifact format.
  • OSCAR-llamacpp and llama-cpp-turboquant-hybrid are llama.cpp, upstream by Georgi Gerganov and contributors; my work is the KV-cache quantization and the hybrid speculative-decoding path respectively.

Fully mine, from an empty repo: ninfer-studio, KernelOPT, late-sast, Decapybara, spite, NanoHound, SnaffleSnap. The other repos here are either upstream projects I run, or older tools I did not write; each repo's own history is the record.

Pinned Loading

  1. late-sast late-sast Public

    Autonomous SAST scanner — Docker sandbox, graph-first taint tracing, live exploit verification

    Go 3

  2. ninfer-ext ninfer-ext Public archive

    ninfer-ext: Forked for speed.

    C++ 26 5