feature/mdbf: gemlite対応とHessian/scale_bits修正 - #28
Merged
aki916f merged 6 commits intoJul 4, 2026
Conversation
- Fixed a discrepancy in the definition of Hessian - Added `scale_bits` as an argument to `rank_from_bpw()`; the default is 16
aki916f
reviewed
Jul 2, 2026
| P: int = 2, | ||
| min_rank: int = 1, | ||
| rounding: Literal["floor", "ceil", "round"] = "floor", | ||
| scale_bits: int = 16, |
Contributor
There was a problem hiding this comment.
docstringにscale_bitsの追記をお願いします。
Contributor
|
ご対応ありがとうございます！gemliteの部分は特に問題ないと思います。 |
…ixes Document the scale_bits argument added to rank_from_bpw() and record the GemLite inference path, OSVD Hessian fix, and rank_from_bpw fix in CHANGELOG.md (PR review feedback).
…tests - test_mdbflinear_falls_back_to_dense_when_gemlite_unavailable: verify MDBFLinear falls back to the dense path (bit-identical) when GemLite support is unavailable, even if use_gemlite=True is forced. - test_rank_from_bpw_matches_paper_formula: verify rank_from_bpw() round-trips with bpw_from_rank() at scale_bits=16 and that scale_bits=0 admits a higher rank for the same budget. - test_lowrank_osvd_beats_plain_svd_in_hessian_error: regression test for the lowrank_osvd() Hessian fix (a9b46df) exercising the production function directly against a non-diagonal Hessian. - test_osvd_hessian_bug.py: proof-of-concept numerically demonstrating that the previous H^{1/2} formulation inflated the Hessian-weighted reconstruction error for non-diagonal Hessians. PR review feedback (test plan coverage).
Contributor
|
@fujisawa-yoshihiko |
…English Translate the Japanese module docstring, function/class docstrings, and inline comments in the OSVD Hessian proof-of-concept test to English. No logic changes (test still passes).
…ory)
The OSVD tests described the pre-fix "buggy" lowrank_osvd (Q^T missing)
and asserted facts about "the current implementation" that no longer hold
after the fix. Reframe them as a self-contained property/regression check:
the full H^{1/2}=Q diag(sqrt(λ)) Q^T whitening is optimal for the H-weighted
objective, and a variant dropping the trailing Q^T is strictly worse for a
non-diagonal H. Rename the _osvd_buggy helper / test methods accordingly.
No logic changes (tests still pass).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MDBF量子化レイヤーに GemLite による1-bit推論高速化を追加し、OSVD初期化のHessian定義と
rank_from_bpw()のBPW計算を修正しました。1. GemLite推論対応 (
42b0a62)MDBFLinear/MultipathMDBFLinearに、±1符号行列（A_sign,B_sign）向けの GemLite 1-bit matmul を追加dbf/dbf_layer.py）と同様の設計で、利用可能な場合は符号行列の matmul を GemLite カーネルに委譲use_gemlite=None）ではl == 1のときのみ有効化（l > 1では rank-l 振幅を外側で適用するため GemLite 経路が遅くなる）use_gemlite=Trueで強制有効化可能MDBF.create_linear_module()からuse_gemliteを渡せるように変更2. OSVD Hessian 定義の修正 (
a9b46df)lowrank_osvd()においてW @ H^{1/2}の計算で固有ベクトルの転置@ eig_vecs.Tが欠落していた問題を修正W_tilde = W @ Q @ diag(√λ)W_tilde = W @ Q @ diag(√λ) @ Q^T（論文の定義に一致）3. BPW → rank 換算の修正 (
a9b46df)rank_from_bpw()でscale_bits=0がハードコードされていた問題を修正scale_bitsを引数化し、デフォルトを16に変更（FP16 envelope パラメータを BPW に含める）b = P * [r(n+m) + 16l(n+m+2r)] / (nm)に整合変更ファイル
onecomp/quantizer/mdbf/mdbf_layer.pyonecomp/quantizer/mdbf/_mdbf.pyuse_gemlite引数の伝播onecomp/quantizer/mdbf/initialize.pyonecomp/quantizer/mdbf/utils.pyrank_from_bpw()のscale_bits修正検証
TinyLlama-1.1B / Llama-2-7B において、1 bpw（
l=8, P=1, act_init=osvd）設定で修正前後を比較：scale_bits修正により target BPW と実際の BPW が一致するようになったTest plan
l=1）に推論結果が dense パスと一致することrank_from_bpw()がscale_bits=16で論文式の BPW を返すことact_init="osvd"）が activation-aware モードで動作すること