Repository navigation
Qualify MiniMax-H3 Ref2VA five-second Ascend optimization - #294
Conversation
Agent-assisted-by: Codex (implementation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
| @@ -0,0 +1,9 @@ | |||
| { | |||
There was a problem hiding this comment.
P1|新 evidence bundle 会使现有库存检查失败(静态可确定)。 此 PR 新增 evidence/h3-ref2va-ascend-293/,但没有同步登记到 tools/evidence_inventory.json。main 的 tools/validate_evidence.py::validate_inventory() 严格比较 evidence/ 顶层目录与 bundles 键,发现新增目录即报 inventory directories differ 并以非零状态退出;贡献约定也将该校验列为 nightly 检查。另需注意:登记该 manifest.json 后,未列入 inventory 会被判为未分类;若直接按 bundle-sha256 校验,清单内两项未随 PR 提交的 .pth 权重会按缺失的本地文件报错。建议登记 kind: campaign,明确 manifest 的本地可验证文件与外部权重来源/不可用边界,并用已有的只读 inventory 检查验证修正(不要求本任务执行)。依据:tools/validate_evidence.py 的 actual != set(entries)、unclassified manifest、verify_file 路径。
There was a problem hiding this comment.
The inventory-registration P1 is fixed at 288102b7; this scoped confirmation does not clear the two quality-gate P2s or approve the Draft.
- The new campaign directory is registered. Exact
historical-sha256mappings cover the client plus five local validator files, and both unvendored weight files are explicitly external. All six local SHA-256 values match their manifests, so those weights are no longer treated as missing local inventory files. - The latest increment changes only README/pilot data; the inspected registration and six mapped files are unchanged from
8bb7e797. Both installed-package jobs pass at the new head. Its full-check jobs are still running, and task qualification is skipped for the Draft; the older head's successful Ready run is not a new-head pass. - The existing author still owns the audio coverage and loaded LPIPS-weight identity threads. Declaring weights external fixes inventory classification; it does not verify the weights actually used for scoring. No further inventory change is requested by this check.
Evidence and verification limits
Assessed head: 288102b72ef0dc4af8c2687e4f0b54d02c9858f6; API base / observed main: 218565698d850e955f85d0d394a5f0e5e3619812.
Current inventory and validator manifest were checked against the existing validate_manifest local/external contract and raw file bytes. Current Ready run and current-head declaration status were reread before posting.
Static source/byte-hash and retained hosted-status inspection only. I did not run the inventory command, media smoke, LPIPS, models or devices. Campaign measurements remain author-reported with public raw-media/log availability limits; no speedup, lifecycle or whole-campaign acceptance follows. This reply does not mark any reviewer thread resolved.
| ).stdout | ||
| audio = np.frombuffer(pcm, dtype="<f4").reshape(-1, gates["channels"]).T.copy() | ||
| audios.append(audio) | ||
| if audios[0].shape != audios[1].shape: |
There was a problem hiding this comment.
P2|两臂同样截短音轨时可能错误通过质量门禁。 #293 冻结的是 5s / 15s 视频及音频质量,而 probe() 只核对视频帧数、音频采样率和通道;此处仅要求两臂 decoded PCM 的 shape 相同,没有要求音轨覆盖对应的输出时长或检查 A/V 起止时间戳。因而若两个 15s 视频都只附带相同的短音频(例如 0.5s),其余指标可全部合格并得到 PASS,但完整音视频交付并未验证。建议根据期望时长检查 PCM 样本数(允许明确的编码 padding 容差),同时检查音视频 PTS/覆盖范围,并加一个“两臂都缺尾部音频”的负例。此处是基于代码路径的风险判断,未实际生成媒体复现。
There was a problem hiding this comment.
对本线程 P2 做范围校正(新 5s 证据,不重复要求修复):当前 head 017a5c95 的 results/final-5s/quality-index.json 共 14 条比较均记录每通道 audio_samples=165888;validator/gates.json 冻结采样率为 32 kHz,折算 5.184 s,与 124 帧/24 FPS 的 5.167 s 视频名义时长相符。因而这批 5s 收据没有呈现此前举例的“两臂同样只有 0.5s 音轨”场景;不应把原来的静态反例说成已在本次硬件结果中复现。
剩余边界仍是通用验证器的代码路径:compare.py 仅比较两臂 PCM shape,未核对请求期望时长和 A/V PTS;上述样本数也不能单独证明音轨内容或时间戳完整。对后续 15s/其他输出复用时,仍应按本线程所述建立预期覆盖与负例。这里仅审查提交的源码及 JSON,不代表复验外部 MP4 或 LPIPS 评分器。
| import lpips | ||
| import torch | ||
|
|
||
| loss_fn = lpips.LPIPS(net="alex").eval() |
There was a problem hiding this comment.
P2|冻结的 LPIPS 权重身份尚未在执行入口强制校验。 validator/manifest.json 给出 AlexNet 与 LPIPS 权重的 SHA-256,但这里 lpips.LPIPS(net="alex") 直接依赖环境中的权重获取/缓存路径,compare() 并不核验实际载入的 .pth 内容是否匹配清单。不同缓存/安装状态可能使用未认证的评分器,却仍按同一固定 LPIPS 阈值出结论。建议在评分前验证实际权重字节的固定摘要、将使用的 scorer 身份记录进质量证据,校验失败则不出 PASS;无需为此自动下载或在本任务内运行模型。
Agent-assisted-by: Codex (implementation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Keep the campaign INCONCLUSIVE pending clean teardown and repeated unprofiled comparisons. Attribute the policy to upstream and link the downstream lifecycle draft. Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Keep quality gates unchanged and distinguish pilot observations from final AB/BA acceptance. Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Retain the slower cold request and six terminate signals without claiming a speedup. Quality gates remain unchanged for isolated configuration screening. Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
| "profiler": false, | ||
| "cache_summary": false, | ||
| "block_cache": false, | ||
| "wave_timeout_s": 14400, |
There was a problem hiding this comment.
P2|单请求超时长于整轮 NPU 预约,失败归因与多臂覆盖存在风险。 这里 wave_timeout_s、http_timeout_s 都设为 14,400 秒(4 小时),但同一计划第 33 行将全部 kv2 → kvdevice2 → kvdevice8 放进共享 3 小时 hard reservation,且 165 分钟就会 soft TERM。若第一臂的 warmup 像此前 KV2 一样阻塞,外层结束/杀进程将早于内部超时,后续臂可能完全没有样本,且缺少可归因的请求级 timeout 收据;每臂还有 2 次 warmup + 1 次测量。建议冻结**小于剩余预约预算(预留启动/清理空间)**的单请求/单臂截止时间,记录哪一级 timeout/TERM 先触发以及各臂未完成原因;如果有意只用预约守护程序作为故障上限,则在计划中明确它与请求超时的优先级和失败证据格式。该问题是静态计划约束冲突,未复现硬件行为,也不要求本评审任务执行实验。
There was a problem hiding this comment.
补充当前 323bf306 的受影响计划:新增的 cache-range-pilot.json 也存在同一 timeout 层级问题,而且四臂共用的预约更短。
wave_timeout_s/http_timeout_s仍为 14,400 秒;共享预约为 2 小时,105 分钟 soft TERM,180 秒后强制终止。每臂还有两次 warmup 和三次测量。- 若前臂请求阻塞,预约层会先终止,内部请求 timeout 无法作为该次失败边界,后续臂可能没有样本。请把这份计划一并纳入本线程的处置:按剩余预约时间约束请求/单臂截止时间,或明确预约 watchdog 优先及中断、未执行各臂的证据记录。
这里只补充新计划的静态证据,不重复开问题;没有运行设备,也不判断未公开的执行脚本是否已另行处理。原始失败与未完成记录应保留。
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
The new pr-references failure is a PR-body citation issue; this Draft is still awaiting the existing quality-gate review items.
- P2 — source citation triggers the closing-syntax check. The INT8/RainFusion row at PR-body line 16 places a closing keyword immediately before a bare upstream PR number. The current-head job reports that exact line and exits 1. This is an advisory text check, not a failed media or model test.
- Minimal fix for the existing author: reword the source reference as “multi-video correction” and link explicitly to vLLM-Omni PR 9241. Preserve the source attribution while removing closing syntax. The next ordinary push will check the current body; no manual workflow dispatch is requested.
- The audio-coverage and loaded-scorer review items remain separate. Retaining the interrupted A1 attempt and restarting the whole paired campaign does not yet establish final performance or lifecycle acceptance.
Evidence and limits
Reviewed head: 3cdcdb43b9b85e537528e847437bfbd0cf6c3a1e; API base: 218565698d850e955f85d0d394a5f0e5e3619812; observed main: 50c19da43e0b358e6f990c00719891ad308a8ac1.
Current-head failing job logs identify PR-body line 16. The checker and Ready workflow were read; this job is documented as advisory, not a required repository check. Other current-head software jobs succeeded; Draft task qualification was skipped. Static text/source and hosted-log inspection only; no PR code, media scorer, model or device was run by this task. This COMMENTED review does not approve the PR or resolve any reviewer thread.
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Agent-assisted-by: Codex Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
The PR-body reference-check finding is fixed at 017a5c95; this is a scoped repair confirmation, not campaign approval.
- The rewritten body no longer contains the flagged source wording, and the current-head
pr-referencesjob succeeds. No further citation change is requested by this review. - Current full-check, installed-package, qualification and declaration/policy checks succeed. These software/admission results do not independently verify the reported media scores, scorer weights or latency measurements.
- The remaining loaded-weight identity thread and audio-coverage clarification retain their stated scope. The new 5s receipts must not be described as reproducing the earlier short-audio counterexample; whole-campaign acceptance remains with the requester and the existing author retains follow-up ownership.
Evidence and limits
Head: 017a5c95bd77788b92bd3fff1b1648a8e214adb1; API base: 218565698d850e955f85d0d394a5f0e5e3619812; observed main: eabad4ad57c0af9407d33da83bb117201d5e5cce.
Follow-up to the earlier citation review. Current body, exact-head hosted checks/policy, changed paths, submitted final-summary/quality JSON and discussion were inspected. The validator source is unchanged in this increment; full 144-file and downstream/raw-media review remains incomplete. I did not run the validator, LPIPS, media, model, device or benchmark code. External raw media, weights and runtime logs were not independently replayed; the reported 5s result is not a new independent performance or quality acceptance. No reviewer thread is resolved by this COMMENTED review.
Dong1017
left a comment
There was a problem hiding this comment.
LGTM for merging at 017a5c95 as five-second campaign evidence. This review is scoped to the PR; acceptance of the #293 campaign stays with its requester.
Checked:
- All nine checks pass at this head, and the PR merges cleanly. It adds 144 files totaling 0.52 MB, so the evidence-size concern in #273 (R6) does not apply.
- The headline matches the retained values: in
results/final-5s/acceptance.json, the server means of 396.30 s and 418.24 s (baseline) and 106.66 s and 109.58 s (candidate) give 407.27 → 108.12 s, 73.45% lower. - All 14 entries in
results/final-5s/quality-index.jsonpass the frozen gates invalidator/gates.json. The closest margins are audio spectral cosine 0.813 (gate 0.80, disclosed in the PR body) and SSIM 0.976 (gate 0.97). - Every commit carries a DCO sign-off and an
Agent-assisted-bytrailer.
Open threads; none blocks this merge:
- LPIPS weight identity (P2):
compare.pystill loads LPIPS weights from the environment without checking the manifest hash, so the scores are not attested. The largest recorded LPIPS is 0.026 against the 0.20 gate, so a different weight file is unlikely to change these verdicts. Add the hash check before the validator is reused. - Audio coverage (P2): the 5 s receipts record 165,888 samples per channel at 32 kHz (5.18 s). The validator still compares only PCM shape, so the 15 s qualification needs the expected-duration check.
- Pilot timeouts (P2): per that thread, the request timeouts in the two pilot plans exceed their reservations. This affects how pilot failures are attributed, not the final A/B/B/A result.
Not checked: raw media, logs and weights are external, and I did not rerun the validator, models or devices. The measurements remain author-reported.
Refs #293
Five-second delivery
Deliver the MiniMax-H3 Ref2VA Ascend campaign requested in #293: 407.27 → 108.12 s HTTP E2E, 73.45% lower (3.77× inverse-latency speedup) on eight Ascend 910B1 NPUs. All 26 requests, 14 quality comparisons and 32 actual worker exits pass. Fifteen-second qualification is explicitly deferred.
This adds frozen plans, portable client/coverage/validator scripts, source attribution, raw-value sanitized receipts, an integrity manifest and repeated-trial charts. The final A/B/B/A campaign uses two independent servers per arm, two warmups plus three measured requests per server. Sample SD is across server means, not six independent trials. Mean sampled peak HBM rises 3.10%. The family audio cosine of 0.813411 is close to the 0.80 gate; this is lossy caching, not semantic-quality certification.
Final report and source/benefit table distinguishes the qualified existing Cache-DiT profile from the optional device-KV single-pair pilot (44.86% lower latency, not enabled in final profile). Every adopted optimization has an origin and evidence limit. Rejected audio ownership, failed RPC attempts and the interrupted first final campaign are retained without mixing their samples into the accepted result.
All downstream engine changes are consolidated in vLLM-Omni #9698. Measured runtime:
098b51f6b96dd160a0959e74e8822422527ff06d; baseline:c7f56f0155953c7d4a77276a2f4fcbd844ab0da2. Baseline teardown uses matched diagnostic 60/90-second budgets; candidate uses native production budgets.Validation: evidence inventory hashes and focused inventory tests; complete target-NPU campaign; all 14 frozen-gate comparisons. Raw media, model weights and host logs stay external, with public hashes and explicit availability limits. No global optimum, p95/p99, throughput, CUDA or 15-second result is claimed.
Agent-assisted-by: Codex
Remaining work: 15-second qualification is deferred. Goal acceptance remains with the requester; this PR does not request issue closure. Knowledge authorship policy reports Not applicable for these campaign evidence paths.