Skip to content

Stop MiniMax-H3 aborting on default cfg-scale and on --vae-on-cpu - #3

Merged
danielhanchen merged 1 commit into
masterfrom
fix/h3-cfg-and-audio-vae-cpu
Aug 8, 2026
Merged

Stop MiniMax-H3 aborting on default cfg-scale and on --vae-on-cpu#3
danielhanchen merged 1 commit into
masterfrom
fix/h3-cfg-and-audio-vae-cpu

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Two independent SIGABRTs in MiniMax-H3, both reachable from ordinary invocations. Neither is a bad-input problem; one of them fires on the default arguments.

1. H3 crashes on the default --cfg-scale

sd.cpp defaults --cfg-scale to 7.0. H3 aborts at any value above 1.0:

ggml/src/ggml.c: GGML_ASSERT(!ggml_is_transposed(a)) failed

So sd-cli --mode vid_gen on H3, with no cfg flag at all, dies with SIGABRT instead of rendering.

--cfg-scale before
1.0 renders
1.5 abort, exit 134
4.0 abort, exit 134
omitted (7.0) abort, exit 134

Cause. H3 is distilled with guidance baked in and has no negative-prompt semantics. Its empty uncond prompt encodes to zero tokens, so constructing the uncond branch produces the transposed tensor that assert rejects.

Fix. There is no correct cfg > 1 behaviour to implement for a CFG-free model, so warn and clamp to 1.0 rather than asserting deep inside ggml:

[WARN] MiniMax-H3 is a distilled, CFG-free model; forcing cfg-scale from 7.00 to 1.0

2. --vae-on-cpu aborts whenever an audio VAE is present

ggml/src/ggml-cpu/ops.cpp: GGML_ASSERT(src0->type == GGML_TYPE_F16) failed

Bisected across the offload flags: --vae-on-cpu with --audio-vae aborts; the same command without --audio-vae renders fine; --vae-tiling, --offload-to-cpu and --clip-on-cpu are each individually fine. So the trigger is the audio VAE, not the video one.

Cause. ggml_conv_1d / ggml_conv_1d_dw build an F16 im2col, and the CPU backend additionally requires the kernel to be F16 (ggml_compute_forward_im2col_f16 asserts it). audio_conv_weight_type maps only BF16 to F16 and lets F32 through, so an F32 audio conv kernel reaches a CPU path that cannot run it. Converting the checkpoint to fp16 does not help: the type is imposed in this code, not by the file, so it cannot be worked around by shipping different weights.

Fix. Cast the kernel in-graph, and only when the runner is genuinely on a CPU backend, so GPU precision is untouched and no weight is degraded at load time.

Worth noting for review: all four ggml_conv_1d sites in ltx_audio_vae.hpp need this, not just the two module-level ones. The STFT forward_basis and the transposed-conv reversed_filter are computed F32 tensors that hit the same assert. An earlier attempt patching only the module sites still aborted identically. depthwise_conv_transpose1d consequently takes the runner context instead of a bare ggml_context.

Verification

minimax_h3_fl2va_pruned q4_K, 640x384, 25 frames, 4 steps, seed 1234, --rng cpu, --backend te=cpu, --diffusion-fa.

case before after
--cfg-scale 4.0 exit 134 exit 0
cfg omitted (default 7.0) exit 134 exit 0
--vae-on-cpu --audio-vae exit 134 exit 0
full low_vram flag set exit 134 exit 0
baseline exit 0 exit 0, byte-identical
--offload-to-cpu exit 0 exit 0, byte-identical

The baseline and offload outputs are byte-identical before and after, so nothing on the GPU path moved. The two CPU-VAE cases differ from the baseline by 114 bytes, which is the expected consequence of their conv kernels now running F16 on the CPU.

Scope

Fix 2 lives in the shared LTX audio VAE, so it also applies to LTX itself; it is a no-op unless a conv kernel is F32 and the runner is on CPU. Fix 1 is gated on the MiniMax-H3 version check and touches no other family.

Two independent SIGABRTs, both reachable from ordinary invocations.

1. cfg-scale. H3 is distilled with guidance baked in and has no negative prompt
semantics; its empty uncond prompt encodes to zero tokens, so building the uncond
branch trips GGML_ASSERT(!ggml_is_transposed(a)) in ggml.c. sd.cpp defaults
--cfg-scale to 7.0, so a bare `sd-cli --mode vid_gen` on H3 crashes rather than
rendering. Measured: cfg 1.0 renders, cfg 1.5 and cfg 4.0 both abort, exit 134.

There is no correct cfg > 1 behaviour to implement for a CFG-free model, so warn
and clamp to 1.0 instead of asserting deep inside ggml.

2. --vae-on-cpu. ggml_conv_1d and ggml_conv_1d_dw build an F16 im2col, and the
CPU backend additionally requires the kernel itself to be F16;
ggml_compute_forward_im2col_f16 asserts it. audio_conv_weight_type maps only
BF16 to F16 and lets F32 through, so H3's F32 audio conv kernels abort with
GGML_ASSERT(src0->type == GGML_TYPE_F16) as soon as the audio VAE decodes on the
CPU. Converting the checkpoint to fp16 does not help: the type is imposed here,
not by the file.

Cast the kernel in-graph, and only when the runner is actually on a CPU backend,
so GPU precision is untouched and no weight is degraded at load. All four
conv_1d sites in this file need it, not just the module ones: the STFT
forward_basis and the transposed-conv reversed_filter are computed F32 tensors
that reach the same assert. depthwise_conv_transpose1d therefore takes the
runner context rather than a bare ggml_context.

Verified on minimax_h3_fl2va_pruned q4_K, 640x384, 25 frames, 4 steps, seed 1234:

    case                              before      after
    --cfg-scale 4.0                   exit 134    exit 0
    cfg omitted (default 7.0)         exit 134    exit 0
    --vae-on-cpu --audio-vae          exit 134    exit 0
    full low_vram flag set            exit 134    exit 0
    baseline                          exit 0      exit 0, byte-identical
    --offload-to-cpu                  exit 0      exit 0, byte-identical

The two CPU-VAE cases differ from the baseline by 114 bytes, which is the
expected consequence of their conv kernels now running F16 on the CPU.
@danielhanchen

Copy link
Copy Markdown
Member Author

Validated both fixes end to end on the CUDA build of this branch, MiniMax-H3 with the audio VAE attached, 640x384, 25 frames, 4 steps.

The default cfg-scale abort. With --cfg-scale omitted entirely, so sd.cpp applies its own default of 7.0:

generate_video completed in 14.71s
save result video to .../pr3_defaultcfg.webm
exit 0

Before the clamp this was GGML_ASSERT(!ggml_is_transposed(a)) failed, SIGABRT, exit 134. That is the plain --mode vid_gen invocation with no guidance flag, which is what anyone following the README would type first.

The --vae-on-cpu abort with an audio VAE.

decode_first_stage completed, taking 77.25s
generate_video completed in 97.71s
exit 0

Before the conv1d kernel cast this was GGML_ASSERT(src0->type == GGML_TYPE_F16) failed, SIGABRT, exit 134. Worth noting all four ggml_conv_1d / ggml_conv_1d_dw sites in ltx_audio_vae.hpp needed the cast, not just the two obvious ones. Patching two left the abort completely unchanged, which is a good way to convince yourself a fix works when it does not.

One measurement worth having alongside the fix: --vae-on-cpu is now correct but very expensive on H3. decode_first_stage goes from 3.25s on the GPU to 77.25s on the CPU, a 24x hit, taking the whole render from 14.71s to 97.71s. So the flag is worth having work rather than crash, but it is not a good default for this model, and a caller with the VRAM headroom should keep the VAE on device.

@danielhanchen
danielhanchen merged commit 1c0397a into master Aug 8, 2026
11 of 20 checks passed
@danielhanchen
danielhanchen deleted the fix/h3-cfg-and-audio-vae-cpu branch August 8, 2026 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant