Fix Qwen2.5-Omni Token2Wav DiT rotary embedding layout (interleaved cos/sin) by HenryVarro666 · Pull Request #47403 · huggingface/transformers

HenryVarro666 · 2026-07-18T21:26:06Z

What does this PR do?

The Token2Wav DiT applies its rotary embedding with the interleaved convention — the released checkpoint pairs adjacent channels (2i, 2i + 1) — but since #39847 (🚨 [v5] Refactor RoPE for layer types) Qwen2_5OmniDiTRotaryEmbedding inherited LlamaRotaryEmbedding.forward, which builds cos/sin with the half-split layout torch.cat((freqs, freqs)), while the module kept its interleaved rotate (rotate_half_codec). A half-split cos/sin combined with an interleaved rotate is not a valid rotation: RoPE's defining property — ⟨R_m·q, R_n·k⟩ depending only on the relative offset m − n — is destroyed. It is applied to head 0 of every DiT attention layer, silently degrading the predicted mel-spectrogram and thus the synthesized audio for return_audio=True.

This is a regression, not the checkpoint design. Before #39847 (e.g. v4.57.0), Qwen2_5OmniDiTRotaryEmbedding.forward built the embedding as torch.stack((freqs, freqs), dim=-1).reshape(...) — the interleaved layout matching rotate_half_codec. The original add PR #36752 was integration-tested against Qwen/Qwen2.5-Omni-7B with that convention, so the released weights expect it.

The fix (as suggested in review — thanks @ebezzam!)

Rather than restoring the interleaved cos/sin layout (the first version of this PR), the fix now keeps the standard half-split rotary embedding and aligns the application with library conventions, per the review:

drop the custom apply_rotary_pos_emb/rotate_half_codec and use the kernelized apply_rotary_pos_emb from Llama;
add a deinterleave_head_dim helper and reorder q/k (head 0 only) from the checkpoint's interleaved layout to the half-split layout before the rotation.

This is numerically equivalent to the checkpoint's original interleaved rotation: deinterleave_head_dim maps each pair (2i, 2i + 1) to positions (i, d/2 + i), which the half-split cos/sin rotates by the same θ_i. The rotated q/k differ from the interleaved formulation only by a channel permutation applied consistently to both, which the attention scores q·kᵀ are invariant to — verified: attention scores match the interleaved-layout fix to ~1e-5 (fp32), while the broken main path differs by ~4e1. RoPE translation invariance is restored (drift ~1e-5 across offsets vs ~10.7 on main).

The text model's mrope (Qwen2_5OmniRotaryEmbedding) legitimately uses the half-split convention end-to-end and is untouched; qwen3_omni_moe has its own rotary and is unaffected (independently confirmed by @pepepeboom, who also reproduced the root cause — thanks!).

Tests

Replaces the previous unit tests with an integration test (test_small_model_integration_test_token2wav_regression), as requested in review: it drives the Token2Wav module of the released checkpoint with pre-recorded talker codec tokens (fixture produced by the reproducer below) and checks the generated waveform against expected outputs — torch.testing.assert_close on a strided sample signature spanning the whole utterance (2,040 points; a contiguous head slice would only cover the leading silence) plus the overall RMS. Token2Wav runs in fp32 with a single seeded torch.randn, so the comparison is stable on a given device. Measured margins (H100):

	max abs diff on signature (atol 5e-3)	RMS (expected 0.0344, rtol 1%)
this PR	2.8e-4 (same-device noise level; 7.7e-4 with TF32 disabled)	0.034378 ✅
`main`	1.9e-1 — 38× the tolerance	0.00726 — off by 79% ❌

The test deliberately replays captured codes through model.token2wav instead of running the full generate() pipeline: the bf16 thinker/talker greedy decode can flip tokens across GPU architectures (see the per-device Expectations tables already in this test file), which would make an end-to-end audio comparison flaky, while the fp32 Token2Wav component — where the regression lives — is stable.

Reproducer script that produced the fixture (run on the fixed branch)

"""Reproducer for the Qwen2.5-Omni Token2Wav audio regression fixture (PR #47403).

Runs a fully deterministic `generate()` (greedy thinker + greedy talker, seed 0), captures the
talker codec tokens fed into Token2Wav, then replays them through `model.token2wav` exactly as
`test_small_model_integration_test_token2wav_regression` does, and stores codes + expected
waveform in the JSON fixture.
"""
import json
import os

import torch

from transformers import AutoProcessor, Qwen2_5OmniForConditionalGeneration

MODEL_ID = "Qwen/Qwen2.5-Omni-7B"

processor = AutoProcessor.from_pretrained(MODEL_ID)
model = Qwen2_5OmniForConditionalGeneration.from_pretrained(MODEL_ID, dtype=torch.bfloat16, device_map="auto")

messages = [
    {
        "role": "system",
        "content": [
            {
                "type": "text",
                "text": "You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.",
            }
        ],
    },
    {"role": "user", "content": [{"type": "text", "text": "Say hello."}]},
]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt", padding=True).to(model.device)

captured = {}
orig_forward = model.token2wav.forward


def capture_forward(code, **kwargs):
    captured["codes"] = code.detach().cpu()
    return orig_forward(code, **kwargs)


model.token2wav.forward = capture_forward
torch.manual_seed(0)
with torch.no_grad():
    text_ids, audio = model.generate(
        **inputs,
        thinker_do_sample=False,
        thinker_temperature=0,
        thinker_max_new_tokens=32,
        talker_do_sample=False,
        talker_max_new_tokens=120,
    )
model.token2wav.forward = orig_forward

codes = captured["codes"]
print("decoded text:", repr(processor.decode(text_ids[0], skip_special_tokens=True)))
print("codes:", codes.shape, "in-generate audio:", audio.shape)

# Replay through token2wav exactly as the test does (this is what the fixture certifies).
speaker_params = model.speaker_map["Chelsie"]
torch.manual_seed(0)
with torch.no_grad():
    waveform = model.token2wav(
        codes.to(model.device),
        conditioning=speaker_params["cond"].to(model.device).float(),
        reference_mel=speaker_params["ref_mel"].to(model.device).float(),
    )
waveform = waveform.float().cpu().squeeze()
rms = waveform.pow(2).mean().sqrt().item()
print("replay waveform:", tuple(waveform.shape), "rms:", rms)

# Determinism sanity: replay a second time. Same-device reruns agree to ~4e-4 (kernel
# autotuning noise), far below the test tolerance (atol 5e-3) and the bug magnitude (~1.9e-1).
torch.manual_seed(0)
with torch.no_grad():
    waveform2 = model.token2wav(
        codes.to(model.device),
        conditioning=speaker_params["cond"].to(model.device).float(),
        reference_mel=speaker_params["ref_mel"].to(model.device).float(),
    ).float().cpu().squeeze()
print("rerun max abs diff (determinism):", (waveform - waveform2).abs().max().item())

onset = int((waveform.abs() > 0.01).nonzero()[0]) if (waveform.abs() > 0.01).any() else -1
print("speech onset sample:", onset, "of", waveform.shape[-1])
STRIDE = 28
fixture = {
    "talker_codes": codes.squeeze(0).tolist(),
    "waveform_length": waveform.shape[-1],
    "waveform_stride": STRIDE,
    "waveform_strided": waveform[::STRIDE].tolist(),
    "waveform_rms": rms,
}
out = "tests/fixtures/qwen2_5_omni/expected_audio_regression.json"
os.makedirs(os.path.dirname(out), exist_ok=True)
with open(out, "w") as f:
    json.dump(fixture, f)
print("fixture written:", out, f"({os.path.getsize(out) / 1024:.0f} KB)")

Before submitting

Was this discussed/approved via a GitHub issue? Qwen2.5-Omni Token2Wav DiT uses mismatched RoPE layout (half-split cos/sin with interleaved rotate) — regression from #39847 #47328
Did you write any new necessary tests?

Who can review?

@eustlb @ebezzam

Thanks to @pepepeboom for independently reproducing the root cause and confirming the regression on main.

Disclosure: the root-cause analysis, regression bisect, reproduction, and tests in this PR were prepared with AI assistance and independently verified by me.

The Token2Wav DiT applies an interleaved rotate (`rotate_half_codec`, pairing channels (2i, 2i+1)), but since huggingface#39847 the DiT rotary embedding built cos/sin with the half-split `torch.cat((freqs, freqs))` layout inherited from `LlamaRotaryEmbedding`. A half-split cos/sin combined with an interleaved rotate is not a valid rotation: RoPE's translation invariance is destroyed on head 0 of every DiT attention layer, degrading the predicted mel-spectrogram (and thus the synthesized audio). Override `Qwen2_5OmniDiTRotaryEmbedding.forward` to emit the interleaved `repeat_interleave(freqs, 2)` layout, restoring the pre-huggingface#39847 behavior that matches `rotate_half_codec` and the released weights (originally integration-tested in huggingface#36752). Add CPU regression tests asserting the interleaved layout and RoPE translation invariance.

ebezzam

@HenryVarro666 thanks for spotting this regression and opening a PR!

I was able to observe it myself with this script and these output audio to compare generated audio (BEFORE and AFTER fix).

I agree with you that at this point we don't want to change the released weights. Although I think we can do a different fix to better align with library conventions, as that was the intention of #39847. To that end, I've left some comment.

cc @zucchini-nlp for another opinion!

ebezzam · 2026-07-20T13:46:51Z

+    @torch.no_grad()
+    @dynamic_rope_update  # power user: used with advanced RoPE types (e.g. dynamic rope)
+    def forward(self, x, position_ids):
+        inv_freq_expanded = self.inv_freq[None, :, None].float().expand(position_ids.shape[0], -1, 1).to(x.device)
+        position_ids_expanded = position_ids[:, None, :].float()
+
+        device_type = x.device.type if isinstance(x.device.type, str) and x.device.type != "mps" else "cpu"
+        with maybe_autocast(device_type=device_type, enabled=False):  # Force float32
+            freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
+            # The DiT applies an interleaved rotary (``rotate_half_codec``, pairing channels
+            # ``(2i, 2i + 1)``), so cos/sin must interleave the frequencies to match. The
+            # half-split ``torch.cat((freqs, freqs))`` layout mismatches the rotate and breaks
+            # RoPE's translation invariance. Regression from #39847; see #47328.
+            emb = torch.repeat_interleave(freqs, 2, dim=-1)
+            cos = emb.cos() * self.attention_scaling
+            sin = emb.sin() * self.attention_scaling
+
+        return cos.to(dtype=x.dtype), sin.to(dtype=x.dtype)


let's keep as is and rather change apply_rotary_pos_emb and how we use it (see here)

Suggested change

@torch.no_grad()

@dynamic_rope_update # power user: used with advanced RoPE types (e.g. dynamic rope)

def forward(self, x, position_ids):

inv_freq_expanded = self.inv_freq[None, :, None].float().expand(position_ids.shape[0], -1, 1).to(x.device)

position_ids_expanded = position_ids[:, None, :].float()

device_type = x.device.type if isinstance(x.device.type, str) and x.device.type != "mps" else "cpu"

with maybe_autocast(device_type=device_type, enabled=False): # Force float32

freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)

# The DiT applies an interleaved rotary (``rotate_half_codec``, pairing channels

# ``(2i, 2i + 1)``), so cos/sin must interleave the frequencies to match. The

# half-split ``torch.cat((freqs, freqs))`` layout mismatches the rotate and breaks

# RoPE's translation invariance. Regression from #39847; see #47328.

emb = torch.repeat_interleave(freqs, 2, dim=-1)

cos = emb.cos() * self.attention_scaling

sin = emb.sin() * self.attention_scaling

return cos.to(dtype=x.dtype), sin.to(dtype=x.dtype)

ebezzam · 2026-07-20T13:50:52Z

(not on you) but let's actually use apply_rotary_pos_emb from Llama to take advantage of the kernel implementation for speed

from ..llama.modeling_llama import LlamaRotaryEmbedding, apply_rotary_pos_emb, rotate_half

ebezzam · 2026-07-20T13:51:59Z

we can remove this custom implementation and use that of Llama, which has a kernel implementation

and instead define a method

def deinterleave_head_dim(x: torch.Tensor) -> torch.Tensor: """Reorder the head dim from interleaved RoPE layout `(x0, x1, x2, x3, ...)` to the standard half-split layout `(x0, x2, ..., x1, x3, ...)` so the kernelized `apply_rotary_pos_emb` can be used. """ return torch.cat((x[..., 0::2], x[..., 1::2]), dim=-1)

and use it here like so:

# apply rotary position embedding # Due to training process, only first head is applied with RoPE, will be fixed at next release # The checkpoint uses interleaved RoPE, so deinterleave q/k before the half-split rotation. cos, sin = position_embeddings query[:, :1], key[:, :1] = apply_rotary_pos_emb( deinterleave_head_dim(query[:, :1]), deinterleave_head_dim(key[:, :1]), cos, sin )

ebezzam · 2026-07-20T14:00:56Z

+
+
+@require_torch
+class Qwen2_5OmniDiTRotaryEmbeddingTest(unittest.TestCase):


instead of this test, let's instead add an integration test to this collection which checks that the audio outputs don't regress.

For example, see this test which uses a separate script to compute expected output and applies the model to make sure outputs stay similar here. It is for a codec model, but a similar idea can be applied for audio generation models (be sure to set a seed and temperature=0 for deterministic outputs).

…av DiT

HenryVarro666 · 2026-07-20T15:31:51Z

Thanks a lot for the thorough review @ebezzam — and for independently confirming the regression with your script and the before/after audio!

Your direction sounds good to me: deinterleaving q/k ahead of the kernelized apply_rotary_pos_emb applies exactly the same rotation as the interleaved cos/sin + rotate_half_codec (the rotated q/k differ only by a channel permutation applied consistently to both, which the attention scores are invariant to — I verified the attention logits match to ~1e-14). So it stays checkpoint-faithful while aligning with the library's RoPE conventions and picking up the kernel path. I've updated the PR accordingly, following your suggestions.

For the integration test, I followed the xcodec2 fixture pattern with one adaptation: it replays talker codec tokens captured from a deterministic seeded generate() through model.token2wav (fp32), rather than comparing end-to-end generate() audio. The bf16 thinker/talker greedy decode can flip tokens across GPU architectures — the per-device Expectations tables in this test file show exactly that for text — which would make an end-to-end audio fixture arch-fragile, while the fp32 Token2Wav component (where the regression lives) is stable. Measured on the fixture hardware (H100): the fixed path matches the fixture to ~3e-4 (tolerance 5e-3, and ~8e-4 with TF32 disabled as a numerics-perturbation proxy), while main fails by ~38× with RMS off by 79%. Happy to switch to an end-to-end comparison if you'd prefer that, though!

zucchini-nlp · 2026-07-21T11:22:07Z

+def deinterleave_head_dim(x: torch.Tensor) -> torch.Tensor:
+    """Reorder the head dim from interleaved RoPE layout `(x0, x1, x2, x3, ...)` to the standard half-split layout
+    `(x0, x2, ..., x1, x3, ...)` so the kernelized `apply_rotary_pos_emb` can be used.
+    """
+    return torch.cat((x[..., 0::2], x[..., 1::2]), dim=-1)
+


hmm, maybe we can permute once in conversion mapping instead of de-interleaving at each forward run?

good point! so adding a mapping here?

yep, there is a class PermuteForRope which currently is broken but easy to fix the permutation axis, and add in mapping

@HenryVarro666 can you switch to this and remove deinterleave_head_dim?

Thanks both, happy to move toward the library convention here!

One wrinkle worth flagging before wiring it up: in this DiT, RoPE is applied to head 0 only (query[:, :1] / key[:, :1]), while to_q/to_k are a single fused Linear(hidden_size, num_heads * head_dim). PermuteForRope permutes all heads, so applying it to the whole to_q/to_k weight would deinterleave head 0 (what we want) but also permute heads 1–15, which are used un-rotated — that would change their attention scores. So the conversion op has to permute only the first head_dim rows (and, since to_q/to_k carry bias, the first head_dim bias entries too), leaving the other heads untouched — i.e. a small custom op rather than PermuteForRope as-is, plus a matching reverse_op so save_pretrained round-trips back to the interleaved checkpoint.

I'm glad to implement that (permute the head-0 slice at load via a qwen2_5_omni_dit conversion mapping, and drop deinterleave_head_dim). One thing to weigh: since the deinterleave here only ever touches head 0's (batch, 1, seq, head_dim) slice, the per-forward cost is negligible, so in this specific case the speed benefit of moving it to load time is marginal. Either is fine by me:

(a) keep the current runtime deinterleave_head_dim (localized, already verified against the checkpoint), or

(b) move it to a load-time head-0 conversion op as suggested.

Just let me know which you'd prefer and I'll push it. Either way, on the test I'll drop the fixture file and switch to a lightweight integration check (~50 codes in / 50 output values compared via Expectation) with a linked gist for reproducing the expected values, following the audioflamingo3 example. 👍

huh, didn't know that, it makes DiT similar to some recent deepseek with NOPE dims. In that case i think it is not worth converting at run-time and we can keep the de-interleaved

zucchini-nlp · 2026-07-21T11:24:55Z

not a fan of this, can we check a slice of logits/codes/mean/std and keep the expected values as testing_utils.Expectation?

audio conventions of adding such fixture files 🙈 but I'm also fine with what @zucchini-nlp is suggesting!

adding a link to a script for computing the expected values (as mentioned here) so we can reproduce them if needed in the future

only inputing 50 codes and comparing 50 output values so the test is lighter

ahh i see, didn't know that. Yeah, whatever works for audio better, just suggesting how we did in text generation models :)

zucchini-nlp · 2026-07-21T11:25:42Z

        self.assertFalse(torch.isnan(output[1]).any().item())

+    @slow
+    def test_small_model_integration_test_token2wav_regression(self):


hopefully this fits in out runners 😅

ebezzam · 2026-07-21T11:50:52Z

+        Regression test for #47328: the Token2Wav DiT applies an interleaved rotary embedding
+        (pairing channels `(2i, 2i + 1)`), so its `cos`/`sin` must be applied accordingly; a
+        half-split application silently degrades the generated audio. This test drives the
+        Token2Wav module of the released checkpoint with pre-recorded talker codec tokens and
+        checks that the generated waveform does not regress. The waveform is compared on a
+        strided sample signature spanning the whole utterance (a contiguous head slice would
+        only cover leading silence) plus its overall RMS.
+
+        The fixture was produced with the reproducer script in PR #47403 (deterministic
+        `generate()` run: greedy thinker/talker, `torch.manual_seed(0)`, then the captured
+        talker codes replayed through `model.token2wav` exactly as below).


can you instead put a link to a gist like this? we we can reproduce the expected output in the future

ebezzam · 2026-07-21T11:52:56Z

audio conventions of adding such fixture files 🙈 but I'm also fine with what @zucchini-nlp is suggesting!

adding a link to a script for computing the expected values (as mentioned here) so we can reproduce them if needed in the future

only inputing 50 codes and comparing 50 output values so the test is lighter

HenryVarro666 · 2026-07-22T20:11:28Z

Thanks @zucchini-nlp @ebezzam, that makes sense. I've kept the runtime deinterleave_head_dim (source unchanged from the last push), since as you note this head-0-only RoPE is closer to the NoPE-dim case and the per-forward deinterleave only ever touches head 0's (batch, 1, seq, head_dim) slice.

I reworked the audio regression test along the lines you both suggested (pushed to the same branch):

Dropped the expected_audio_regression.json fixture.
It now feeds a fixed 50 talker codes into token2wav and checks a 50-sample signature strided across the whole utterance plus the overall std/RMS, stored inline as Expectations.
Added a linked gist that recomputes the expected values from those 50 codes, following the audioflamingo3 example: https://gist.github.com/HenryVarro666/4f633e37835ca9e019ed7f789309775b

Values were captured on an H100 (cc 9.0). On main the regression trips it clearly (30/50 signature samples off, worst 0.078 vs the 5e-3 tolerance, and overall energy collapses ~3.6x). Happy to add an Expectations entry for your CI hardware if it differs.

zucchini-nlp

Thanks a lot, lgtm! Lets wait for Eric and then we can merge

ebezzam · 2026-07-23T16:08:20Z

run-slow: qwen2_5_omni

github-actions · 2026-07-23T16:08:20Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen2_5_omni

github-actions · 2026-07-23T16:09:47Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen2_5_omni"]
quantizations: []

HuggingFaceDocBuilderDev · 2026-07-23T16:19:40Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2026-07-23T16:20:40Z

CI recap

Dashboard: View test results in Grafana
Latest run: 29755563644:2
Result: success | Jobs: 15 | Tests: 172,163 | Failures: 0 | Duration: 13h 53m

github-actions · 2026-07-23T16:26:09Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	80a1af70	workflow commit (merge commit)
PR	0ceab83d	branch commit (from PR)
main	00d795b4	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

ebezzam

thanks @HenryVarro666 for the updates! I pushed some small changes and ran our tests. Everything checks out 👍

thanks for the contribution!

ebezzam · 2026-07-23T16:29:33Z

+        signature = waveform[:: waveform.shape[-1] // 50][:50]
+        expected_signatures = Expectations(
+            {
+                ("cuda", 8): torch.tensor([


added outputs for our CI/machines

ebezzam · 2026-07-23T16:30:29Z

+    @slow
+    def test_small_model_integration_test_token2wav_regression(self):
+        """
+        reproducer (for the expected values below): https://gist.github.com/ebezzam/12286028df44e91434f7c770efc4e5b5


shortened description to just the reproducer (copied yours and made small tweaks in the comment header)

HenryVarro666 · 2026-07-23T17:10:57Z

Thanks @ebezzam @zucchini-nlp for the careful review, and for handling the merge-main and CI outputs on your side. Glad to get this one fixed in.

…os/sin) (huggingface#47403) * Fix Qwen2.5-Omni Token2Wav DiT rotary embedding layout The Token2Wav DiT applies an interleaved rotate (`rotate_half_codec`, pairing channels (2i, 2i+1)), but since huggingface#39847 the DiT rotary embedding built cos/sin with the half-split `torch.cat((freqs, freqs))` layout inherited from `LlamaRotaryEmbedding`. A half-split cos/sin combined with an interleaved rotate is not a valid rotation: RoPE's translation invariance is destroyed on head 0 of every DiT attention layer, degrading the predicted mel-spectrogram (and thus the synthesized audio). Override `Qwen2_5OmniDiTRotaryEmbedding.forward` to emit the interleaved `repeat_interleave(freqs, 2)` layout, restoring the pre-huggingface#39847 behavior that matches `rotate_half_codec` and the released weights (originally integration-tested in huggingface#36752). Add CPU regression tests asserting the interleaved layout and RoPE translation invariance. * Use kernelized apply_rotary_pos_emb with deinterleaved q/k in Token2Wav DiT * Replace DiT rotary unit tests with Token2Wav audio regression test * Replace audio-regression fixture with inline Expectations-based test * Add expected outputs for our CI. --------- Co-authored-by: Eric B <ebezzam@gmail.com> Co-authored-by: Eric Bezzam <4757445+ebezzam@users.noreply.github.com>

The generated `modeling_*.py` files for qwen3_tts and its two audio tokenizers were last produced on 2026-06-30 and had drifted from the parent models they inherit from (qwen3, qwen2_5_omni, qwen3_omni_moe, mimi). Re-running `utils/modular_model_converter.py` picks up those changes: `nn.Buffer` instead of `register_buffer` (huggingface#47722), the `inv_freq` linting annotation removal (huggingface#47753), the kernel decorator rename to `use_kernel_forward_from_hub` (huggingface#46883), the Token2Wav DiT interleaved-RoPE fix (huggingface#47403), removal of the legacy FlashAttention2 / SdpaAttention classes, and assorted spelling fixes (huggingface#47684). Three fixes were needed in the modular sources so that regeneration does not regress behaviour: - Move `Qwen3TTSConfig.get_text_config` into `modular_qwen3_tts.py`. It had only ever been added to the generated configuration file, so the converter dropped it. - Pin `Qwen3TTSTalkerModel._no_split_modules` to `["Qwen3TTSTalkerDecoderLayer"]`. The converter resolves the inherited value with the most common prefix (`Qwen3TTS`) and would otherwise point at the wrong decoder layer class. - Add a `Qwen3TTSTokenizerMultiCodebookSnakeBeta` alias and use it in the code2wav decoder. Upstream renamed `SnakeBeta` to `Qwen2_5OmniSnakeBeta`, which left an undefined `SnakeBeta` reference in the generated file. Conversion is idempotent, `ruff check` / `ruff format --check` pass, and all touched files compile. Claude-Session: https://claude.ai/code/session_01WLU94QdJCmuVf181X98tno

ebezzam reviewed Jul 20, 2026

View reviewed changes

ebezzam mentioned this pull request Jul 20, 2026

Add support for batched Qwen2.5/3-Omni audio generation #47186

Merged

HenryVarro666 added 2 commits July 20, 2026 10:27

Use kernelized apply_rotary_pos_emb with deinterleaved q/k in Token2W…

ddead35

…av DiT

Replace DiT rotary unit tests with Token2Wav audio regression test

a3c47fa

zucchini-nlp reviewed Jul 21, 2026

View reviewed changes

ebezzam reviewed Jul 21, 2026

View reviewed changes

Replace audio-regression fixture with inline Expectations-based test

de8c561

zucchini-nlp approved these changes Jul 23, 2026

View reviewed changes

ebezzam and others added 2 commits July 23, 2026 18:04

Add expected outputs for our CI.

9a3eb07

Merge branch 'main' into fix-47328-dit-rotary-interleaved

0ceab83

ebezzam approved these changes Jul 23, 2026

View reviewed changes

ebezzam added this pull request to the merge queue Jul 23, 2026

Merged via the queue into huggingface:main with commit 363b2a9 Jul 23, 2026
115 checks passed



		@require_torch
		class Qwen2_5OmniDiTRotaryEmbeddingTest(unittest.TestCase):

Conversation

HenryVarro666 commented Jul 18, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

The fix (as suggested in review — thanks @ebezzam!)

Tests

Before submitting

Who can review?

Uh oh!

ebezzam left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HenryVarro666 commented Jul 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HenryVarro666 commented Jul 22, 2026

Uh oh!

zucchini-nlp left a comment

Choose a reason for hiding this comment

Uh oh!

ebezzam commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Jul 23, 2026

Uh oh!

github-actions Bot commented Jul 23, 2026

CI recap

Uh oh!

github-actions Bot commented Jul 23, 2026

CI Results

Commit Info

Uh oh!

ebezzam left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

HenryVarro666 commented Jul 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

HenryVarro666 commented Jul 18, 2026 •

edited by github-actions Bot

Loading