Issues

• gather_for_metrics keeps padding duplicates for a sized IterableDataset with dispatch_batches=False

  huggingface/accelerate#4359 · Filter by author Zhuoxi2000 opened on Oct 1, 2026
  1

• FSDP2 get_state_dict() does not preserve tied-weight aliasing

  huggingface/accelerate#4339 · Filter by author UnitedSnakes opened on Sep 26, 2026

• Resuming with load_state + skip_first_batches (checkpointing example) does not restore the shuffle order

  huggingface/accelerate#4330 · Filter by author SeverinVisionary opened on Sep 24, 2026

• [Bug] RNG state restoration failure swallowed silently — resumed training diverges without warning

  huggingface/accelerate#4283 · Filter by author cmaguire001 opened on Sep 19, 2026
  1

• PrefixedDataset violates the Mapping contract

  huggingface/accelerate#4257 · Filter by author betacatsling opened on Sep 14, 2026

• [Bug] Fused AdamW AMP overflow leaves step_was_skipped false and advances the scheduler

  huggingface/accelerate#4242 · Filter by author gss10282023 opened on Sep 10, 2026

• use_stateful_dataloader + num_workers>0 draws epoch-0 permutation during prepare() before cross-rank RNG sync — corrupts DDP data partition

  huggingface/accelerate#4204 · Filter by author GoldenStain opened on Sep 2, 2026
  4

• Skip automatic DataLoader sharding when the loader is already rank-sharded

  huggingface/accelerate#4199 · Filter by author YeonwooSung opened on Sep 2, 2026
  1

• use_stateful_dataloader under multi-process training restores the cursor, not the shuffle order — sampler permutation is never serialized

  huggingface/accelerate#4195 · Filter by author GoldenStain opened on Aug 30, 2026
  2

• Feature request: FSDP2 QLoRA

  feature request
  huggingface/accelerate#3874 · Filter by author ojh31 opened on Dec 5, 2025
  3

• datasets Iterable Dataset sharding support

  feature request
  huggingface/accelerate#3547 · Filter by author ValMystletainn opened on May 5, 2025
  2

• [CPU offload hooks] hooks with overlapped transfers and computations

  feature request
  huggingface/accelerate#3267 · Filter by author sayakpaul opened on Nov 29, 2024
  1

• accelerator.prepare

  huggingface/accelerate#3265 · Filter by author wfan1203 opened on Nov 26, 2024
  56

• [BUG] Accelerate 1.0.1 failed to train multiple zero-3 models

  wip
  huggingface/accelerate#3170 · Filter by author nrailg opened on Oct 15, 2024
  5

• DeepSpeedEngineWrapper.backward() does a bit too much

  enhancementfeature request
  huggingface/accelerate#2951 · Filter by author zhc7 opened on Jul 22, 2024
  19

• Plan to support FSDP2?

  enhancementfeature request
  huggingface/accelerate#2873 · Filter by author ByronHsu opened on Jun 19, 2024
  12

  

• Dataloader WeightedRandomSampler + Distributed Training

  enhancementfeature request
  huggingface/accelerate#2865 · Filter by author FrsECM opened on Jun 17, 2024
  4

• gather objects in TPU is not supported

  feature requestTPU
  huggingface/accelerate#2858 · Filter by author carlesoctav opened on Jun 15, 2024
  4

• torch.compile not working with device_map and multiple GPUs

  big model inferencewip
  huggingface/accelerate#2592 · Filter by author sheepymeh opened on Mar 27, 2024
  2

• [feature request] making the use of the accelerate launcher optional

  enhancementfeature request
  huggingface/accelerate#2585 · Filter by author stas00 opened on Mar 25, 2024
  6

• Why does accelerate's MegatronLMPlugin use its own megatron?

  wip
  huggingface/accelerate#2390 · Filter by author zhangsheng377 opened on Jan 29, 2024
  19

• [feature-request] Add OpenVINO as an inference-only backend

  feature request
  huggingface/accelerate#2306 · Filter by author Vipitis opened on Jan 5, 2024
  2

• Accelerate integration with Transformer Engine crashes when using FlashAttention

  enhancementfeature request
  huggingface/accelerate#2154 · Filter by author rationalism opened on Nov 14, 2023
  4

• WebLoader Support

  enhancementfeature request
  huggingface/accelerate#2083 · Filter by author karamavusibrahim opened on Oct 25, 2023
  9

• [FSDP] support activation offloading with FSDP

  enhancementfeature request
  huggingface/accelerate#2038 · Filter by author shijie-wu opened on Oct 6, 2023
  5