Issues

• 📋 Documentation Enhancement Suggestion

  turboderp/exllama#319 · Filter by author croviatrust opened on Mar 16, 2026
  1

• Beating the (probably dead) Metal horse

  turboderp/exllama#318 · Filter by author bleedingedgedebian opened on Apr 4, 2025
  1

• Run on CPU without AVX2

  turboderp/exllama#315 · Filter by author ZanMax opened on Apr 14, 2024
  3

• piece id is out of range

  turboderp/exllama#314 · Filter by author chethanwiz opened on Apr 9, 2024
  3

• When will the bfloat16 type of GPTQ algorithm be supported?

  turboderp/exllama#310 · Filter by author Kelang-Tian opened on Dec 20, 2023

• Does it support safetytensor formate?>

  turboderp/exllama#309 · Filter by author lucasjinreal opened on Nov 28, 2023

• Error when using Beam Search

  turboderp/exllama#308 · Filter by author bibekyess opened on Nov 17, 2023

• Occasionally RuntimeError

  turboderp/exllama#307 · Filter by author leegohi04517 opened on Nov 16, 2023

• Using Exllama backend requires all the modules to be on GPU - how?

  turboderp/exllama#306 · Filter by author tigerinus opened on Nov 6, 2023
  1

• Changing hyper-parameters after initilization without reloading weights from disk.

  turboderp/exllama#299 · Filter by author kmccleary3301 opened on Sep 28, 2023

• doesn't use CUDA_HOME?

  turboderp/exllama#293 · Filter by author j2l opened on Sep 20, 2023

• OSError: CUDA_HOME environment variable is not set.

  turboderp/exllama#291 · Filter by author jamesbraza opened on Sep 17, 2023
  8

• CodeLLaMA + LoRA: RuntimeError: CUDA error: an illegal memory access was encountered

  turboderp/exllama#290 · Filter by author juanps90 opened on Sep 15, 2023
  3

• GPU Inference from IPython

  turboderp/exllama#289 · Filter by author Rajmehta123 opened on Sep 15, 2023

• followed instructions with error

  turboderp/exllama#288 · Filter by author hiqsociety opened on Sep 15, 2023
  2

• Multi-GPU issues

  turboderp/exllama#281 · Filter by author nktice opened on Sep 9, 2023
  9

• Support for Baichuan2 models

  turboderp/exllama#280 · Filter by author bernardx opened on Sep 9, 2023
  1

• Progress on the rewrite for older cards (Like the P40)

  turboderp/exllama#279 · Filter by author TimyIsCool opened on Sep 8, 2023
  1

• remove tokens that exceed the max_seq_len

  turboderp/exllama#274 · Filter by author p11188536 opened on Sep 5, 2023
  1

• Completion abruptly stopped - RuntimeError: CUDA error: an illegal memory access was encountered

  turboderp/exllama#273 · Filter by author Thireus opened on Sep 4, 2023
  1

• YaRN Support

  turboderp/exllama#272 · Filter by author grimulkan opened on Sep 2, 2023
  8

• Codelama support

  turboderp/exllama#270 · Filter by author ParisNeo opened on Sep 1, 2023
  11

• Support for AMD ROCM

  turboderp/exllama#268 · Filter by author yehowshuaradialrad opened on Aug 30, 2023
  1

• Is it possible and efficient if load layer on demand?

  turboderp/exllama#267 · Filter by author fahadh4ilyas opened on Aug 30, 2023
  2

• Speed on A100

  turboderp/exllama#266 · Filter by author Ber666 opened on Aug 30, 2023
  4