Repository navigation
About reproducing the results in the blog #166
Description
Thank you very much for your great work.
May I ask how to reproduce the results in the blog video of your README and which GPU you are using. The main concern is that I use the following command and cannot get the results.
dflash benchmark transformers \
--model Qwen/Qwen3-8B --draft z-lab/Qwen3-8B-DFlash-b16 \
--dataset gsm8k --max-samples 200 \
--reasoning off --max-new-tokens 512
Thr result is:
Baseline throughput: 45.48 tok/s
DFlash throughput: 215.20 tok/s
Decoding speedup: 4.73
Average Acceptance length: 6.39
Acceptance length histogram: ['0.0%', '11.2%', '16.4%', '12.0%', '8.7%', '6.9%',
'6.2%', '5.5%', '4.3%', '4.8%', '2.5%', '3.7%', '2.3%', '2.5%', '2.5%', '2.0%', '8.4%']
But the DFlash throught in the video is 400+ tok/s
Reacted by Yeonjoon Jung and Haoran Wan
Activity