Will Linux be supported?
Activity
G2G2G2G commentedon Mar 12, 2023
.... just compile it you literally have to clone it and type "make" read the readme...
Lupul commentedon Mar 12, 2023
I compiled it and run it on Ubuntu 22.04 with AMD 5700G cpu and got ~300ms/token (7B), ~550ms/token (13B)
ggerganov commentedon Mar 12, 2023
Linux and Mac OS are full supported through the Makefile
JonWhiteRA commentedon Apr 25, 2023
Is there a preferred version of gcc for linux?
G2G2G2G commentedon Apr 25, 2023
bigger number = better
CRITICAL: turbo3 perplexity is 165.6 vs q8_0 6.1 — quality broken #30
CRITICAL: found TWO root causes for PPL=165 #30
fix: restore inverse rotation in dequant — PPL 6.19 (1.2% of q8_0) #31 …
docs: perplexity 6.194 confirmed — 1.4% of q8_0 #30
docs: complete quality benchmark summary + lessons learned #30
156 remaining items
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 15: campaign memory wins
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
flagos: SpacemiT provider: SpacemiT's Q4_0 scale fix (spacemit-com/ll…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…