SOTA 2-bit quants - part 2 - #4856

Merged

ggerganov merged 11 commits into

Jan 11, 2024

Commits on Jan 9, 2024

iq2_xs: basics
Iwan Kawrakow
committed
3569fa3
iq2_xs: this should have been in the basics
Iwan Kawrakow
committed
9f21b82
iq2_xs: CUDA and scalar CPU works
Iwan Kawrakow
committed
9b6e38d
iq2_xs: WIP Metal
Iwan Kawrakow
committed
0aacd55
iq2_xs: Metal now works
Iwan Kawrakow
committed
55e2cae
iq2_xs: working, but dog slow, ARM_NEON dot product
Iwan Kawrakow
committed
ff49d87
iq2_xs: better ARM_NEON dot product
Iwan Kawrakow
committed
52ea3f7

Commits on Jan 10, 2024

iq2_xs: AVX2 dot product - 19.5 t/s
Iwan Kawrakow
committed
3198e94
iq2_xs: faster AVX2 dit product
Iwan Kawrakow
committed
8299b03
iq2_xs: had forgotten to delete iq2-data.h
Iwan Kawrakow
committed
a1610b0

Commits on Jan 11, 2024

Add llama enum for IQ2_XS
Iwan Kawrakow
committed
9bfcb16