Repository navigation

k-quants with super-block size of 64 - #2001

Merged

ggerganov merged 35 commits into

masterfrom

ik/k_quants_64

Jun 26, 2023

Merged

k-quants with super-block size of 64#2001
ggerganov merged 35 commits into
masterfrom
ik/k_quants_64

Commits on Jun 26, 2023
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  d2f12ac
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  9fe2a2b
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  1f6195c
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  aebd547
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  2b2ab31
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  bcf8c5c
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  c6c3536
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  5aae4b8
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  41e46ec
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  460dd84
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  3bd9ae7
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  03f30c8
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  cda47a6
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  80c75fe
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  2b2a13c
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  9d27d8d
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  2ff543c
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  d92c5a9
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  fae24af
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  e1bbcfc
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  167a0bb
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  6081a65
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  ff83e32
- k_quants: WIP super-blocks with 64 weights
  Iwan Kawrakow
  committed
  285eeb1
- k_quants: call them _K, not _k, also on Metal
  Iwan Kawrakow
  committed
  8b98d01
- k_quants: correctly define QK_K in llama.cpp
  Iwan Kawrakow
  committed
  558a194
- Fixed bug in q4_K quantization added with the 64-block addition
  Iwan Kawrakow
  committed
  333ffcc
- Simplify via lambda
  Iwan Kawrakow
  committed
  88412a1
- k_quants: swicth Q3_K to 4-bit scales when QK_K = 64
  Iwan Kawrakow
  committed
  aeefd4e
- k_quants: switch Q4_K to 4-bit scales when QK_K = 64
  Iwan Kawrakow
  committed
  ce19b96
- k_quants: forgot to add the Metal changes in last commit
  Iwan Kawrakow
  committed
  4f61506
- k_quants: change Q5_K to be type 0 when QK_K = 64
  Iwan Kawrakow
  committed
  ccf4901
- k_quants: AVX2 implementation for new 64-weight Q5_K
  Iwan Kawrakow
  committed
  2da3a59
- k_quants: 10% faster ARM_NEON Q5_K dot product
  Iwan Kawrakow
  committed
  53e81ca
- k_quants: fixed issue caused by merging with master
  Iwan Kawrakow
  committed
  5fd8337

Commits

Commits on Jun 26, 2023