Repository navigation
A typo in readme? #271
Description
Memory/Disk Requirements
As the models are currently fully loaded into memory, you will need adequate disk space to save them and sufficient RAM to load them. At the moment, memory and disk requirements are the same.
| model | original size | quantized size (4-bit) |
|---|---|---|
| 7B | 13 GB | 3.9 GB |
| 15B | 24 GB | 7.8 GB |
| 30B | 60 GB | 19.5 GB |
| 65B | 120 GB | 38.5 GB |
Isn't it 13B and not 15B?
Metadata
Metadata
Assignees
Labels
No labels
Activity
prusnak commentedon Mar 18, 2023
Yes, that's a typo. Fix in #272
Merge pull request ggml-org#271 from baptisterajaut/fix/checkpoint-re…
release: ship CPU backend variants in the CUDA and HIP bundles (ggml-…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…
rdna-boosts: block 05: flash-attention band, native quantized K/V and…