You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
LLAMA 2 70B convert fails with: failed to find n_mult for (n_ff=28672, n_embd=8192) #2286
python C:\src\llama.cpp\convert.py --outfile ggml-model-f16.bin --outtype f16 .
Loading model file model-00001-of-00015.safetensors
Loading model file model-00001-of-00015.safetensors
Loading model file model-00002-of-00015.safetensors
Loading model file model-00003-of-00015.safetensors
Loading model file model-00004-of-00015.safetensors
Loading model file model-00005-of-00015.safetensors
Loading model file model-00006-of-00015.safetensors
Loading model file model-00007-of-00015.safetensors
Loading model file model-00008-of-00015.safetensors
Loading model file model-00009-of-00015.safetensors
Loading model file model-00010-of-00015.safetensors
Loading model file model-00011-of-00015.safetensors
Loading model file model-00012-of-00015.safetensors
Loading model file model-00013-of-00015.safetensors
Loading model file model-00014-of-00015.safetensors
Loading model file model-00015-of-00015.safetensors
Loading vocab file tokenizer.model
Traceback (most recent call last):
File "C:\src\llama.cpp\convert.py", line 1264, in
main()
File "C:\src\llama.cpp\convert.py", line 1253, in main
params = Params.load(model_plus)
File "C:\src\llama.cpp\convert.py", line 203, in load
params = Params.loadHFTransformerJson(model_plus.model, hf_transformer_config_path)
File "C:\src\llama.cpp\convert.py", line 187, in loadHFTransformerJson
n_mult = find_n_mult(n_ff, n_embd);
File "C:\src\llama.cpp\convert.py", line 140, in find_n_mult
raise Exception(f"failed to find n_mult for (n_ff={n_ff}, n_embd={n_embd}).")
Exception: failed to find n_mult for (n_ff=28672, n_embd=8192).
Activity
Green-Sky commentedon Jul 20, 2023
see #2276
klosax commentedon Jul 20, 2023
Workaround: #2276 (comment)
Green-Sky commentedon Jul 20, 2023
can you try
4096?(taken from https://huggingface.co/meta-llama/Llama-2-70b/blob/main/params.json)
klosax commentedon Jul 20, 2023
It does not matter as the
n_ffparameter gets hardcoded by the PR:https://github.com/ggerganov/llama.cpp/blob/2d2bb6b0af7df316c297ec0fbc9c8af3566c23f1/llama.cpp#L1029-L1030
FNsi commentedon Jul 20, 2023
I'd think u just need to delete that exception about line 140 in convert.py
Green-Sky commentedon Jul 20, 2023
ah well, it does not matter, since both are workarounds.
gguf will have an n_ff hyperparameter.
nalzok commentedon Jul 27, 2023
Looks like it should be
n_mult = 4096instead ofn_mult = 256? #2276 (comment)Green-Sky commentedon Jul 27, 2023
the continuation of this issue is tracked here: #2376
after #2276 solved most issues