Skip to content

LLAMA 2 70B convert fails with: failed to find n_mult for (n_ff=28672, n_embd=8192) #2286

Closed

Description

@Melbar666

python C:\src\llama.cpp\convert.py --outfile ggml-model-f16.bin --outtype f16 .
Loading model file model-00001-of-00015.safetensors
Loading model file model-00001-of-00015.safetensors
Loading model file model-00002-of-00015.safetensors
Loading model file model-00003-of-00015.safetensors
Loading model file model-00004-of-00015.safetensors
Loading model file model-00005-of-00015.safetensors
Loading model file model-00006-of-00015.safetensors
Loading model file model-00007-of-00015.safetensors
Loading model file model-00008-of-00015.safetensors
Loading model file model-00009-of-00015.safetensors
Loading model file model-00010-of-00015.safetensors
Loading model file model-00011-of-00015.safetensors
Loading model file model-00012-of-00015.safetensors
Loading model file model-00013-of-00015.safetensors
Loading model file model-00014-of-00015.safetensors
Loading model file model-00015-of-00015.safetensors
Loading vocab file tokenizer.model
Traceback (most recent call last):
File "C:\src\llama.cpp\convert.py", line 1264, in
main()
File "C:\src\llama.cpp\convert.py", line 1253, in main
params = Params.load(model_plus)
File "C:\src\llama.cpp\convert.py", line 203, in load
params = Params.loadHFTransformerJson(model_plus.model, hf_transformer_config_path)
File "C:\src\llama.cpp\convert.py", line 187, in loadHFTransformerJson
n_mult = find_n_mult(n_ff, n_embd);
File "C:\src\llama.cpp\convert.py", line 140, in find_n_mult
raise Exception(f"failed to find n_mult for (n_ff={n_ff}, n_embd={n_embd}).")
Exception: failed to find n_mult for (n_ff=28672, n_embd=8192).

Activity

  1. Green-Sky commented on Jul 20, 2023

    @Green-Sky
    Collaborator

    see #2276

  2. klosax commented on Jul 20, 2023

    @klosax
    Contributor

    Workaround: #2276 (comment)

  3. Green-Sky commented on Jul 20, 2023

    @Green-Sky
    Collaborator
  4. klosax commented on Jul 20, 2023

    @klosax
    Contributor

    can you try 4096 ?

    It does not matter as the n_ff parameter gets hardcoded by the PR:
    https://github.com/ggerganov/llama.cpp/blob/2d2bb6b0af7df316c297ec0fbc9c8af3566c23f1/llama.cpp#L1029-L1030

  5. FNsi commented on Jul 20, 2023

    @FNsi
    Contributor

    I'd think u just need to delete that exception about line 140 in convert.py

  6. Green-Sky commented on Jul 20, 2023

    @Green-Sky
    Collaborator

    It does not matter as the n_ff parameter gets hardcoded by the PR:

    ah well, it does not matter, since both are workarounds.

    gguf will have an n_ff hyperparameter.

  7. nalzok commented on Jul 27, 2023

    @nalzok

    Workaround: #2276 (comment)

    Looks like it should be n_mult = 4096 instead of n_mult = 256? #2276 (comment)

  8. Green-Sky commented on Jul 27, 2023

    @Green-Sky
    Collaborator

    the continuation of this issue is tracked here: #2376

    after #2276 solved most issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions