The problem with the conversion with the new convert.py

Hello! Help me figure out:

F:\Models\digitous-Alpacino13b>convert.py --dump-single F:\Models\digitous-Alpacino13b\4bit.safetensors
Traceback (most recent call last):
File "F:\Models\digitous-Alpacino13b\convert.py", line 1145, in
main()
File "F:\Models\digitous-Alpacino13b\convert.py", line 1116, in main
model_plus = lazy_load_file(args.model)
File "F:\Models\digitous-Alpacino13b\convert.py", line 853, in lazy_load_file
return lazy_load_safetensors_file(fp, path)
File "F:\Models\digitous-Alpacino13b\convert.py", line 753, in lazy_load_safetensors_file
model = {name: convert(info) for (name, info) in header.items()}
File "F:\Models\digitous-Alpacino13b\convert.py", line 753, in
model = {name: convert(info) for (name, info) in header.items()}
File "F:\Models\digitous-Alpacino13b\convert.py", line 745, in convert
assert 0 <= begin <= end <= len(byte_buf)
AssertionError

What is the error here - in the script or maybe there is a problem in the model? The model is from here: https://huggingface.co/digitous/Alpacino13b/tree/main

Same error here going on with me, i'm using a different model however but get the exact same error produced on latest commit

Traceback (most recent call last):
  File "D:\llama\llama.cpp\convert.py", line 1146, in <module>
    main()
  File "D:\llama\llama.cpp\convert.py", line 1126, in main
    model_plus = load_some_model(args.model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 1053, in load_some_model
    models_plus.append(lazy_load_file(path))
                       ^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 854, in lazy_load_file
    return lazy_load_safetensors_file(fp, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 754, in lazy_load_safetensors_file
    model = {name: convert(info) for (name, info) in header.items()}
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 754, in <dictcomp>
    model = {name: convert(info) for (name, info) in header.items()}
                   ^^^^^^^^^^^^^
  File "D:\llama\llama.cpp\convert.py", line 746, in convert
    assert 0 <= begin <= end <= len(byte_buf)
AssertionError

Apparently I fixed it by using Linux? Use WSL or an alternative and run the script again. I'm not versed in python that much to figure out the error but it doesn't seem to fall into an assertionerror on linux with no alterations whatsoever.

Let's see...

convert.py: Fix loading safetensors and ggml format on Windows

just checked and the commit fixed it!

The same mistake. Maybe I'm doing something wrong?

C:\llama.cpp>convert.py 4bit.safetensors --outtype q4_1 --outfile 4ggml.bin
Loading model file 4bit.safetensors
Traceback (most recent call last):
File "C:\llama.cpp\convert.py", line 1145, in
main()
File "C:\llama.cpp\convert.py", line 1125, in main
model_plus = load_some_model(args.model)
File "C:\llama.cpp\convert.py", line 1052, in load_some_model
models_plus.append(lazy_load_file(path))
File "C:\llama.cpp\convert.py", line 853, in lazy_load_file
return lazy_load_safetensors_file(fp, path)
File "C:\llama.cpp\convert.py", line 753, in lazy_load_safetensors_file
model = {name: convert(info) for (name, info) in header.items()}
File "C:\llama.cpp\convert.py", line 753, in
model = {name: convert(info) for (name, info) in header.items()}
File "C:\llama.cpp\convert.py", line 745, in convert
assert 0 <= begin <= end <= len(byte_buf)
AssertionError

convert.py: Fix loading safetensors and ggml format on Windows (#991)

Seeing similar issues still. Latest repo trying to set up an apple m2 implementation using mixtral:

% python3 convert.py ./models/mixtral-instruct-8x7b/ \
         --outfile ./models/mixtral-instruct-8x7b/ggml-model-f16.gguf \
         --outtype f16
Loading model file models/mixtral-instruct-8x7b/model-00001-of-00019.safetensors
Traceback (most recent call last):
  File "/Users/user/Documents/Workspace/llama/convert.py", line 1279, in <module>
    main()
  File "/Users/user/Documents/Workspace/llama/convert.py", line 1207, in main
    model_plus = load_some_model(args.model)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Documents/Workspace/llama/convert.py", line 1140, in load_some_model
    models_plus.append(lazy_load_file(path))
                       ^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Documents/Workspace/llama/convert.py", line 797, in lazy_load_file
    raise ValueError(f"unknown format: {path}")
ValueError: unknown format: models/mixtral-instruct-8x7b/model-00001-of-00019.safetensors

Seeing similar issues still.

It's a very different issue. Open a new issue.

SrVill

Facing the same issue in Google Colab. Any Solution?

convert.py: Fix loading safetensors and ggml format on Windows (ggml-…

Fix RoPE cache on multi-GPU setup (ggml-org#966)

convert.py: Fix loading safetensors and ggml format on Windows (ggml-…

Navigation Menu

Repository navigation

The problem with the conversion with the new convert.py #966

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

The problem with the conversion with the new convert.py #966

Description

Activity

kickturn commented on Apr 14, 2023

kickturn commented on Apr 15, 2023

comex commented on Apr 15, 2023

kickturn commented on Apr 15, 2023

SrVill commented on Apr 15, 2023

NextGA-OSS commented on Dec 16, 2023

prusnak commented on Dec 16, 2023

shrijayan commented on Dec 19, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions