Repository navigation
[Windows / Pycharm] llama_cpp_python installs and runs successfully without BLAS backend, but does not work with cuBLAS flags #228
Description
Prerequisites
Please answer the following questions for yourself before submitting an issue.
- I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new bug or useful enhancement to share.
Expected Behavior
I was able to run pip install llama-cpp-python successfully in PyCharm Terminal and use it in my Python code, so when I saw that additional flags needed to be set for cuBLAS, I expected the following code to run successfully:
$env:CMAKE_ARGS='-DLLAMA_CUBLAS=on'
$env:FORCE_CMAKE=1
$env:LLAMA_CUBLAS=1
pip install llama-cpp-python
Current Behavior
I'm still getting the BLAS = 0 indicator, based on the printout of the terminal:
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | VSX = 0 |
Environment and Context
I'm running PyCharm 2022.2 with Python 3.9, and I have llama-cpp-python 0.1.50 installed. I have an NVIDIA GeForce RTX 3070 and have been able to get cuBLAS up and running with the regular llama.cpp.
Activity
[-][Windows / Pycharm] llama_cpp_python installs and runs successfully, but does not work with cuBLAS flags[/-][+][Windows / Pycharm] llama_cpp_python installs and runs successfully without BLAS backend, but does not work with cuBLAS flags[/+]gjmulder commentedon May 18, 2023
Can we see the output of your package build with the
--verboseflag added?technicolor-twelve commentedon May 18, 2023
@gjmulder This is the output when I run
pip install llama-cpp-pythonwith the previous flags enabled. Is this what you're looking for?llama-cpp-python pip install output.txt
abetlen commentedon May 18, 2023
@technicolor-twelve try
pip install llama-cpp-python --force-reinstall --upgrade --no-cache-diras it's using a cached version of the package.technicolor-twelve commentedon May 18, 2023
That forced a new install, though now when I run
from llama_cpp import LlamaI get the following runtime error:llama runtime error.txt
Which is weird since I could navigate to the referenced and see the .dll file there. I tried adding an
os.add_dll_directoryto specifically include the path but the error still remains.gjmulder commentedon May 23, 2023
That looks like a path issue. Can you review #225, please?
technicolor-twelve commentedon May 24, 2023