Repository navigation
termux build issues #389
Description
Activity
I resolved ninja installation by termux-chroot. Also added following to
project CMakeLists.txt:
set(CMAKE_C_COMPILER "/data/data/com.termux/files/usr/bin/clang")
set(CMAKE_CXX_COMPILER "/data/data/com.termux/files/usr/bin/clang++")
And passing opencl args:
CMAKE_ARGS="-DLLAMA_CLBLAST=on -DLLAMA_OPENBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python
Here's output eror:
../CMake-src/Utilities/cmlibarchive/libarchive/archive.h:101:10: fatal error: 'android_lf.h' file not found
#include "android_lf.h"Looks to be related to scikit-build/cmake-python-distributions#223
The longer term solution seems to be migrating the project to scikit-build-core (something I'm in the process of doing). However someone in the thread mentioned they were able to get their example to work by building cmake and ninja from source on the android.
Also tried following:
cmake .. -DLLAMA_CLBLAST=on -DLLAMA_OPENBLAS=on
make
make installHere's output:
CMake Error: Target DependInfo.cmake file not found
[100%] Built target run
Install the project...
-- Install configuration: "RelWithDebInfo"
-- Up-to-date: /usr/local/llama_cpp/libllama.soIs installing successful?
Python:
import llama_cpp
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'llama_cpp'while trying to make it work on termux i ended up writing a small llama.cpp wrapper myself, instructions to use which are here, at the time llama.cpp cache files could be read but not generated/edited in termux, but that problem is sorted now, so put
cache_is_supported_trust_me_bro=Trueindiscord/termux/settings.pyto use it.these highlighted lines are all you need to make a wrapper of your own
- Is it compatible with babyagi and other stuff that needs llama python wrapper?…On Wed, Jun 28, 2023, 10:37 PM Subhranshu Sharma ***@***.***> wrote: while trying to make it work on termux i ended up writing a small llama.cpp wrapper <https://github.com/SubhranshuSharma/The_Elite_Order_of_Pure_Thought/tree/cb063f5ac0d8c8a26bc75f4f0de18cc8cee062ef/discord/termux> myself, instructions to use which are here <https://github.com/SubhranshuSharma/The_Elite_Order_of_Pure_Thought/tree/cb063f5ac0d8c8a26bc75f4f0de18cc8cee062ef#linuxtermux>, at the time llama.cpp cache files could be read but not generated/edited in termux, but that problem is sorted now, so put cache_is_supported_trust_me_bro=True in discord/termux/settings.py to use it. these highlighted lines <https://github.com/SubhranshuSharma/The_Elite_Order_of_Pure_Thought/blob/cb063f5ac0d8c8a26bc75f4f0de18cc8cee062ef/discord/termux/the_elite_bot_termux.py#L40C1-L51C71> are all you need to make a wrapper of your own — Reply to this email directly, view it on GitHub <#389 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD5KKLEJQUGTJKV45ZHQCJTXNR6I5ANCNFSM6AAAAAAZKDJVOQ> . You are receiving this because you commented.Message ID: ***@***.***>
Is it compatible with babyagi and other stuff that needs llama python wrapper?
nope, my wrapper is tightly integrated with my usecase and isn't a separate installable package, was thinking of doing so, but would be too much of a headache
@SubhranshuSharma so for your wrapper you just built the libllama.so seperately correct, did you just build llama.cpp with the Makefile? Maybe one solution is to avoid building llama.cpp on install by setting an environment variable / path to a pre-built library.
@abetlen Yeah, I run
git clone https://github.com/ggerganov/llama.cpp ~/llama.cpp;cd ~/llama.cpp;makeI suggest, try building llama.cpp but don't crash if u can't, just check if the
mainfile exists at a pre defined/user inputted path, if not useif os.system('git clone https://github.com/ggerganov/llama.cpp ~/llama.cpp;cd ~/llama.cpp;make')!=0:print('windows is for pussies, install git and cmake');exit()to clone and make it, way more simple and easy to maintain.The longer term solution seems to be migrating the project to scikit-build-core
Any progress about it?
Any progress about it?
@SubhranshuSharma @Freed-Wu implementing in #499 but I just have some issues with Macos still
unrelated question: is there any way of storing cache files on disk for quick reboot in the api
implementing in #499 but I just have some issues with Macos still
i would still suggest treating this repo and llama.cpp as different things and not letting failure in one stop the other (for as long as its possible), so make the compilation a try except pass, if compile fails, force user to set a system variable pointing to llama.cpp, I would also suggest keeping the system variable at first priority, as in if the system variable is set, it will be given the first priority. That way the project will be more robust, letting people find workarounds to issues that originate in llama.cpp.
@SubhranshuSharma sorry for this very late reply but I finally merged in #499.
You can now set
CMAKE_ARGS="-DLLAMA_BUILD=OFF pip install llama-cpp-pythonto avoid building from source, then just setLLAMA_CPP_LIBto the path to the shared library to use a pre-built library.So can we close this issue now?
@Freed-Wu , @abetlen , @SubhranshuSharma
I also tried packaging a python binary with llama-cpp-python, but when I try to run the binary standalone it fails with below errorINFO:root:Loading Model: TheBloke/synthia-7b-v1.3.Q5_K_M.gguf, on: cpu INFO:root:This action can take a few minutes! INFO:root:synthia-7b-v1.3.Q5_K_M.gguf INFO:root:Using Llamacpp for GGUF/GGML quantized models ERROR:root:Exception occurred: Shared library with base name 'llama' not foundBut when I run simply by
python3 main.pyit works fine.For pyinstaller do I need to generate the shared library seperatly and then use
LLAMA_CPP_LIBBut I am using termex, plain terminal.
You can now set
CMAKE_ARGS="-DLLAMA_BUILD=OFF pip install llama-cpp-pythonto avoid building from source, then just setLLAMA_CPP_LIBto the path to the shared library to use a pre-built library.this is the error i still get in termux when running
CMAKE_ARGS="-DLLAMA_BUILD=OFF" pip install llama-cpp-pythonam i missing something
@SubhranshuSharma if you want to build cmake module termux/termux-packages#10065
or build without it
-
CMAKE_ARGS="-DLLAMA_BUILD=OFF" pip install llama-cpp-python --no-build-isolation -v
-
check error message and install missing modules
repeat 1-2 steps
-
is the python library working for anyone?
@Freed-Wu is this related to adding original llama.cpp to termux package manager? if yes, llama.cpp was working out of the box on termux anyway, that's why i could make my own usecase-specific python wrapper around it, to quote myself:
@abetlen Yeah, I run
git clone https://github.com/ggerganov/llama.cpp ~/llama.cpp;cd ~/llama.cpp;make@romanovj this solution of yours did install cmake without errors, now
pip listis returningcmakeand a simpleimport cmakeis working, butpip install llama-cpp-pythonstill gives error sayingNo module named 'cmake', so maybe not all the references to installed libraries were updated.and your second solution worked, and
pip listis returningllama_cpp_pythonbutimport llama_cppis returningshared library with base name 'llama' not found@SubhranshuSharma
still gives [error](https://pastebin.com/FkXtBYcm) saying No module named 'cmakereinstall cmake
pkg rei cmakealso you can copy compiled cmake wheel to ~/wheels folder and install modules with
pip install ... --find-links ~/wheels
You need to build shared libs for llama.cpp like this
cmake -S . build64 -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=$PREFIX -DLLAMA_CLBLAST=ON -DLLAMA_OPENBLAS=ON
cmake --build build64then export path to libllama.so
export LLAMA_CPP_LIB=/data/data/com.termux/files/home/llama.cpp/build64/libllama.sopython import llama_cppok
Reacted by शुभ्रांशु शर्माHey guys, I was finally able to put some time into this, the following worked for me:
pkg install python-pip python-cryptography cmake ninja autoconf automake libandroid-execinfo patchelf MATHLIB=m CFLAGS=--target=aarch64-unknown-linux-android33 LDFLAGS=-lpython3.11 pip install numpy --force-reinstall --no-cache-dir pip install llama-cpp-python --verbose
Reacted by syddharth, DeadBranch and शुभ्रांशु शर्माReacted by DeadBranch@abetlen it works inconsistently, on a clean termux install with python installed i usually also haveto install
libexpatthenopensslthen runpkg update && pkg upgradethis works more consistently for me, keep selecting default answers to prompts
pkg install libexpat openssl python-pip python-cryptography cmake ninja autoconf automake libandroid-execinfo patchelf pkg update && pkg upgrade pip install llama-cpp-pythonthen run
python -c 'import llama_cpp'to check if it installedReacted by Andrei and sergey21000Does the llama-cpp-server work in termux?
@pinyaras original llama.cpp has its own server which is stable
the python server additionally requires rust which is unstable on termux but is working for now, after installing previous requirements runpkg install rust pip install llama-cpp-python[server]
pip install llama-cpp-python doesn't work on termux and gives this error, even if ninja is installed it gives same error, might be a hardcoded absolute path problem
i have tried to use portable venv setup on my linux machine but running it on termux gave a dependency not found error, so maybe some paths in the source code were still absolute(even after the correction attempts in the blog post)
tried using pyinstaller, but it doesn't support this library yet, same missing dependency issue
another option is to use docker on termux but that requires root privileges and custom kernel
i hv tried to look into the source code of this repo but donno where to start, any hint on where to start?
the original llama.cpp library works fine on termux but doesn't have a server inbuilt, and doesn't work well unless using bash
should i make a pull request editing the readme, linking to the docker workaround on rooted phones