Windows: ryzenai-server.exe (--device onnx) in build-windows.sh

docs/onnx.md

@@ -56,5 +56,5 @@ llama-server's extra routes (`/tokenize`, `/slots`, …) answer 501 on `--device
 
 ## Next
 
-- **Windows.** `ryzenai-server.exe` in `scripts/build-windows.sh`: ONNX Runtime GenAI's Windows builds include DirectML, the GPU on Windows.
+- **Windows.** Done for the CPU: `scripts/build-windows.sh` builds `ryzenai-server.exe` ([Windows](windows.md)). The GPU on Windows would need ONNX Runtime's DirectML builds, whose `DirectML.dll` is under Microsoft's redistributable licence rather than MIT.
 - **The Ryzen AI NPU.** On a machine with Ryzen AI Software installed.

docs/windows.md

@@ -17,17 +17,18 @@ limitations under the License.
 
 # Windows
 
-`1bit.exe` runs on Windows 10 and later (x64): `1bit serve` with the Vulkan backend, `1bit route` (Laya) and `1bit comfy`. It is cross-built on Linux. There is no NPU lane, HRX, ROCm or ZINC on Windows yet.
+`1bit.exe` runs on Windows 10 and later (x64): `1bit serve` with the Vulkan backend and with ONNX Runtime GenAI models (`--device onnx`, on the CPU), `1bit route` (Laya) and `1bit comfy`. It is cross-built on Linux. There is no NPU lane, HRX, ROCm or ZINC on Windows yet.
 
 ```sh
 scripts/build-windows.sh <out-dir>     # ~2 min on Strix Halo once the downloads are cached
 ```
 
-The script writes two single, static executables:
+The script writes:
 - **`<out-dir>/1bit.exe`** (3.2 MB). It imports only `KERNEL32`, `WS2_32` and the UCRT that ships with Windows 10.
 - **`<out-dir>/llama-server.exe`**, the Vulkan pin (`third_party/llama.cpp-vulkan`) built for Windows. It also imports `vulkan-1.dll`, from the GPU driver.
+- **`<out-dir>/ryzenai-server.exe`** (the ONNX backend, [ONNX Runtime](onnx.md)), with Microsoft's `onnxruntime-genai.dll`, `onnxruntime.dll` and `onnxruntime_providers_shared.dll` beside it.
 
-Keep the two side by side: `1bit.exe serve -m model.gguf` finds `llama-server.exe` next to itself. `--llama-server PATH` or `ONEBIT_LLAMA_SERVER` names another one.
+Keep them together. `1bit.exe serve -m model.gguf` finds `llama-server.exe` next to itself, and `1bit.exe serve -m <dir with genai_config.json>` finds `ryzenai-server.exe`. `--llama-server PATH`, `ONEBIT_LLAMA_SERVER` and `ONEBIT_ONNX_SERVER` name other ones.
 
 ## What it downloads
 
@@ -40,6 +41,7 @@ Every download is pinned and checked against its sha256:
 | Vulkan-Headers `vulkan-sdk-1.4.357.0` | llama.cpp's Vulkan backend |
 | the Vulkan loader's `vulkan-1.def`, same tag | the import library, made with `llvm-dlltool`: no Vulkan SDK or Windows machine needed |
 | SPIRV-Headers, same tag | llama.cpp's Vulkan backend |
+| ONNX Runtime GenAI 0.11.2 `win-x64` and ONNX Runtime 1.23.2 `win-x64` (Microsoft, MIT) | ryzenai-server.exe (CPU). The DirectML builds would add the GPU, but `DirectML.dll` is under Microsoft's redistributable licence, not MIT, so they are not fetched. |
 
 Host tools: `cmake`, `git`, `curl`, a native C/C++ compiler (llama.cpp's shader generator runs on the build machine) and `glslc`.
 
@@ -50,17 +52,22 @@ Host tools: `cmake`, `git`, `curl`, a native C/C++ compiler (llama.cpp's shader
 - **cpp-httplib.** It is built at the Windows 10 API level, with its non-blocking DNS lookup off, because MinGW's headers lack `GetAddrInfoExCancel` and the engine only ever connects to 127.0.0.1.
 - **`1bit comfy`.** It spawns `comfyui_cpp.exe` and waits for it, since there is no exec in place on Windows.
 - **llama.cpp workaround.** The pinned llama.cpp uses `std::function` in `ggml-vulkan-types.h` without including `<functional>`. libc++ does not include it transitively, so the script adds it from the command line.
+- **ryzenai-server's CMake** is used unmodified through a wrapper project:
+  - the wrapper clears its MSVC-only `/SUBSYSTEM:CONSOLE` link flag (MinGW links console programs by default);
+  - a one-line `Wbemidl.h` covers MinGW's lowercase `wbemidl.h`;
+  - the MinGW-built exe links Microsoft's MSVC-built `onnxruntime-genai.dll` through its import library.
 
 ## Tested
 
 On Strix Halo under Wine 11, whose Vulkan goes to the host's RADV driver, so on the real Radeon 8060S. Measured 2026-09-25:
 - `1bit.exe serve -m Qwen3-0.6B-Q4_K_M.gguf` found `llama-server.exe` beside it, answered "The capital of France is Paris." (132 tok/s) and "Three colors: green, red, and blue.", and streamed a reply as SSE.
 - `taskkill /F /IM 1bit.exe` left no `llama-server.exe` behind.
+- `1bit.exe serve -m <Qwen2.5-0.5B CPU ONNX dir>` (no `--device`) chose `onnx`, started `ryzenai-server.exe` beside it and answered "The capital of France is Paris." (`ryzenai-server.exe` alone: 61 tok/s under Wine). After `taskkill /F`, nothing was left running.
 - The Linux build of the same sources passes its tests.
 
 It has not run on a real Windows machine yet.
 
 ## Next
 
-- **The ONNX backend on Windows:** `--device onnx` works on Linux ([ONNX Runtime](onnx.md)); `ryzenai-server.exe` in this script is next.
+- **ONNX on the Windows GPU (DirectML)**, once the `DirectML.dll` licence question is settled. The Ryzen AI NPU through Ryzen AI Software (its own licence, installed by the user).
 - **The NPU lane on Windows XRT.**

scripts/build-windows.sh

@@ -16,9 +16,10 @@
 #
 # build-windows.sh <out-dir>
 #
-# Cross-builds the engine for Windows 10+ x64 on Linux (docs/windows.md): <out-dir>/1bit.exe and,
+# Cross-builds the engine for Windows 10+ x64 on Linux (docs/windows.md): <out-dir>/1bit.exe;
 # from the Vulkan pin (third_party/llama.cpp-vulkan), <out-dir>/llama-server.exe with the Vulkan
-# backend. Both are single static executables; 1bit.exe finds llama-server.exe beside it.
+# backend; and from third_party/ryzenai-server, <out-dir>/ryzenai-server.exe with Microsoft's ONNX
+# Runtime GenAI and ONNX Runtime DLLs (--device onnx, CPU). 1bit.exe finds its backends beside it.
 # Everything it downloads is pinned and checked against its sha256: llvm-mingw (clang 23: the
 # engine is C++26), PCRE2 (the tokenizer), Vulkan-Headers, the Vulkan loader's export list
 # (vulkan-1.def -> the import library, so no Windows machine or Vulkan SDK is needed) and
@@ -42,6 +43,10 @@ VK_DEF=https://raw.githubusercontent.com/KhronosGroup/Vulkan-Loader/$VK_TAG/load
 VK_DEF_SHA=9ba339b7f5ee2df28487698a6840ecf095fac58b415c2c89ad9f161e9d316378
 SPIRV_HEADERS=https://github.com/KhronosGroup/SPIRV-Headers/archive/refs/tags/$VK_TAG.tar.gz
 SPIRV_HEADERS_SHA=4d703067a7e06331ccb37bdfed3f9b7879cc61969a2689ae95c95db34a47ff07
+OGA_WIN=https://github.com/microsoft/onnxruntime-genai/releases/download/v0.11.2/onnxruntime-genai-0.11.2-win-x64.zip
+OGA_WIN_SHA=31aeeb4fa7e1d9bf284f6215d60e0025d534b99add7b0daeaccd729ee8ad1595
+ORT_WIN=https://github.com/microsoft/onnxruntime/releases/download/v1.23.2/onnxruntime-win-x64-1.23.2.zip
+ORT_WIN_SHA=0b38df9af21834e41e73d602d90db5cb06dbd1ca618948b8f1d66d607ac9f3cd
 
 command -v glslc >/dev/null || { echo "build-windows.sh: glslc (shaderc) is required"; exit 1; }
 
@@ -119,4 +124,30 @@ cmake -S "$root/third_party/llama.cpp-vulkan" -B "$work/llama" -DCMAKE_TOOLCHAIN
 cmake --build "$work/llama" --target llama-server -j"$jobs" > "$work/llama.build.log" 2>&1 || { tail -20 "$work/llama.build.log"; exit 1; }
 cp "$work/llama/bin/llama-server.exe" "$out/"
 
-ls -la "$out/1bit.exe" "$out/llama-server.exe"
+# 6. ryzenai-server.exe (--device onnx), from its pin, against Microsoft's Windows releases of ONNX
+# Runtime GenAI and ONNX Runtime (MIT; CPU). Upstream's CMake is used unmodified through a small
+# wrapper that clears its MSVC-only /SUBSYSTEM:CONSOLE link flag (MinGW links console programs by
+# default), and a one-line Wbemidl.h shim covers the case of MinGW's wbemidl.h.
+git -C "$root" submodule update --init third_party/ryzenai-server
+unzip_to() { [ -d "$2" ] || { mkdir -p "$2.tmp" && (cd "$2.tmp" && cmake -E tar xf "$1") && mv "$2.tmp" "$2"; }; }
+unzip_to "$(fetch $OGA_WIN $OGA_WIN_SHA oga-win.zip)" "$work/oga-win"
+unzip_to "$(fetch $ORT_WIN $ORT_WIN_SHA ort-win.zip)" "$work/ort-win"
+oga=$(ls -d "$work"/oga-win/*/)
+ort=$(ls -d "$work"/ort-win/*/)
+mkdir -p "$work/rz-wrap" "$work/rz-shim" "$work/oga-root"
+echo '#include <wbemidl.h>' > "$work/rz-shim/Wbemidl.h"
+cat > "$work/rz-wrap/CMakeLists.txt" <<'RZ'
+cmake_minimum_required(VERSION 3.20)
+project(ryzenai-server-mingw CXX)
+add_subdirectory(${RYZENAI_SERVER_SRC} rz)
+set_target_properties(ryzenai-server PROPERTIES LINK_FLAGS "")
+RZ
+cmake -S "$work/rz-wrap" -B "$work/rz" -DRYZENAI_SERVER_SRC="$root/third_party/ryzenai-server" \
+    -DCMAKE_TOOLCHAIN_FILE="$work/toolchain.cmake" -DCMAKE_BUILD_TYPE=Release -DOGA_ROOT="$work/oga-root" \
+    -DOGA_INCLUDE="${oga}include" -DOGA_LIB="${oga}lib/onnxruntime-genai.lib" -DCMAKE_EXE_LINKER_FLAGS=-static \
+    "-DCMAKE_CXX_FLAGS=-D_WIN32_WINNT=0x0A00 -I$work/rz-shim" > "$work/rz.cmake.log"
+cmake --build "$work/rz" -j"$jobs" > "$work/rz.build.log" 2>&1 || { tail -20 "$work/rz.build.log"; exit 1; }
+cp "$work/rz/bin/ryzenai-server.exe" "${oga}lib/onnxruntime-genai.dll" "${ort}lib/onnxruntime.dll" \
+   "${ort}lib/onnxruntime_providers_shared.dll" "$out/"
+
+ls -la "$out"/*.exe "$out"/*.dll