mindfork 0.14.1 — a rented GPU box that starts, or says why not

mindfork 0.14.1 is out. It is a release of fixes, and every one of them was found on a rented GPU — the machine the one-line install was written for.

What a B200 said, and what it meant

The line ran on a RunPod pod with a B200, the cuda-13 build of llama.cpp and a Qwen model at a 131k context. The build installed and listed the card; then both servers died while warming up, and the app's status read corrupt GGUF or out of memory?. Neither was the cause. The server's own log had it: the provided PTX was compiled with an unsupported toolchain.

llama.cpp ships ready-made GPU code for a few cards only. For the others — a B200, an H100, an A100 — it ships an intermediate form that the NVIDIA driver compiles when the server starts, and a driver can compile it only from a CUDA no newer than itself. The build was CUDA 13.4, the pod's driver 13.2. The way out is the cuda-12 build, or a newer driver.

The app now says so:

• A server that dies shows the error it logged, not a guess. For a build newer than the driver it names the way out: the cuda-12 build, or a newer driver.
• A port another program holds is refused. RunPod's own web server sits on 8001, the embedding server's default port. Its answer passed for a ready server, so the embedder was relaunched 17 times in ten seconds. The app now does not start a server on a taken port; it says the port is taken and which setting moves it.
• A crash gets its three restarts. Each crash of the chat server used to cost two launches 40 ms apart, the second cutting the first short; now one crash is one restart. A server that could not be started at all is left alone until its settings change.
• A long status is cut, not hidden, in the settings' Model/server section. That is the one place the embedding server's reason is shown, and a long one used to be left out.

A minute back on every start

The cuda-12 build works on a B200, but it pays for that compile on every start. Measured on a fresh pod:

About 60 seconds of each start is the compile. The driver keeps what it compiled in a cache, but that cache is on the pod's container disk, which a stop clears — so every start after a stop paid again. The app now keeps the cache with its own data, in data/cuda-cache/ (134 MB on that pod), which on RunPod is on the volume. The compile is paid once per card, with nothing to set. A CUDA_CACHE_PATH of your own still wins.

A bare Linux image

llama.cpp's Linux builds need OpenMP's runtime, libgomp1. RunPod's images carry it; a minimal one such as ubuntu:24.04 does not, and there the release went wrong three times over. mindfork llama setup installed a build that could not start, printed the loader's error as its version, and then said no GPU had been found. The servers blamed the model again.

• install.sh installs the library when the line also installs llama.cpp — through apt, dnf, pacman or zypper. With --no-deps it prints the command to run instead.
• llama setup refuses a build that does not start. It names the missing library and its package, and keeps the download, so running it again after the fix fetches nothing.
• A server that cannot load a library says which one.

Your data

Nothing stored changes, and no migration runs. The one new thing on disk is data/cuda-cache/. It is the driver's derived data and is not in a backup. Delete it at any time, and the next start compiles again. 0.14.0 never looks at it if you go back.

The full list is in the changelog; downloads are on the releases page, and cargo install mindfork builds the same version from crates.io.