| 23 Jul 2024 |
|  @trofi:matrix.org left the room. | 19:05:15 |
| 24 Jul 2024 |
 Pascal | Does anyone else have trouble building ollama-cuda from nixos-unstable lately? | 05:14:53 |
 hexa | Pascal: anything concrete that you are seeing? | 09:40:09 |
|  kaya 𖤐 changed their profile picture. | 21:52:00 |
| 25 Jul 2024 |
| Pascal | Compilation of nixpkgs#ollama-cuda fails with: '/nix/store/4fxpzb7rqf19r5hypxpzbf72jaa6mphw-cuda_cudart-12.2.140-dev/include/cuda_runtime.h:82:10: fatal error: crt/host_config.h: No such file or directory' | 05:54:18 |
| Pascal | @hexa: 👆 | 05:55:35 |
| Pascal | This is on non-NixOS (Ubuntu 22.04). | 06:00:40 |
| Pascal | * @hexa (UTC+1) 👆
| 06:02:11 |
 ˈt͡sɛːzaɐ̯ | I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from an ~hour ago.) | 06:26:48 |
| ˈt͡sɛːzaɐ̯ | * I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from half an ~hour ago.) | 06:27:09 |
| ˈt͡sɛːzaɐ̯ | * Pascal: I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from half an ~hour ago.) | 06:27:22 |
| Pascal | In reply to @julius:mtx.liftm.de
Pascal: I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from half an ~hour ago.) Thanks! I'll give that one a go later. | 09:17:32 |
| Pascal | @julius:mtx.liftm.de: Just checked: Works on NixOS, but still fails on Ubuntu (using nix run --impure). Same error: crt/host_config.h not found. Dang... it worked until 3 weeks or so ago.
It fails during 'Compiling the CUDA compiler identification source file' oddly using /usr/local/cuda/bin/nvcc despite finding /nix/store/...-cuda_nvcc... 🤷♂️ | 15:07:36 |
 SomeoneSerge (back on matrix) | In reply to @phirsch:matrix.org
@julius:mtx.liftm.de: Just checked: Works on NixOS, but still fails on Ubuntu (using nix run --impure). Same error: crt/host_config.h not found. Dang... it worked until 3 weeks or so ago.
It fails during 'Compiling the CUDA compiler identification source file' oddly using /usr/local/cuda/bin/nvcc despite finding /nix/store/...-cuda_nvcc... 🤷♂️ Try entering a shell with cudaPackages.cuda_nvcc I guess | 15:13:03 |
| Pascal | Since I don't have any CUDA env vars set nor nvcc on the PATH, I suspect that the nvcc location is explicitly specified in the derivation's build script... | 15:13:14 |
| SomeoneSerge (back on matrix) | I didn't check but I'd guess ollama doesn't propagate nvcc (not necessarily bad) and then it ends up picking up the bad nvcc from ubuntu (/usr/...) | 15:14:17 |
| Pascal | Now this is weird: Simply running
nix run github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama serve
(without -cuda!) works, showing 'library=cuda' and accurate GPU info... Can't claim to understand why, but it looks like I don't actually need the '-cuda' variant. (Suspect that it's somehow managing to dload() the system CUDA SOs?) | 15:31:57 |
| Pascal | @ss:someonex.net: Thanks for the tips (only saw your replies after refreshing just now). Might try that as well (although I might already be good according to the above). | 15:41:55 |
| 26 Jul 2024 |
|  Jason Schnitzer joined the room. | 01:35:00 |
|  nathan72419 joined the room. | 16:11:45 |
