| 25 Jul 2024 |
 Pascal | This is on non-NixOS (Ubuntu 22.04). | 06:00:40 |
| Pascal | * @hexa (UTC+1) 👆
| 06:02:11 |
 ˈt͡sɛːzaɐ̯ | I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from an ~hour ago.) | 06:26:48 |
| ˈt͡sɛːzaɐ̯ | * I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from half an ~hour ago.) | 06:27:09 |
| ˈt͡sɛːzaɐ̯ | * Pascal: I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from half an ~hour ago.) | 06:27:22 |
| Pascal | In reply to @julius:mtx.liftm.de
Pascal: I can't reproduce that on a nixos, github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama-cuda builds fine. (should master from half an ~hour ago.) Thanks! I'll give that one a go later. | 09:17:32 |
| Pascal | @julius:mtx.liftm.de: Just checked: Works on NixOS, but still fails on Ubuntu (using nix run --impure). Same error: crt/host_config.h not found. Dang... it worked until 3 weeks or so ago.
It fails during 'Compiling the CUDA compiler identification source file' oddly using /usr/local/cuda/bin/nvcc despite finding /nix/store/...-cuda_nvcc... 🤷♂️ | 15:07:36 |
 SomeoneSerge (utc+3) | In reply to @phirsch:matrix.org
@julius:mtx.liftm.de: Just checked: Works on NixOS, but still fails on Ubuntu (using nix run --impure). Same error: crt/host_config.h not found. Dang... it worked until 3 weeks or so ago.
It fails during 'Compiling the CUDA compiler identification source file' oddly using /usr/local/cuda/bin/nvcc despite finding /nix/store/...-cuda_nvcc... 🤷♂️ Try entering a shell with cudaPackages.cuda_nvcc I guess | 15:13:03 |
| Pascal | Since I don't have any CUDA env vars set nor nvcc on the PATH, I suspect that the nvcc location is explicitly specified in the derivation's build script... | 15:13:14 |
| SomeoneSerge (utc+3) | I didn't check but I'd guess ollama doesn't propagate nvcc (not necessarily bad) and then it ends up picking up the bad nvcc from ubuntu (/usr/...) | 15:14:17 |
| Pascal | Now this is weird: Simply running
nix run github:NixOS/nixpkgs/289eafaaf02177e5814d8738cf57e259f9eae46e#ollama serve
(without -cuda!) works, showing 'library=cuda' and accurate GPU info... Can't claim to understand why, but it looks like I don't actually need the '-cuda' variant. (Suspect that it's somehow managing to dload() the system CUDA SOs?) | 15:31:57 |
| Pascal | @ss:someonex.net: Thanks for the tips (only saw your replies after refreshing just now). Might try that as well (although I might already be good according to the above). | 15:41:55 |
| 26 Jul 2024 |
|  Jason Schnitzer joined the room. | 01:35:00 |
|  nathan72419 joined the room. | 16:11:45 |
 connor (he/him) (UTC-7) | As a heads up, I’m moving this week so my availability will probably be more limited than it was already for a few weeks :/ | 16:27:09 |
| 27 Jul 2024 |
| Pascal | @SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
| 07:04:48 |
| 28 Jul 2024 |
|  matthewcroughan changed their display name from matthewcroughan to matthewcroughan - going to nix.camp. | 16:12:17 |
| 29 Jul 2024 |
|  kaya changed their profile picture. | 07:47:57 |
|  @ironbound:hackerspace.pl removed their display name P_Big. | 14:47:13 |
| @ironbound:hackerspace.pl removed their profile picture. | 14:47:26 |
| 30 Jul 2024 |
 srhb | I'm waving around a big hammer here. Does anyone want to save cuda-modules/aliases.nix? 😁
https://github.com/NixOS/nixpkgs/pull/331017 | 05:57:45 |
| SomeoneSerge (utc+3) | In reply to @srhb:matrix.org
I'm waving around a big hammer here. Does anyone want to save cuda-modules/aliases.nix? 😁
https://github.com/NixOS/nixpkgs/pull/331017 Yes we want to save aliases. I think the solution should be to ensure these attributes are kept lazy, not to remove them | 07:06:42 |
| srhb | Sad. Alright, I'll draft them :) | 07:06:57 |
| srhb | The aliases I nuked are still OK to go by now, right? | 07:07:11 |
| SomeoneSerge (utc+3) | I need to fetch my laptop:) | 07:07:36 |
| srhb | And laziness won't save that torch check, right? (equality has no choice but to strict) | 07:07:54 |
| srhb | Though I suppose I could exempt those exact attributes in the torch check. Lots of spooky action at a distance though. | 07:09:22 |
| SomeoneSerge (utc+3) | Ooh, that, the package set comparison. I forgot it was there | 07:10:30 |
| srhb | I understand why it's there, but I think it should go. | 07:11:48 |
| SomeoneSerge (utc+3) | Yea the check is quite a bit of a heuristic actually | 07:12:14 |
