| 7 Dec 2024 |
 SomeoneSerge (back on matrix) | In reply to @connorbaker:matrix.org
Does anyone have a NixOS system they recommend using to test eval performance? Working on cppnix? | 01:40:03 |
 connor (he/him) | Kind of? More like preliminary work for a talk I plan to give at Nix Planet | 06:52:28 |
| SomeoneSerge (back on matrix) | Looking forward to watch the record:) | 12:28:11 |
| 8 Dec 2024 |
 kaya 𖤐 | Hi, im attempting to upstream my nix derivation for exllamav2 from https://github.com/BatteredBunny/nix-ai-stuff/blob/main/pkgs/exllamav2.nix
For some reason its complaining about CUDA_HOME being missing even though im specifying it which im kind of confused about, i thought maybe i would replace torch with torchWithCuda but then i get some mysterious error which i dont get in the flake
Anyone had issues with anything similar? Current attempt for anyone curious https://gist.github.com/BatteredBunny/2212ac469f07244d954bf556f128cb07 | 16:42:14 |
| kaya 𖤐 | Pretty sure the biggest difference between the flake and my upstreaming attempt is that in the flake i have allowUnfree and allowUnfree as true, but those options should carry over (i assume) as i also have those enabled on my nixos and im building with --impure | 16:44:00 |
| kaya 𖤐 | * Pretty sure the biggest difference between the flake and my upstreaming attempt is that in the flake i have allowUnfree and cudaSupport as true, but those options should carry over (i assume) as i also have those enabled on my nixos and im building with --impure | 16:44:11 |
| kaya 𖤐 | * Pretty sure the biggest difference between the flake and my upstreaming attempt is that in the flake i have allowUnfree and cudaSupport as true, but those options should carry over (i assume) as i also have those enabled on my nixos config and im building with --impure | 16:50:40 |
| 9 Dec 2024 |
| SomeoneSerge (back on matrix) |
For some reason its complaining about CUDA_HOME being missing even though im specifying it which im kind of confused
It might just want some more components than just nvcc and cudart. Also off the top of my head not sure which outputs are propagated into the symlinkJoin
| 00:11:43 |
| SomeoneSerge (back on matrix) | Could you publish the full logs? | 00:11:56 |
 hexa | is there a relationship between cuda and the open nvidia kmod? | 03:40:17 |
| hexa | because my cuda things stopped working some time after migrating to 24.11 | 03:40:27 |
| hexa | though nvidia-smi is working | 03:40:47 |
| hexa | but ollama and wyoming-faster-whisper can't init cuda | 03:41:08 |
| hexa | will try to drop hardening next, as usual 😄 | 03:42:19 |
| hexa | ok, DevicePolicy related 🙂 | 03:46:59 |
| hexa | ok, apparently not | 15:44:08 |
