| 4 Dec 2024 |
 connor (he/him) | I probably will; I'm also planning to submit two talks | 04:38:33 |
| 6 Dec 2024 |
|  vannagamma joined the room. | 00:01:17 |
| connor (he/him) | Does anyone have a NixOS system they recommend using to test eval performance? | 05:49:07 |
| connor (he/him) | Ideally something which takes on the order of 30s or so to eval | 05:51:25 |
|  kaya 𖤐 changed their profile picture. | 21:17:32 |
| 7 Dec 2024 |
 SomeoneSerge (back on matrix) | In reply to @connorbaker:matrix.org
Does anyone have a NixOS system they recommend using to test eval performance? Working on cppnix? | 01:40:03 |
| connor (he/him) | Kind of? More like preliminary work for a talk I plan to give at Nix Planet | 06:52:28 |
| SomeoneSerge (back on matrix) | Looking forward to watch the record:) | 12:28:11 |
| 8 Dec 2024 |
| kaya 𖤐 | Hi, im attempting to upstream my nix derivation for exllamav2 from https://github.com/BatteredBunny/nix-ai-stuff/blob/main/pkgs/exllamav2.nix
For some reason its complaining about CUDA_HOME being missing even though im specifying it which im kind of confused about, i thought maybe i would replace torch with torchWithCuda but then i get some mysterious error which i dont get in the flake
Anyone had issues with anything similar? Current attempt for anyone curious https://gist.github.com/BatteredBunny/2212ac469f07244d954bf556f128cb07 | 16:42:14 |
| kaya 𖤐 | Pretty sure the biggest difference between the flake and my upstreaming attempt is that in the flake i have allowUnfree and allowUnfree as true, but those options should carry over (i assume) as i also have those enabled on my nixos and im building with --impure | 16:44:00 |
| kaya 𖤐 | * Pretty sure the biggest difference between the flake and my upstreaming attempt is that in the flake i have allowUnfree and cudaSupport as true, but those options should carry over (i assume) as i also have those enabled on my nixos and im building with --impure | 16:44:11 |
| kaya 𖤐 | * Pretty sure the biggest difference between the flake and my upstreaming attempt is that in the flake i have allowUnfree and cudaSupport as true, but those options should carry over (i assume) as i also have those enabled on my nixos config and im building with --impure | 16:50:40 |
| 9 Dec 2024 |
| SomeoneSerge (back on matrix) |
For some reason its complaining about CUDA_HOME being missing even though im specifying it which im kind of confused
It might just want some more components than just nvcc and cudart. Also off the top of my head not sure which outputs are propagated into the symlinkJoin
| 00:11:43 |
| SomeoneSerge (back on matrix) | Could you publish the full logs? | 00:11:56 |
 hexa | is there a relationship between cuda and the open nvidia kmod? | 03:40:17 |
| hexa | because my cuda things stopped working some time after migrating to 24.11 | 03:40:27 |
| hexa | though nvidia-smi is working | 03:40:47 |
| hexa | but ollama and wyoming-faster-whisper can't init cuda | 03:41:08 |
| hexa | will try to drop hardening next, as usual 😄 | 03:42:19 |
| hexa | ok, DevicePolicy related 🙂 | 03:46:59 |
| hexa | ok, apparently not | 15:44:08 |
| hexa | nvidia_uvm doesn't get loaded at boot anymore | 15:44:25 |
| hexa | * it seems like nvidia_uvm doesn't get loaded at boot anymore | 15:44:31 |
| SomeoneSerge (back on matrix) | https://github.com/NixOS/nixpkgs/issues/334180 | 15:52:57 |
| hexa | fbdcdde Kiskae 2024-05-22 13:46 +0200 308│ # Don't add `nvidia-uvm` to `kernelModules`, because we want
fbdcdde Kiskae 2024-05-22 13:46 +0200 309│ # `nvidia-uvm` be loaded only after `udev` rules for `nvidia` kernel
fbdcdde Kiskae 2024-05-22 13:46 +0200 310│ # module are applied.
fbdcdde Kiskae 2024-05-22 13:46 +0200 311│ #
fbdcdde Kiskae 2024-05-22 13:46 +0200 312│ # Instead, we use `softdep` to lazily load `nvidia-uvm` kernel module
fbdcdde Kiskae 2024-05-22 13:46 +0200 313│ # after `nvidia` kernel module is loaded and `udev` rules are applied.
fbdcdde Kiskae 2024-05-22 13:46 +0200 314│ extraModprobeConfig = ''
fbdcdde Kiskae 2024-05-22 13:46 +0200 315│ softdep nvidia post: nvidia-uvm
fbdcdde Kiskae 2024-05-22 13:46 +0200 316│ '';
| 16:03:03 |
| SomeoneSerge (back on matrix) | Yeah, somehow softdep breaks with the open driver? | 16:03:59 |
| hexa | nope, reformat | 16:03:59 |
| hexa | ok, has been there since 2023.11 | 16:04:16 |
| hexa | and yeah, I'm on the open driver | 16:04:19 |
| SomeoneSerge (back on matrix) | https://github.com/NixOS/nixpkgs/issues/334180#issuecomment-2284518816 | 16:04:30 |
