| 15 Oct 2025 |
 danielrf | connor (he/him) (UTC-7): Hey, just fyi. This looks very similar to an issue we had fixed in jetpack-nixos: https://github.com/NixOS/nixpkgs/issues/451912 I can't recall if our fix was generic enough to also be applicable to the nixpkgs' nvidia-container-toolkit | 03:56:45 |
 connor (burnt/out) (UTC-8) | pain | 07:03:11 |
| connor (burnt/out) (UTC-8) | Yeah Jared had written a udevadm settle for some devices; when I refactored to use upstream’s container toolkit stuff I commented it out hoping waiting on the modprobe nvgpu service was enough | 07:05:36 |
| connor (burnt/out) (UTC-8) | Relevant PRs:
- https://github.com/anduril/jetpack-nixos/pull/317
- https://github.com/anduril/jetpack-nixos/pull/331
| 07:06:36 |
 Gaétan Lepage | connor (he/him) (UTC-7) SomeoneSerge (back on matrix) Torch 2.9.0 was just released.
I'm working on the bump.
It requires libnvshmem_host.so.3. I never heard of OpenSHMEM before.
Do we have it already in nixpkgs? | 18:50:04 |
 apyh | oh i have this in a fork, sec | 19:43:52 |
|  Ari Lotter joined the room. | 19:44:18 |
| Ari Lotter | (still me sorry, diff devices w bad key management) | 19:44:26 |
| Ari Lotter | https://github.com/PsycheFoundation/psyche/blob/main/nix/nvshmem.nix | 19:44:30 |
| Ari Lotter | we don't have nvshmem in nixpkgs | 19:44:35 |
| Ari Lotter | i'm using this with torch 2.9.0 :) | 19:45:38 |
| Gaétan Lepage | Thanks for sharing! I guess we'll need to cleanly upstream this then? | 20:03:40 |
| Gaétan Lepage | * Thanks for sharing! I guess we'll need to cleanly upstream this then. | 20:03:41 |
| connor (burnt/out) (UTC-8) | I could probably add it to the CUDA 13 PR; nvshmem is one of the dependencies of libcublasmp I didn’t try to package | 20:53:02 |
| 16 Oct 2025 |
| Ari Lotter | lmk if i can help - 2.9.0 (nightly) is in active usage in the above project | 00:46:36 |
| connor (burnt/out) (UTC-8) | Looks like it should be very doable to package — it’s a redist so shouldn’t be too bad and can re-use all the helpers we’ve got for that. Will take a closer look tomorrow | 04:53:50 |
 Niclas Overby Ⓝ | Is there something like rust-overlay for CUDA, so you can specify exactly which CUDA version to use? | 11:39:49 |
| connor (burnt/out) (UTC-8) | You can specify which CUDA version to use currently so long as it is a CUDA version supported in-tree by using an overlay or the cudaPackages.pkgs pattern (see Nixpkgs manual) | 14:08:11 |
| connor (burnt/out) (UTC-8) | Arbitrary versions aren’t something doable with the current state of things because of the amount of patching required for each package (which varies by version of course) | 14:10:31 |
| connor (burnt/out) (UTC-8) | Plus, some of these binaries aren’t stand-alone — NVCC for example requires a host compiler.
So if we wanted to support arbitrary CUDA versions, we’d need to somehow know ahead of time which host compilers and versions are supported by all NVCC releases (we have a list we maintain in tree but it’s updated manually by reading release notes).
And then we’d need to use the appropriate version of the host compiler… but what if it’s not packaged in Nixpkgs? CUDA 11 releases used GCC 9, 10, and 11 and those aren’t maintained in-tree any more. | 14:14:59 |
