| 1 Nov 2025 |
| @eveeifyeve:matrix.org left the room. | 23:19:24 |
| 2 Nov 2025 |
| connor (burnt/out) (UTC-8) changed their display name from connor (burnt/out) (UTC-7) to connor (burnt/out) (UTC-8). | 08:13:06 |
Gaétan Lepage | RE cuda_nvcc leaking into nccl:
As a sidenote, I realized that one could remove cuda_nvcc from nativeBuildInputs and (getInclude cuda_nvcc) from buildInputs without breaking nccl's build.
This most probably works because of the makeFlags.
Unfortunately, this does not help with the leakage. | 10:58:02 |
| felix joined the room. | 14:30:39 |
connor (burnt/out) (UTC-8) | Gaétan Lepageis https://github.com/NixOS/nixpkgs/pull/457803 ready to merge? I’ll approve and merge if so | 15:15:24 |
Gaétan Lepage | It fixes the leak for nccl, but firefox gets gcc-wrapper from onnxruntime too. | 16:42:16 |
Gaétan Lepage | I'm about to push a commit that handles that too. I'm compiling rn. | 16:42:28 |
Gaétan Lepage | Rebuilt onnxruntime. It now doesn't depend on cuda_nvcc at runtime.
I'm now rebuilding firefox which should not have cuda_nvcc in its closure anymore. | 17:07:36 |
Gaétan Lepage | 😭 cudaPackages.cuda_cudart depends on cudaPackages.cuda_nvcc at runtime too!!!
Not because of a path leak in the binary this time, just because nvcc is in cudart's propagatedBuildInputs (I think?)
❯ nix why-depends --precise $(nom-build --arg config '{ allowUnfree = true; cudaSupport = true; }' -A firefox-unwrapped) $(nom-build --arg config '{ allowUnfree = true; cudaSupport = true; }' -A cudaPackages.cuda_nvcc)
Finished at 18:16:53 after 1s
Finished at 18:16:53 after 0s
/nix/store/yy1z5y3iql9r4kpslxnjdwcygx52ssl8-firefox-unwrapped-144.0.2
└───lib/firefox/libonnxruntime.so: …st be specified....../nix/store/jk4a7v44fc83ykc15b31r4m21yqc92sp-onnxruntime-1.22.2/lib/.....onn…
→ /nix/store/jk4a7v44fc83ykc15b31r4m21yqc92sp-onnxruntime-1.22.2
└───lib/libonnxruntime_providers_cuda.so: …nn-9.13.0.50-lib/lib:/nix/store/80x699lyc99dahf85iqdv6z1f0vv6vz2-cuda12.8-cuda_cudart-12.8.90/li…
→ /nix/store/80x699lyc99dahf85iqdv6z1f0vv6vz2-cuda12.8-cuda_cudart-12.8.90
└───nix-support/propagated-build-inputs: …fhjm-setup-cuda-hook /nix/store/ygd3s9zm1pf77n3q3ac63v58www5scbc-cuda12.8-cuda_nvcc-12.8.93 /nix…
→ /nix/store/ygd3s9zm1pf77n3q3ac63v58www5scbc-cuda12.8-cuda_nvcc-12.8.93
| 18:19:31 |
Gaétan Lepage | Actually, rebasing my PR on top of [SomeoneSerge (back on matrix)'s](https://github.com/NixOS/nixpkgs/pull/457424) worked! | 20:15:56 |
Gaétan Lepage | * Actually, rebasing my PR on top of Serge's worked! | 20:16:12 |
| 3 Nov 2025 |
connor (burnt/out) (UTC-8) | Are they good to go or do they need more testing? | 00:25:39 |
Gaétan Lepage | According to me, they are both good to go.
Let's wait for SomeoneSerge (back on matrix)'s ACK just to be sure. | 00:26:12 |
connor (burnt/out) (UTC-8) | Thank you both for working on that | 00:26:26 |
Gaétan Lepage | But I confirm that firefox builds fine (no gcc-wrapper triggering disallowedRequisited) with both PRs applied. | 00:26:58 |
Daniel Fahey | CUDA refactor victim fix https://github.com/NixOS/nixpkgs/pull/457870 ready to merge | 13:09:11 |
| Collin Arnett changed their profile picture. | 15:23:43 |
Ari Lotter | is this a horrible idea, if i need cuda support and don't want to wait hours for builds? :)
(final: prev: {
python312Packages = prev.python312Packages.override {
overrides = pyfinal: pyprev: {
torch = pyfinal.torch-bin;
};
};
})
| 21:28:33 |
Gaétan Lepage | RE {cudaPackages.nccl, onnxruntime}: remove reference to nvcc in binary:
We need to patch both nccl's libnccl.so and onnxruntime's libonnxruntime_providers_cuda.so for the fix to actually work. | 23:10:06 |
| 4 Nov 2025 |
connor (burnt/out) (UTC-8) | should be fine, but I'd always recommend using pythonPackagesExtensions since it's a little nicer to use | 06:38:57 |
SomeoneSerge (back on matrix) | I still have no explanation for why we cannot seem to reproduce the nvcc reference with saxpy | 15:07:47 |
SomeoneSerge (back on matrix) | It's frustrating | 15:08:03 |
SomeoneSerge (back on matrix) | Elaborated on github, but here for redundancy: the reference in onnxruntime only appears when nvcc is propagated by all these cuda libs, https://github.com/NixOS/nixpkgs/pull/457424#issuecomment-3475736738 | 15:11:32 |
Gaétan Lepage | TIL: python3Packages.torchWithRocm is apprently sensitive to config.cudaSupport. | 20:11:25 |
Ari Lotter | ugh i wish we could compile packages with cudaCapabilities individually per-capability and merge them later, it's such a nightmare adding one new capability level and it causing a huge 8-hour recompile.. | 20:40:40 |
connor (burnt/out) (UTC-8) | These aliases must die, they make my life so difficult | 21:45:22 |
connor (burnt/out) (UTC-8) | Join the club
And it’s not even like we could do a mega-build in an intermediate derivation and then prune unused capabilities according to whatever the user requested because the amount of generated device code is so large linking will fail lmao | 21:46:17 |
connor (burnt/out) (UTC-8) | Gaétan Lepage are any of SomeoneSerge (back on matrix)’s comments on https://github.com/NixOS/nixpkgs/pull/457803 actionable or is it good to merge? | 21:48:00 |
connor (burnt/out) (UTC-8) | Also, would you mind reviewing https://github.com/NixOS/nixpkgs/pull/458619? | 21:48:09 |
hacker1024 | This is most likely due to a dependency, but I will also point out that all torch variants are at the moment due to an unconditional version access
https://github.com/NixOS/nixpkgs/blob/b3d51a0365f6695e7dd5cdf3e180604530ed33b4/pkgs/development/python-modules/torch/source/default.nix#L458
| 21:48:19 |