!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

276 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda57 Servers

Load older messages


SenderMessageTime
1 Nov 2025
@eveeifyeve:matrix.org@eveeifyeve:matrix.org left the room.23:19:24
2 Nov 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) changed their display name from connor (burnt/out) (UTC-7) to connor (burnt/out) (UTC-8).08:13:06
@glepage:matrix.orgGaétan Lepage RE cuda_nvcc leaking into nccl:
As a sidenote, I realized that one could remove cuda_nvcc from nativeBuildInputs and (getInclude cuda_nvcc) from buildInputs without breaking nccl's build.
This most probably works because of the makeFlags.
Unfortunately, this does not help with the leakage.
10:58:02
@felix:neode.sefelix joined the room.14:30:39
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) Gaétan Lepageis https://github.com/NixOS/nixpkgs/pull/457803 ready to merge? I’ll approve and merge if so 15:15:24
@glepage:matrix.orgGaétan Lepage It fixes the leak for nccl, but firefox gets gcc-wrapper from onnxruntime too. 16:42:16
@glepage:matrix.orgGaétan LepageI'm about to push a commit that handles that too. I'm compiling rn.16:42:28
@glepage:matrix.orgGaétan Lepage Rebuilt onnxruntime. It now doesn't depend on cuda_nvcc at runtime.
I'm now rebuilding firefox which should not have cuda_nvcc in its closure anymore.
17:07:36
@glepage:matrix.orgGaétan Lepage

😭 cudaPackages.cuda_cudart depends on cudaPackages.cuda_nvcc at runtime too!!!

Not because of a path leak in the binary this time, just because nvcc is in cudart's propagatedBuildInputs (I think?)

❯ nix why-depends --precise $(nom-build --arg config '{ allowUnfree = true; cudaSupport = true; }' -A firefox-unwrapped) $(nom-build --arg config '{ allowUnfree = true; cudaSupport = true; }' -A cudaPackages.cuda_nvcc)
Finished at 18:16:53 after 1s
Finished at 18:16:53 after 0s
/nix/store/yy1z5y3iql9r4kpslxnjdwcygx52ssl8-firefox-unwrapped-144.0.2
└───lib/firefox/libonnxruntime.so: …st be specified....../nix/store/jk4a7v44fc83ykc15b31r4m21yqc92sp-onnxruntime-1.22.2/lib/.....onn…
    → /nix/store/jk4a7v44fc83ykc15b31r4m21yqc92sp-onnxruntime-1.22.2
    └───lib/libonnxruntime_providers_cuda.so: …nn-9.13.0.50-lib/lib:/nix/store/80x699lyc99dahf85iqdv6z1f0vv6vz2-cuda12.8-cuda_cudart-12.8.90/li…
        → /nix/store/80x699lyc99dahf85iqdv6z1f0vv6vz2-cuda12.8-cuda_cudart-12.8.90
        └───nix-support/propagated-build-inputs: …fhjm-setup-cuda-hook /nix/store/ygd3s9zm1pf77n3q3ac63v58www5scbc-cuda12.8-cuda_nvcc-12.8.93 /nix…
            → /nix/store/ygd3s9zm1pf77n3q3ac63v58www5scbc-cuda12.8-cuda_nvcc-12.8.93
18:19:31
@glepage:matrix.orgGaétan Lepage Actually, rebasing my PR on top of [SomeoneSerge (back on matrix)'s](https://github.com/NixOS/nixpkgs/pull/457424) worked! 20:15:56
@glepage:matrix.orgGaétan Lepage * Actually, rebasing my PR on top of Serge's worked! 20:16:12
3 Nov 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Are they good to go or do they need more testing?00:25:39
@glepage:matrix.orgGaétan Lepage According to me, they are both good to go.
Let's wait for SomeoneSerge (back on matrix)'s ACK just to be sure.
00:26:12
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Thank you both for working on that00:26:26
@glepage:matrix.orgGaétan Lepage But I confirm that firefox builds fine (no gcc-wrapper triggering disallowedRequisited) with both PRs applied. 00:26:58
@daniel-fahey:matrix.orgDaniel FaheyCUDA refactor victim fix https://github.com/NixOS/nixpkgs/pull/457870 ready to merge13:09:11
@collinarnett:matrix.orgCollin Arnett changed their profile picture.15:23:43
@arilotter:matrix.orgAri Lotter

is this a horrible idea, if i need cuda support and don't want to wait hours for builds? :)

(final: prev: {
  python312Packages = prev.python312Packages.override {
    overrides = pyfinal: pyprev: {
      torch = pyfinal.torch-bin;
    };
  };
})
21:28:33
@glepage:matrix.orgGaétan Lepage

RE {cudaPackages.nccl, onnxruntime}: remove reference to nvcc in binary:

We need to patch both nccl's libnccl.so and onnxruntime's libonnxruntime_providers_cuda.so for the fix to actually work.

23:10:06
4 Nov 2025
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) should be fine, but I'd always recommend using pythonPackagesExtensions since it's a little nicer to use 06:38:57
@ss:someonex.netSomeoneSerge (back on matrix)I still have no explanation for why we cannot seem to reproduce the nvcc reference with saxpy15:07:47
@ss:someonex.netSomeoneSerge (back on matrix)It's frustrating15:08:03
@ss:someonex.netSomeoneSerge (back on matrix)Elaborated on github, but here for redundancy: the reference in onnxruntime only appears when nvcc is propagated by all these cuda libs, https://github.com/NixOS/nixpkgs/pull/457424#issuecomment-347573673815:11:32
@glepage:matrix.orgGaétan Lepage TIL: python3Packages.torchWithRocm is apprently sensitive to config.cudaSupport. 20:11:25
@arilotter:matrix.orgAri Lotterugh i wish we could compile packages with cudaCapabilities individually per-capability and merge them later, it's such a nightmare adding one new capability level and it causing a huge 8-hour recompile..20:40:40
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)These aliases must die, they make my life so difficult21:45:22
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Join the club And it’s not even like we could do a mega-build in an intermediate derivation and then prune unused capabilities according to whatever the user requested because the amount of generated device code is so large linking will fail lmao21:46:17
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8) Gaétan Lepage are any of SomeoneSerge (back on matrix)’s comments on https://github.com/NixOS/nixpkgs/pull/457803 actionable or is it good to merge? 21:48:00
@connorbaker:matrix.orgconnor (burnt/out) (UTC-8)Also, would you mind reviewing https://github.com/NixOS/nixpkgs/pull/458619?21:48:09
@hacker1024:matrix.orghacker1024

This is most likely due to a dependency, but I will also point out that all torch variants are at the moment due to an unconditional version access

https://github.com/NixOS/nixpkgs/blob/b3d51a0365f6695e7dd5cdf3e180604530ed33b4/pkgs/development/python-modules/torch/source/default.nix#L458

21:48:19

Show newer messages


Back to Room ListRoom Version: 9