!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

288 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda56 Servers

You have reached the beginning of time (for this room).

SenderMessageTime
13 Dec 2024
@sielicki:matrix.orgsielickigenerally just a review of the nix code and anything I did stupid07:42:31
@msanft:matrix.orgMoritz Sanft I just noticed another weird thing while trying to hunt down that Perl dependency:

As I'm building the driver for a server scenario, I removed the graphics and X11 stuff from the libPath. I still had the Perl dependency in my image though. When analyzing its chain, I saw the following:
/nix/store/1gx9dgmj33jd1753fww5cmq0q087q48n-nixos-system-nixos-25.05pre-git
└───/nix/store/czlpjck5z3vsgw1w9szinwnv15l4a2n3-system-path
    └───/nix/store/7m1var7g0swf2ikn3d3swsxk5w6lbcpv-nvidia-persistenced-550.90.07
        └───/nix/store/wh45iphj9kr43mxq0wks9qam2swabf6f-nvidia-x11-550.90.07-6.11
            └───/nix/store/lcq3ibmsb6c2jgqp3yfi1yp773x5wz19-mesa-24.2.6
                └───/nix/store/0i5icd6l3pkjckipa5f94jv7dsj5md70-lm-sensors-3.6.0
                    └───/nix/store/3vq9qasxlqpyq1k95nq3s13g2m6w59ay-perl-5.40.0

Now, when I remove the persistenced, the dependency is gone. This means that the persistenced depends on another NVIDIA driver than what the system actually uses, somehow. The driver that's used in the system is at /nix/store/zsdr4vrybbik9hb8nss6fbmi71wsqhv3-nvidia-x11-550.90.07-6.11. When I now run nix derivation show /path/to/persistenced-package, I see the following:

"postFixup": "# Save a copy of persistenced for mounting in containers\nmkdir $out/origBin\ncp $out/{bin,origBin}/nvidia-persistenced\npatchelf --set-interpreter /lib64/ld-linux-x86-64.so.2 $out/origBin/nvidia-persistenced\n\npatchelf --set-rpath \"$(patchelf --print-rpath $out/bin/nvidia-persistenced):/nix/store/wh45iphj9kr43mxq0wks9qam2swabf6f-nvidia-x11-550.90.07-6.11/lib\" \\\n  $out/bin/nvidia-persistenced\n",

This means that another driver is used for building the persistenced somehow? Looking at the packaging infrastructure, it seems that nvidia_x11 is passed as an argument, which would mean that it should use the same one. However, I fear that there's some kind of evaluation differential here, as the persistenced package might be built before hardware.nvidia.package is even evaluated? Has anyone of you ever run into something similar before? 		08:41:49
@msanft:matrix.orgMoritz Sanft fwiw; Solved it by doing a very dirty hack that overrides the nvidia_x11 used in nvidia-persistenced explicitly:
https://github.com/edgelesssys/contrast/commit/5bf5cb81ce05f6f25b2cdf960ca3ab57a7f3459f 		15:05:40
14 Dec 2024
@matthewcroughan:defenestrate.itmatthewcroughanIs there a way to wrap programs in Nix so that they believe they have a specific directory structure, like an FHS Env, whilst not screwing around too much with things that impact Cuda/GPU access?16:47:31
@matthewcroughan:defenestrate.itmatthewcroughan

I'm trying to package an application that wants access to source code dir paths, and I think this would be a good use of a layer/wrapper that performs symlinking at runtime to change the view of the world from the perspective of the application:

  • https://github.com/BatteredBunny/nix-ai-stuff/blob/main/pkgs/comfyui/default.nix#L54-L70
  • https://github.com/lboklin/nixified-ai/blob/master/projects/comfyui/package.nix#L116-L147
16:52:25
@matthewcroughan:defenestrate.itmatthewcroughaninstead of doing it in the installPhase for example16:52:32
@matthewcroughan:defenestrate.itmatthewcroughanif you enable cudaSupport and rocmSupport, what happens? Do you actually get an output that is usable for both?20:24:53
@sielicki:matrix.orgsielicki matthewcroughan: IMO it might be faster and better for you to write the missing pyproject.toml it needs 20:34:25
@matthewcroughan:defenestrate.itmatthewcroughan Would I not have to rewrite that each and every single time the owner updates the package? 20:34:49

Show newer messages

Back to Room ListRoom Version: 9