In reply to @bjth:matrix.org

How would you go about conditionally setting cudaCapabilities when instantiating nixpkgs? I.e.

Image I have this.

{
  inputs = {
    nixpkgs = "github:nixos/nixpkgs?ref=nixos-25.05";
  };
  
  outputs = { self, nixpkgs }: {
    packages.x86_64-linux.default = let
      pkgs = import nixpkgs {
        overlays = [ ];
        config = {
          allowUnfree = true;
          cudaSupport = true;
          cudaCapabilities = [ "..." "..." ];
        };
      };
    in
    pkgs.hello;
    
    packages.aarch64-linux.default = let
      pkgs = import nixpkgs {
        overlays = [ ];
        config = {
          allowUnfree = true;
          cudaSupport = true;
          cudaCapabilities = if isJetson then [ "..." "..." ] else [ "..." "..." ];
        };
      };
    in
    pkgs.hello;
  };
}

It's the aarch64-linux part specifically that I'm a bit stuck on. I have some cloud servers that have an NVIDIA GPUs in them that run aarch64-linux, but I also have some Jetson devices that are also considered aarch64-linux.

And if I understand the whole thing correctly, I can't just set the cudaCapabilities list to include both the non-jetson and jetson capabilities, right? Or at least, than isJetsonBuild would just always eval to true even if the build was meant for the cloud server.

Probably something stupid I'm just overlooking, sorry for bothering. 😅

Aarch based nvidia data center gpus 👀, yeah if you get the correct map of the cuda capabilities it should work fine

Edit: misread, isJetsonBuild sounds funky so not sure

i'm confused about the compatibility story between whatever libcuda.so file i have in /run/opengl-driver and my nvidia kernel module. i've read through <nixos/modules/hardware/video/nvidia.nix> and i see that hardware.graphics.extraPackages basically gets set to pkgs.linuxKernel.packages.linux_6_12.nvidiaPackages.stable.out (or whatever kernel i have selected)

how much drift (if any) is allowed here?

In reply to @jfly:matrix.org
to avoid an XY problem: what i'm actually doing is experimenting with defining systemd nixos containers that run cuda software internally, and i'm not sure how to get the right libcuda.so's in those containers so they play nicely with the host's kernel

Do the systemd NixOS containers provide their own copy of NVIDIA's driver? If not, they wouldn't have libcuda.so available.

afaik, they do not automatically do anything (please correct me if i'm wrong). i making them get their own libcuda.so by explicitly configuring them with hardware.graphics.enable = true; and hardware.graphics.extraPackages.

mounting the cuda runtime from the host makes sense, though! thanks for the link to this nvidia-container-toolkit

Do the systemd NixOS containers provide their own copy of NVIDIA's driver? If not, they wouldn't have libcuda.so available.

They don't (unless forced). Libcuda and its closure are mounted from the host.

The issue is maybe growing stale, but I'd say there haven't been any fundamental updates.

• One bit it doesn't mention is that we rewrote most of the tests in terms of a single primitive, cudaPackages.writeGpuTestPython (can be overridden for e.g. rocm; could be moved outside cuda-modules).
• It's now also clear that the VM tests can also be done, we'd just have to use a separate marker to signal that a builder exposes an nvidia device with a vfio driver.
• If we replace the sandboxing mechanism (e.g. with microvms) it'll get trickier... but again, a low-bandwidth baseline with vfio is definitely achievable.
• And there's still the issue of describing constraints, like listing the architectures or like memory quotas: we need a pluggable mechanism for assessing which builders are compatible with the derivation? Maybe a proxy instead...