In reply to @matthewcroughan:defenestrate.it
Did you guys know that python311Packages.tensorrt is broken because someone updated pkgs/development/cuda-modules/tensorrt/releases.nix without checking that it broke any derivations?

In reply to @connorbaker:matrix.org
What revision of nixpkgs are you on? master fails to build (go-stable-diffusion errors during CMake configure)

hey all, has anyone had success using cuda libraries inside a docker container built with nix? i don't mean running a cuda container on nixos host but the opposite, running a nix container containing cuda program on another host
i build a container with nix and pytorch etc and run it on runpod, it doesnt see nvidia drivers/device though, i guess i am missing something. currently i have:

        dockerImages.default = pkgs.dockerTools.streamLayeredImage {
          name = "ghcr.io/my-image";
          tag = "latest";

          contents = [
            pkgs.bash
            pkgs.uutils-coreutils-noprefix
            pkgs.cacert
            pkgs.libnvidia-container

            pythonEnv
          ];

          config = {
            Cmd = [ "${pkgs.bash}/bin/bash" ];
            Env = [
              "CUDA_PATH=${pkgs.cudatoolkit}"
              "LD_LIBRARY_PATH=${pkgs.linuxPackages_5_4.nvidia_x11}/lib"
            ];
          };
        };

In reply to @grw00:matrix.org

hey all, has anyone had success using cuda libraries inside a docker container built with nix? i don't mean running a cuda container on nixos host but the opposite, running a nix container containing cuda program on another host
i build a container with nix and pytorch etc and run it on runpod, it doesnt see nvidia drivers/device though, i guess i am missing something. currently i have:

        dockerImages.default = pkgs.dockerTools.streamLayeredImage {
          name = "ghcr.io/my-image";
          tag = "latest";

          contents = [
            pkgs.bash
            pkgs.uutils-coreutils-noprefix
            pkgs.cacert
            pkgs.libnvidia-container

            pythonEnv
          ];

          config = {
            Cmd = [ "${pkgs.bash}/bin/bash" ];
            Env = [
              "CUDA_PATH=${pkgs.cudatoolkit}"
              "LD_LIBRARY_PATH=${pkgs.linuxPackages_5_4.nvidia_x11}/lib"
            ];
          };
        };

In reply to @ss:someonex.net
Hard coding linuxPackages in the image is a bad idea. With cuda you normally don't want drivers in the image, you want the host's drivers mounted in the containet

In reply to @grw00:matrix.org
not sure what CDI is, i understand i need the /run/opengl-driver but i'm not sure how to achieve that in docker container

In reply to @grw00:matrix.org
ah kk, got it. i'm specifically trying to use this on runpod.io, i don't think they offer this as a possibility. it seems like the images they offer all have cuda installed in image

In reply to @grw00:matrix.org
not sure what CDI is, i understand i need the /run/opengl-driver but i'm not sure how to achieve that in docker container

❯ ssh 6r0gwnq7twsots-644110b1@ssh.runpod.io

-- RUNPOD.IO --
Enjoy your Pod #6r0gwnq7twsots ^_^

bash-5.2# nvidia-smi
bash: /usr/bin/nvidia-smi: cannot execute: required file not found

In reply to @grw00:matrix.org

❯ ssh 6r0gwnq7twsots-644110b1@ssh.runpod.io

-- RUNPOD.IO --
Enjoy your Pod #6r0gwnq7twsots ^_^

bash-5.2# nvidia-smi
bash: /usr/bin/nvidia-smi: cannot execute: required file not found

In reply to @grw00:matrix.org
ok good info thx. i will check running one of the containers they offer (that does work) and see if there are any external mounts for cuda drivers, i think not though

In reply to @ss:someonex.net
Is this nvidia-smi from your hard coded linuxPackages?

bash-5.2# df
Filesystem     1K-blocks     Used Available Use% Mounted on
overlay         10485760    64056  10421704   1% /
tmpfs              65536        0     65536   0% /dev
tmpfs          132014436        0 132014436   0% /sys/fs/cgroup
shm             15728640        0  15728640   0% /dev/shm
/dev/nvme0n1p2  65478188 24385240  37721124  40% /sbin/docker-init
/dev/nvme0n1p4  52428800        0  52428800   0% /cache
udev           131923756        0 131923756   0% /dev/null
udev           131923756        0 131923756   0% /dev/tty
tmpfs          132014436       12 132014424   1% /proc/driver/nvidia
tmpfs          132014436        4 132014432   1% /etc/nvidia/nvidia-application-profiles-rc.d
tmpfs           26402888    18148  26384740   1% /run/nvidia-persistenced/socket
tmpfs          132014436        0 132014436   0% /proc/asound
tmpfs          132014436        0 132014436   0% /proc/acpi
tmpfs          132014436        0 132014436   0% /proc/scsi
tmpfs          132014436        0 132014436   0% /sys/firmware

ok, checked their ubuntu-based image and mounts look like this:

root@cc04a766e493:~# df
Filesystem     1K-blocks     Used Available Use% Mounted on
overlay         20971520    64224  20907296   1% /
tmpfs              65536        0     65536   0% /dev
tmpfs          132014448        0 132014448   0% /sys/fs/cgroup
shm             15728640        0  15728640   0% /dev/shm
/dev/nvme0n1p2  65478188 18995924  43110440  31% /usr/bin/nvidia-smi
/dev/nvme0n1p4  20971520        0  20971520   0% /workspace
tmpfs          132014448       12 132014436   1% /proc/driver/nvidia
tmpfs          132014448        4 132014444   1% /etc/nvidia/nvidia-application-profiles-rc.d
tmpfs           26402892     8832  26394060   1% /run/nvidia-persistenced/socket
tmpfs          132014448        0 132014448   0% /proc/asound
tmpfs          132014448        0 132014448   0% /proc/acpi
tmpfs          132014448        0 132014448   0% /proc/scsi
tmpfs          132014448        0 132014448   0% /sys/firmware