| 28 Jun 2024 |
 SomeoneSerge (back on matrix) | * Also note that cudaPackages.cudatoolkit is a package for development; e.g. if pixi runs any builds (idk if it does) and you want it to use nixpkgs' cudatoolkit libraries instead of pixi libraries, that's when you include it in the shell | 13:13:30 |
 Titus | Thank you so much for your help, SO appreciated!
yeah, I had added nvidia_x11 only in the latest iteration.. I'll remove it right away.
The weird thing is that even outside the fhs, nvidia smi gives me this N/A: Have you ever seen that before? Does it tell us sth useful?
❯ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0
| 13:40:43 |
| Titus | * Thank you so much for your help, SO appreciated!
yeah, I had added nvidia_x11 only in the latest iteration.. I'll remove it right away.
The weird thing is that even outside the fhs, nvidia smi gives me this N/A: Have you ever seen that before? Does it tell us sth useful?
❯ nvidia-smi
Fri Jun 28 14:55:14 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.78 Driver Version: 550.78 CUDA Version: ERR! |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4090 On | 00000000:01:00.0 On | Off |
| 0% 43C P8 40W / 480W | 3302MiB / 24564MiB | 26% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 4090 On | 00000000:03:00.0 Off | Off |
| 0% 36C P8 29W / 480W | 12MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 13:41:09 |
| SomeoneSerge (back on matrix) | In reply to @titus-von-koeller:matrix.org
Thank you so much for your help, SO appreciated!
yeah, I had added nvidia_x11 only in the latest iteration.. I'll remove it right away.
The weird thing is that even outside the fhs, nvidia smi gives me this N/A: Have you ever seen that before? Does it tell us sth useful?
❯ nvidia-smi
Fri Jun 28 14:55:14 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.78 Driver Version: 550.78 CUDA Version: ERR! |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4090 On | 00000000:01:00.0 On | Off |
| 0% 43C P8 40W / 480W | 3302MiB / 24564MiB | 26% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 4090 On | 00000000:03:00.0 Off | Off |
| 0% 36C P8 29W / 480W | 12MiB / 24564MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
That is odd. Just to make sure, you did reboot after you had set config.boot.kernelPackages.nvidiaPackages.beta and enabled hardware.opengl? If not, please reboot because these drivers are an edge case where mutable state matters... | 13:51:30 |
| SomeoneSerge (back on matrix) | If you did, could post the output of nix run -f '<nixpkgs>' cudaPackages.saxpy? | 13:52:18 |
| Titus | ❯ nix run -f '<nixpkgs>' cudaPackages.saxpy
trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
Start
Runtime version: 12020
Driver version: 0
Host memory initialized, copying to the device
CUDA error at cudaMalloc(&xDevice, N * sizeof(float)): CUDA driver is a stub library
| 14:47:01 |
| Titus | ok, so I updated my pixi-fhs.nix according to my best understanding of what you proposed, unfortunately, the error is still the exact same.
Would you mind taking another look? I also added some other stuff to give a better overview, please let me know if any other info is needed..
https://gist.github.com/Titus-von-Koeller/e541c1175b0a191bac75b72d9964e9d0 | 15:10:22 |
| SomeoneSerge (back on matrix) | Ls /run/opengl-driver/lib/libcuda* | 15:10:39 |
| SomeoneSerge (back on matrix) | Ican habea look tmr, nowI'm just typing from the phone... | 15:11:13 |
| Titus | ❯ ls /run/opengl-driver/lib/libcuda*
/run/opengl-driver/lib/libcuda.so -> /nix/store/2paf3i2g4arx5j4m9l87zdrzsikwmizh-nvidia-x11-550.78-6.6.35/lib/libcuda.so*
/run/opengl-driver/lib/libcuda.so.1 -> /nix/store/2paf3i2g4arx5j4m9l87zdrzsikwmizh-nvidia-x11-550.78-6.6.35/lib/libcuda.so.1*
/run/opengl-driver/lib/libcuda.so.550.78 -> /nix/store/2paf3i2g4arx5j4m9l87zdrzsikwmizh-nvidia-x11-550.78-6.6.35/lib/libcuda.so.550.78*
/run/opengl-driver/lib/libcudadebugger.so -> /nix/store/2paf3i2g4arx5j4m9l87zdrzsikwmizh-nvidia-x11-550.78-6.6.35/lib/libcudadebugger.so*
/run/opengl-driver/lib/libcudadebugger.so.1 -> /nix/store/2paf3i2g4arx5j4m9l87zdrzsikwmizh-nvidia-x11-550.78-6.6.35/lib/libcudadebugger.so.1*
/run/opengl-driver/lib/libcudadebugger.so.550.78 -> /nix/store/2paf3i2g4arx5j4m9l87zdrzsikwmizh-nvidia-x11-550.78-6.6.35/lib/libcudadebugger.so.550.78*
| 15:11:19 |
| SomeoneSerge (back on matrix) | In reply to @titus-von-koeller:matrix.org
❯ nix run -f '<nixpkgs>' cudaPackages.saxpy
trace: warning: cudaPackages.autoAddDriverRunpath is deprecated, use pkgs.autoAddDriverRunpath instead
Start
Runtime version: 12020
Driver version: 0
Host memory initialized, copying to the device
CUDA error at cudaMalloc(&xDevice, N * sizeof(float)): CUDA driver is a stub library
Ok replace run with shell | 15:11:44 |
| SomeoneSerge (back on matrix) | Thein the shell do
LD_ DEBUG=libs saxpy | 15:12:15 |
| Titus | Redacted or Malformed Event | 15:12:20 |
| Titus | Redacted or Malformed Event | 15:12:22 |
| SomeoneSerge (back on matrix) | * Then in the shell do
LD_ DEBUG=libs saxpy
| 15:12:23 |
| Titus | Redacted or Malformed Event | 15:12:42 |
| Titus | ❯ LD_DEBUG=libs saxpy | xclip -sel clip
31778: find library=libcublas.so.12 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/glibc-hwcaps/x86-64-v4/libcublas.so.12
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/glibc-hwcaps/x86-64-v3/libcublas.so.12
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/glibc-hwcaps/x86-64-v2/libcublas.so.12
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcublas.so.12
31778:
31778: find library=libcudart.so.12 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcudart.so.12
31778:
31778: find library=libcublasLt.so.12 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcublasLt.so.12
31778:
31778: find library=libdl.so.2 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v4:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v3:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v2:/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib64/stubs/libdl.so.2
31778: search path=/run/opengl-driver/lib/glibc-hwcaps/x86-64-v4:/run/opengl-driver/lib/glibc-hwcaps/x86-64-v3:/run/opengl-driver/lib/glibc-hwcaps/x86-64-v2:/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/run/opengl-driver/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/run/opengl-driver/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/run/opengl-driver/lib/libdl.so.2
31778: search path=/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/glibc-hwcaps/x86-64-v4:/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/glibc-hwcaps/x86-64-v3:/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/glibc-hwcaps/x86-64-v2:/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib:/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/glibc-hwcaps/x86-64-v4:/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/glibc-hwcaps/x86-64-v3:/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/glibc-hwcaps/x86-64-v2:/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/qdsimqhpg5ln8fy3xnaywrqdqsp7sglh-glib-2.80.2-bin/lib/libdl.so.2
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/libdl.so.2
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/glibc-hwcaps/x86-64-v4:/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/glibc-hwcaps/x86-64-v3:/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/glibc-hwcaps/x86-64-v2:/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libdl.so.2
31778: search path=/run/opengl-driver-32/lib/glibc-hwcaps/x86-64-v4:/run/opengl-driver-32/lib/glibc-hwcaps/x86-64-v3:/run/opengl-driver-32/lib/glibc-hwcaps/x86-64-v2:/run/opengl-driver-32/lib (LD_LIBRARY_PATH)
31778: trying file=/run/opengl-driver-32/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/run/opengl-driver-32/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/run/opengl-driver-32/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/run/opengl-driver-32/lib/libdl.so.2
31778: search path=/run/opengl-driver/lib:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/glibc-hwcaps/x86-64-v4:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/glibc-hwcaps/x86-64-v3:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/glibc-hwcaps/x86-64-v2:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/glibc-hwcaps/x86-64-v4:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/glibc-hwcaps/x86-64-v3:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/glibc-hwcaps/x86-64-v2:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libdl.so.2
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/libdl.so.2
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/libdl.so.2
31778: search path=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/glibc-hwcaps/x86-64-v4:/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/glibc-hwcaps/x86-64-v3:/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/glibc-hwcaps/x86-64-v2:/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib (system search path)
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/glibc-hwcaps/x86-64-v4/libdl.so.2
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/glibc-hwcaps/x86-64-v3/libdl.so.2
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/glibc-hwcaps/x86-64-v2/libdl.so.2
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libdl.so.2
31778:
31778: find library=librt.so.1 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/librt.so.1
31778: search path=/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/librt.so.1
31778: search path=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/librt.so.1
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/librt.so.1
31778: search path=/run/opengl-driver-32/lib (LD_LIBRARY_PATH)
31778: trying file=/run/opengl-driver-32/lib/librt.so.1
31778: search path=/run/opengl-driver/lib:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/librt.so.1
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/librt.so.1
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/librt.so.1
31778: search path=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib (system search path)
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/librt.so.1
31778:
31778: find library=libpthread.so.0 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libpthread.so.0
31778: search path=/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libpthread.so.0
31778: search path=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/libpthread.so.0
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libpthread.so.0
31778: search path=/run/opengl-driver-32/lib (LD_LIBRARY_PATH)
31778: trying file=/run/opengl-driver-32/lib/libpthread.so.0
31778: search path=/run/opengl-driver/lib:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libpthread.so.0
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/libpthread.so.0
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/libpthread.so.0
31778: search path=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib (system search path)
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libpthread.so.0
31778:
31778: find library=libstdc++.so.6 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libstdc++.so.6
31778: search path=/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libstdc++.so.6
31778: search path=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/libstdc++.so.6
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6
31778:
31778: find library=libm.so.6 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libm.so.6
31778: search path=/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libm.so.6
31778: search path=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/libm.so.6
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libm.so.6
31778: search path=/run/opengl-driver-32/lib (LD_LIBRARY_PATH)
31778: trying file=/run/opengl-driver-32/lib/libm.so.6
31778: search path=/run/opengl-driver/lib:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libm.so.6
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/libm.so.6
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/libm.so.6
31778: search path=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib (system search path)
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libm.so.6
31778:
31778: find library=libgcc_s.so.1 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libgcc_s.so.1
31778: search path=/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libgcc_s.so.1
31778: search path=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/libgcc_s.so.1
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libgcc_s.so.1
31778:
31778: find library=libc.so.6 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libc.so.6
31778: search path=/run/opengl-driver/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libc.so.6
31778: search path=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/svh3pgaaisqsnv7wk5i5xf270a5n7p2g-libglvnd-1.7.0/lib/libc.so.6
31778: search path=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib (RUNPATH from file saxpy)
31778: trying file=/nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libc.so.6
31778: search path=/run/opengl-driver-32/lib (LD_LIBRARY_PATH)
31778: trying file=/run/opengl-driver-32/lib/libc.so.6
31778: search path=/run/opengl-driver/lib:/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib:/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib (RUNPATH from file saxpy)
31778: trying file=/run/opengl-driver/lib/libc.so.6
31778: trying file=/nix/store/ldq93dc0pvdj9fskqrkh6zsr45jh7sx7-libcublas-12.2.5.6/lib/libc.so.6
31778: trying file=/nix/store/mdqwq2v8svja3rmk2xb9rq3ikx44vdfd-cuda_cudart-12.2.140/lib/libc.so.6
31778: search path=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib (system search path)
31778: trying file=/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6
31778:
31778:
31778: calling init: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/ld-linux-x86-64.so.2
31778:
31778:
31778: calling init: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6
31778:
31778:
31778: calling init: /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libgcc_s.so.1
31778:
31778:
31778: calling init: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libm.so.6
31778:
31778:
31778: calling init: /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6
31778:
31778:
31778: calling init: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libpthread.so.0
31778:
31778:
31778: calling init: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/librt.so.1
31778:
31778:
31778: calling init: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libdl.so.2
31778:
31778:
31778: calling init: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcublasLt.so.12
31778:
31778: find library=libcuda.so.1 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
31778: trying file=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcuda.so.1
31778:
31778:
31778: calling init: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcuda.so.1
31778:
31778:
31778: calling init: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcudart.so.12
31778:
31778:
31778: calling init: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcublas.so.12
31778:
31778:
31778: initialize program: saxpy
31778:
31778:
31778: transferring control: saxpy
31778:
Start
Runtime version: 12020
Driver version: 0
Host memory initialized, copying to the device
CUDA error at cudaMalloc(&xDevice, N * sizeof(float)): CUDA driver is a stub library
31778:
31778: calling fini: [0]
31778:
31778:
31778: calling fini: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcublas.so.12 [0]
31778:
31778:
31778: calling fini: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcudart.so.12 [0]
31778:
31778:
31778: calling fini: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcublasLt.so.12 [0]
31778:
31778:
31778: calling fini: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libdl.so.2 [0]
31778:
31778:
31778: calling fini: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/librt.so.1 [0]
31778:
31778:
31778: calling fini: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libpthread.so.0 [0]
31778:
31778:
31778: calling fini: /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libstdc++.so.6 [0]
31778:
31778:
31778: calling fini: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libm.so.6 [0]
31778:
31778:
31778: calling fini: /nix/store/p44qan69linp3ii0xrviypsw2j4qdcp2-gcc-13.2.0-lib/lib/libgcc_s.so.1 [0]
31778:
31778:
31778: calling fini: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6 [0]
31778:
31778:
31778: calling fini: /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/ld-linux-x86-64.so.2 [0]
31778:
31778:
31778: calling fini: /nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib/libcuda.so.1 [0]
31778:
| 15:14:18 |
| Titus | i'll keep updating the gist with the output of the commands that you make me run for anyone joining the party and trying to get a complete picture :)
https://gist.github.com/Titus-von-Koeller/e541c1175b0a191bac75b72d9964e9d0 | 15:29:54 |
 hexa | https://github.com/NixOS/nixpkgs/pull/323154#event-13334193852 | 16:57:30 |
| 29 Jun 2024 |
| SomeoneSerge (back on matrix) | In reply to @titus-von-koeller:matrix.org
i'll keep updating the gist with the output of the commands that you make me run for anyone joining the party and trying to get a complete picture :)
https://gist.github.com/Titus-von-Koeller/e541c1175b0a191bac75b72d9964e9d0
31778:
31778: find library=libcuda.so.1 [0]; searching
31778: search path=/nix/store/7w8bfjj6njzc8gbyqibq3ss4yp4fkjky-cuda-merged-12.2/lib (LD_LIBRARY_PATH)
| 02:19:10 |
| SomeoneSerge (back on matrix) | You've set an LD_LIBRARY_PATH to cudatoolkit which currently, after a certain regression, inlcudes the stub driver | 02:19:46 |
| SomeoneSerge (back on matrix) | LD_LIBRARY_PATH is often a footgun | 02:20:05 |
| SomeoneSerge (back on matrix) | * LD_LIBRARY_PATH is often a footgun, it has a higher priority than DT_RUNPATH (which would have lead the linker to /run/opengl-dirver/lib/libcuda.so | 02:20:29 |
| SomeoneSerge (back on matrix) | Unset LD_LIBRARY_PATH or remove cudatoolkit from there | 02:20:38 |
 connor (he/him) | Hey all; I expect I'll be away for about a week starting in a few hours as I try to find housing on the other side of the country :) | 07:36:26 |
|  @nscnt:matrix.org left the room. | 08:09:04 |
| SomeoneSerge (back on matrix) | In reply to @connorbaker:matrix.org
Hey all; I expect I'll be away for about a week starting in a few hours as I try to find housing on the other side of the country :) Wow, moving again? | 11:57:29 |
| SomeoneSerge (back on matrix) | In reply to @ss:someonex.net
Should work this time: https://github.com/NixOS/nixpkgs/pull/323056
Can I bum a nixpgks-review? xD I think this only really breaks jax and tensorflow, which are broken anyway 🤔 | 13:24:52 |
| connor (he/him) | In reply to @ss:someonex.net
Wow, moving again? This time just me (not my dad and brother), since I’m moving for work | 13:52:42 |
|  mib 🥐 changed their profile picture. | 22:24:38 |
