| 27 Jun 2025 |
 connor (burnt/out) (UTC-8) | Off the top of my head -- are you running JetPack 6? On JetPack 5 cuda_compat only works up through 12.2.
The other thing I can think of: make sure the cuda_compat driver comes before the host driver so it's loaded first | 16:13:50 |
| connor (burnt/out) (UTC-8) | IIRC if the host driver is loaded first it ignore the one provided by cuda_compat (I ran into a bunch of issues in my fork of cuda-packages because autoAddDriverRunpath and autoAddCudaCompatHook both append to RUNPATH, so the order they execute in is significant, which is what started the whole process of me writing the array-utilities setup hooks, because if I was going to have to re-arrange arrays (hook order) I wanted to make sure I only had to write the code once and could test it). | 16:16:21 |
| connor (burnt/out) (UTC-8) | SomeoneSerge (Ever OOMed by Element): here's what I got: https://gist.github.com/ConnorBaker/d6791db3dd5a385abfc562af161856e9 | 20:56:29 |
| connor (burnt/out) (UTC-8) | It successfully finds and loads the first vendor libraries it needs (libnvrm_gpu.so and libnvrm_mem.so), but then fails to find dependencies of those (like libnvos.so) because they have empty runpaths! | 20:58:03 |
| connor (burnt/out) (UTC-8) | As an example, doing
sudo /home/connor/.local/state/nix/profile/bin/patchelf --set-rpath '$ORIGIN' /run/opengl-driver/lib/libnvrm_gpu.so
allows it to find more libraries! Not enough to succeed, but it then says
264389: find library=libcuda.so.1 [0]; searching
264389: search path=/nix/store/2jd6vf145m0ldi05rzqwwk5n43405npk-cuda_compat-12.2.34086590/compat:/run/opengl-driver/lib:/nix/store/i7n0xv8v87xybicsqhm4fpq55r0n3qim-cuda_cudart-12.2.140-lib/lib (RUNPATH from file /nix/store/i7n0xv8v87xybicsqhm4fpq55r0n3qim-cuda_cudart-12.2.140-lib/lib/libcudart.so.12)
264389: trying file=/nix/store/2jd6vf145m0ldi05rzqwwk5n43405npk-cuda_compat-12.2.34086590/compat/libcuda.so.1
264389:
264389: find library=libnvrm_gpu.so [0]; searching
264389: search path=/run/opengl-driver/lib:/nix/store/2jd6vf145m0ldi05rzqwwk5n43405npk-cuda_compat-12.2.34086590/compat (RUNPATH from file ./saxpy/bin/saxpy)
264389: trying file=/run/opengl-driver/lib/libnvrm_gpu.so
264389:
264389: find library=libnvrm_mem.so [0]; searching
264389: search path=/run/opengl-driver/lib:/nix/store/2jd6vf145m0ldi05rzqwwk5n43405npk-cuda_compat-12.2.34086590/compat (RUNPATH from file ./saxpy/bin/saxpy)
264389: trying file=/run/opengl-driver/lib/libnvrm_mem.so
264389:
264389: find library=libnvos.so [0]; searching
264389: search path=/run/opengl-driver/lib (RUNPATH from file ./saxpy/bin/saxpy)
264389: trying file=/run/opengl-driver/lib/libnvos.so
264389:
264389: find library=libnvsocsys.so [0]; searching
264389: search path=/run/opengl-driver/lib (RUNPATH from file ./saxpy/bin/saxpy)
264389: trying file=/run/opengl-driver/lib/libnvsocsys.so
264389:
264389: find library=libnvrm_sync.so [0]; searching
264389: search path=/run/opengl-driver/lib (RUNPATH from file ./saxpy/bin/saxpy)
264389: trying file=/run/opengl-driver/lib/libnvrm_sync.so
264389:
264389: find library=libnvsciipc.so [0]; searching
264389: search cache=/nix/store/gydncjm02ww60x9gamkhfwj3f34g3g8m-glibc-2.40-66/etc/ld.so.cache
264389: search path=/nix/store/gydncjm02ww60x9gamkhfwj3f34g3g8m-glibc-2.40-66/lib:/nix/store/0ga1cm2ild3sv9vg64ldizrdpfr72pvv-xgcc-14.3.0-libgcc/lib (system search path)
264389: trying file=/nix/store/gydncjm02ww60x9gamkhfwj3f34g3g8m-glibc-2.40-66/lib/libnvsciipc.so
264389: trying file=/nix/store/0ga1cm2ild3sv9vg64ldizrdpfr72pvv-xgcc-14.3.0-libgcc/lib/libnvsciipc.so
| 21:05:38 |
| connor (burnt/out) (UTC-8) | Perhaps instead of symlinking the host libs, we copy them and patchelf them so they can search in the local directory? | 21:15:48 |
| connor (burnt/out) (UTC-8) | Updated the gist so it does what I proposed in the previous message; seems to work! | 21:47:45 |
 Tristan Ross | I tried OBS and I'm trying out ollama using CUDA, works great on Ampere Altra Max. | 22:01:05 |
| 28 Jun 2025 |
|  Zexin Yuan joined the room. | 05:56:13 |
|  @rdg:matrix.org left the room. | 23:24:12 |
