| 31 Jul 2024 |
 Philip Taron (UTC-8) | That's a baller PR. | 19:06:12 |
| 1 Aug 2024 |
 ˈt͡sɛːzaɐ̯ | In reply to @phirsch:matrix.org
@SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
Weird. I mean, you could build the thing in a container or vm that's actually nixos, and then pull it to your store from there. But this should really work. I wonder how you're running your nix. As user? I guess the sandbox is relaxed? | 10:45:59 |
 SomeoneSerge (back on matrix) | In reply to @phirsch:matrix.org
@SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
You do need to build with cuda support in order to use cuda | 12:51:57 |
 yorickvp | I'm trying to link something to torch, but it complains
┃ > ImportError: /nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/q7hlip3anbg4gd4wqa1lwy0jksk25pck-python3.10-torch-2.3.1-lib/lib/libc10.so)
why does it use gcc-12.3.0-lib?!
| 14:32:46 |
| yorickvp | all I can find is the line
-- Looking for a CUDA host compiler - /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/bin/c++
in the build logs | 14:34:20 |
| yorickvp | okay, that would be because setup-cuda-hook sets that. but it does have -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib -L/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib | 14:44:23 |
| SomeoneSerge (back on matrix) | Are you using multiple nixpkgs revisions? | 14:46:52 |
| SomeoneSerge (back on matrix) | Ah, no, I guess the second one is propagated by something else | 14:47:07 |
| SomeoneSerge (back on matrix) | -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib looks correct | 14:47:15 |
| yorickvp | nixpkgs c66e984bda09e7230ea7b364e677c5ba4f0d36d0 | 14:47:21 |
| SomeoneSerge (back on matrix) | In reply to @yorickvp:matrix.org
all I can find is the line
-- Looking for a CUDA host compiler - /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/bin/c++
in the build logs This is the compiler | 14:47:39 |
| SomeoneSerge (back on matrix) | * This is the compiler, the libstdc++ should be from gcc13 | 14:47:52 |
| yorickvp | $ cat /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/nix-support/cc-ldflags
-L/nix/store/zc0nsv23pakbafngjy32kvhfzb16as43-gcc-13.3.0/lib/gcc/x86_64-unknown-linux-gnu/13.3.0 -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib -L/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib
| 14:48:08 |
| SomeoneSerge (back on matrix) | Yeah the way we worked around cc-wrapper was kind of ugly and relied on gcc respecting priorities/a particular order of flags | 14:51:55 |
| SomeoneSerge (back on matrix) | I'd rather guess that gcc12\s libstdc++ comes from elsewhere, like another dependency. Run with LD_DEBUG=libs | 14:52:22 |
| SomeoneSerge (back on matrix) | In reply to @ss:someonex.net
Yeah the way we worked around cc-wrapper was kind of ugly and relied on gcc respecting priorities/a particular order of flags (yes this is bad and we should fix that) | 14:52:47 |
| yorickvp | run what with LD_DEBUG=libs? | 14:53:06 |
