30 Jul 2024 |
Philip Taron (UTC-8) | I generally check closure size with nix path-info . Do you do that, or something else? | 18:24:13 |
Philip Taron (UTC-8) | On another topic, I see a lot of build spam when building llama.cpp about "nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used."
I'd like to remove that. Is there a pointer you have to get started there? | 18:25:46 |
Philip Taron (UTC-8) | In reply to @philiptaron:matrix.org I generally check closure size with nix path-info . Do you do that, or something else? Using nix path-info results in identical closure sizes. | 18:35:31 |
SomeoneSerge (utc+3) | In reply to @philiptaron:matrix.org On another topic, I see a lot of build spam when building llama.cpp about "nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used."
I'd like to remove that. Is there a pointer you have to get started there? Yeah it's somehwere in the setupCudaHook , I believe connor (he/him) (UTC-5) had actually located the source at some point? | 20:40:53 |
31 Jul 2024 |
SomeoneSerge (utc+3) | connor (he/him) (UTC-5) you might want to know that https://github.com/NixOS/nixpkgs/pull/318614 exists | 07:57:55 |
connor (he/him) (UTC-7) | Oh hell yeah | 15:21:27 |
Philip Taron (UTC-8) | That's a baller PR. | 19:06:12 |
1 Aug 2024 |
ˈt͡sɛːzaɐ̯ | In reply to @phirsch:matrix.org
@SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
Weird. I mean, you could build the thing in a container or vm that's actually nixos, and then pull it to your store from there. But this should really work. I wonder how you're running your nix. As user? I guess the sandbox is relaxed? | 10:45:59 |
SomeoneSerge (utc+3) | In reply to @phirsch:matrix.org
@SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
You do need to build with cuda support in order to use cuda | 12:51:57 |
yorickvp | I'm trying to link something to torch, but it complains
┃ > ImportError: /nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/q7hlip3anbg4gd4wqa1lwy0jksk25pck-python3.10-torch-2.3.1-lib/lib/libc10.so)
why does it use gcc-12.3.0-lib?!
| 14:32:46 |
yorickvp | all I can find is the line
-- Looking for a CUDA host compiler - /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/bin/c++
in the build logs | 14:34:20 |
yorickvp | okay, that would be because setup-cuda-hook sets that. but it does have -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib -L/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib | 14:44:23 |
SomeoneSerge (utc+3) | Are you using multiple nixpkgs revisions? | 14:46:52 |
SomeoneSerge (utc+3) | Ah, no, I guess the second one is propagated by something else | 14:47:07 |
SomeoneSerge (utc+3) | -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib looks correct | 14:47:15 |
yorickvp | nixpkgs c66e984bda09e7230ea7b364e677c5ba4f0d36d0 | 14:47:21 |
SomeoneSerge (utc+3) | In reply to @yorickvp:matrix.org all I can find is the line -- Looking for a CUDA host compiler - /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/bin/c++ in the build logs This is the compiler | 14:47:39 |
SomeoneSerge (utc+3) | * This is the compiler, the libstdc++ should be from gcc13 | 14:47:52 |
yorickvp | $ cat /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/nix-support/cc-ldflags
-L/nix/store/zc0nsv23pakbafngjy32kvhfzb16as43-gcc-13.3.0/lib/gcc/x86_64-unknown-linux-gnu/13.3.0 -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib -L/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib
| 14:48:08 |
SomeoneSerge (utc+3) | Yeah the way we worked around cc-wrapper was kind of ugly and relied on gcc respecting priorities/a particular order of flags | 14:51:55 |
SomeoneSerge (utc+3) | I'd rather guess that gcc12\s libstdc++ comes from elsewhere, like another dependency. Run with LD_DEBUG=libs | 14:52:22 |
SomeoneSerge (utc+3) | In reply to @ss:someonex.net Yeah the way we worked around cc-wrapper was kind of ugly and relied on gcc respecting priorities/a particular order of flags (yes this is bad and we should fix that) | 14:52:47 |
yorickvp | run what with LD_DEBUG=libs? | 14:53:06 |
SomeoneSerge (utc+3) | like
$ LD_DEBUG=libs python
import torch
| 14:53:26 |
SomeoneSerge (utc+3) | * like
$ LD_DEBUG=libs python
import torch
# do whatever you do
| 14:53:34 |
SomeoneSerge (utc+3) | * like
$ LD_DEBUG=libs python my-repro.py
| 14:53:51 |
yorickvp | alright, I'll try that | 14:54:17 |
yorickvp | the failing line is LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${cudaPackages.cuda_cudart.stubs}/lib python -m pybind11_stubgen -o . bindings | 14:55:02 |
yorickvp | In reply to @ss:someonex.net
like
$ LD_DEBUG=libs python my-repro.py
okay, so my libraries have rpath $ORIGIN:/home/yorick/outputs/out/lib:/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib:/nix/store/8rzflwd9bxri4s0bpicm8bkmi2ikmv7n-nccl-2.21.5-1/lib:/nix/store/61q201jxc1g6pkbvhyyriwlm7zasa81k-openmpi-4.1.6/lib:/nix/store/g798k855fny946jnycp61vkzy27kwlyl-libcublas-12.1.3.1-lib/lib:/nix/store/dbwp0scbb0rk78m636sb7cvycz8xzgyh-glibc-2.39-52/lib:/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib:/nix/store/2v1jx43nsp9njldxh4bfljvh5wmnbzk3-python3.10-tensorrt-cu12-libs-10.2.0/lib/python3.10/site-packages/tensorrt_libs:/nix/store/ybqfab6p2p6ir9dcr6gn6rxn825wb86g-cudnn-8.9.7.29-lib/lib | 15:12:20 |
yorickvp | looks like cmake is writing it as a LINK_PATH | 15:22:52 |