| 30 Jul 2024 |
 srhb | No problem, thanks for the response :D | 07:26:53 |
 Philip Taron (UTC-8) | SomeoneSerge (UTC+3): I'm taking a look at your llama.cpp PR. The TODO makes me think that it's in draft actually. Is that the case? | 18:22:57 |
 SomeoneSerge (utc+3) | I mean it's more of a sanity check, I tested this with a bunch of packages in nixpkgs and generally the closures got smaller | 18:23:47 |
| Philip Taron (UTC-8) | I generally check closure size with nix path-info. Do you do that, or something else? | 18:24:13 |
| Philip Taron (UTC-8) | On another topic, I see a lot of build spam when building llama.cpp about "nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used."
I'd like to remove that. Is there a pointer you have to get started there? | 18:25:46 |
| Philip Taron (UTC-8) | In reply to @philiptaron:matrix.org
I generally check closure size with nix path-info. Do you do that, or something else? Using nix path-info results in identical closure sizes. | 18:35:31 |
| SomeoneSerge (utc+3) | In reply to @philiptaron:matrix.org
On another topic, I see a lot of build spam when building llama.cpp about "nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used."
I'd like to remove that. Is there a pointer you have to get started there? Yeah it's somehwere in the setupCudaHook, I believe connor (he/him) (UTC-5) had actually located the source at some point? | 20:40:53 |
| 31 Jul 2024 |
| SomeoneSerge (utc+3) | connor (he/him) (UTC-5) you might want to know that https://github.com/NixOS/nixpkgs/pull/318614 exists | 07:57:55 |
 connor (he/him) (UTC-7) | Oh hell yeah | 15:21:27 |
| Philip Taron (UTC-8) | That's a baller PR. | 19:06:12 |
| 1 Aug 2024 |
 ˈt͡sɛːzaɐ̯ | In reply to @phirsch:matrix.org
@SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
Weird. I mean, you could build the thing in a container or vm that's actually nixos, and then pull it to your store from there. But this should really work. I wonder how you're running your nix. As user? I guess the sandbox is relaxed? | 10:45:59 |
| SomeoneSerge (utc+3) | In reply to @phirsch:matrix.org
@SomeoneSerge (UTC+3) @ˈt͡sɛːzaɐ̯ No dice... While ollama (without '-cuda') somehow manages to get GPU serial and VRAM allocation into, it doesn't use the GPU when actually running a model (outputs 'Not compiled with GPU offload support'). And unfortunately, using 'nix run --impure' as above from within a nix shell with 'nvcc' from nixpkgs still fails because it's using nvcc from /usr/local/...
You do need to build with cuda support in order to use cuda | 12:51:57 |
 yorickvp | I'm trying to link something to torch, but it complains
┃ > ImportError: /nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /nix/store/q7hlip3anbg4gd4wqa1lwy0jksk25pck-python3.10-torch-2.3.1-lib/lib/libc10.so)
why does it use gcc-12.3.0-lib?!
| 14:32:46 |
| yorickvp | all I can find is the line
-- Looking for a CUDA host compiler - /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/bin/c++
in the build logs | 14:34:20 |
| yorickvp | okay, that would be because setup-cuda-hook sets that. but it does have -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib -L/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib | 14:44:23 |
| SomeoneSerge (utc+3) | Are you using multiple nixpkgs revisions? | 14:46:52 |
| SomeoneSerge (utc+3) | Ah, no, I guess the second one is propagated by something else | 14:47:07 |
| SomeoneSerge (utc+3) | -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib looks correct | 14:47:15 |
| yorickvp | nixpkgs c66e984bda09e7230ea7b364e677c5ba4f0d36d0 | 14:47:21 |
| SomeoneSerge (utc+3) | In reply to @yorickvp:matrix.org
all I can find is the line
-- Looking for a CUDA host compiler - /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/bin/c++
in the build logs This is the compiler | 14:47:39 |
| SomeoneSerge (utc+3) | * This is the compiler, the libstdc++ should be from gcc13 | 14:47:52 |
| yorickvp | $ cat /nix/store/vk12rv84vs98bv3wi4jgbpi59lrs3ymj-gcc-wrapper-12.3.0/nix-support/cc-ldflags
-L/nix/store/zc0nsv23pakbafngjy32kvhfzb16as43-gcc-13.3.0/lib/gcc/x86_64-unknown-linux-gnu/13.3.0 -L/nix/store/bn7pnigb0f8874m6riiw6dngsmdyic1g-gcc-13.3.0-lib/lib -L/nix/store/kzx58d5pbb78gnv9s4d62f4r46x9waw9-gcc-12.3.0-lib/lib
| 14:48:08 |
| SomeoneSerge (utc+3) | Yeah the way we worked around cc-wrapper was kind of ugly and relied on gcc respecting priorities/a particular order of flags | 14:51:55 |
| SomeoneSerge (utc+3) | I'd rather guess that gcc12\s libstdc++ comes from elsewhere, like another dependency. Run with LD_DEBUG=libs | 14:52:22 |
| SomeoneSerge (utc+3) | In reply to @ss:someonex.net
Yeah the way we worked around cc-wrapper was kind of ugly and relied on gcc respecting priorities/a particular order of flags (yes this is bad and we should fix that) | 14:52:47 |
| yorickvp | run what with LD_DEBUG=libs? | 14:53:06 |
| SomeoneSerge (utc+3) | like
$ LD_DEBUG=libs python
import torch
| 14:53:26 |
| SomeoneSerge (utc+3) | * like
$ LD_DEBUG=libs python
import torch
# do whatever you do
| 14:53:34 |
| SomeoneSerge (utc+3) | * like
$ LD_DEBUG=libs python my-repro.py
| 14:53:51 |
| yorickvp | alright, I'll try that | 14:54:17 |
