| 6 Jul 2024 |
SomeoneSerge (back on matrix) | In reply to @hexa:lossy.network https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/#nvcc-environment-variables Yes, we use that in setupCudaHook | 18:24:20 |
SomeoneSerge (back on matrix) | In a limited way... | 18:24:31 |
connor (he/him) | Oh that reminds me: At least one Python package with CUDA support in Nixpkgs invokes NVCC from within Python scripts as part of its setup, and so our NVCC_PREPEND_FLAGS (or whatever the name is) in our setup hook is ignored
Don’t think that’s relevant but just something irritating I found some number of weeks ago (I believe I patched it in the PR I have to update the CUDA packaging) | 23:36:00 |
| 7 Jul 2024 |
| ornx joined the room. | 19:45:55 |
ornx | this doesn't work (nvidia-smi says my card is there etc). am i holding it wrong or is it broken?
$ nix develop
[snip]
$ nvcc foo.cu
In file included from /nix/store/fydjj6z3nyi1ywqbzzw7ai12ncjx9kwy-cuda-merged-12.2/include/cuda_runtime.h:82,
from <command-line>:
/nix/store/fydjj6z3nyi1ywqbzzw7ai12ncjx9kwy-cuda-merged-12.2/include/crt/host_config.h:143:2: error: #error -- unsupported GNU version! gcc versions later than 12 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
143 | #error -- unsupported GNU version! gcc versions later than 12 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
| ^~~~~
flake.nix i am using as shell:
{
description = "cuda development environment";
inputs = {
nixpkgs = {
url = "github:NixOS/nixpkgs/4284c2b73c8bce4b46a6adf23e16d9e2ec8da4bb";
};
};
outputs = { self, nixpkgs }:
let
system = "x86_64-linux";
pkgs = import nixpkgs {
inherit system;
config.allowUnfree = true;
config.cudaSupport = true;
};
in {
devShells.${system}.default = pkgs.mkShell {
buildInputs = with pkgs; [
cudatoolkit linuxPackages.nvidia_x11
cudaPackages.cudnn
libGLU libGL
xorg.libXi xorg.libXmu freeglut
xorg.libXext xorg.libX11 xorg.libXv xorg.libXrandr zlib
ncurses5 stdenv.cc binutils
];
shellHook = ''
export LD_LIBRARY_PATH="${pkgs.linuxPackages.nvidia_x11}/lib"
'';
};
};
}
| 19:47:46 |
aidalgol | Maybe leave out stdenv.cc from the mkShell inputs? | 19:50:57 |
ornx | no such luck, same error | 19:53:36 |
ornx | even NIXPKGS_ALLOW_UNFREE=1 nix-shell -p cudaPackages_12.cudatoolkit gives me that error | 19:54:13 |
ornx | although i'm not sure what commit that's on | 19:54:29 |
ornx | $ nix-shell -p gcc12 --run 'gcc --version'
gcc (GCC) 13.3.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
| 19:55:37 |
ornx | i'm not sure what is going on here exactly, is it supposed to do that? | 19:55:54 |
aidalgol | That seems wrong. I get that, too. Also,
❯ nix-shell -p gcc12 --run 'which gcc'
/nix/store/mpm3i0sbqc9svfch6a17179fs64dz2kv-gcc-wrapper-13.3.0/bin/gcc
| 19:58:57 |
ornx | very odd... | 20:02:48 |
SomeoneSerge (back on matrix) | `*mkShell.override { stdenv = cudaPackages.backendStdenv; } | 20:53:38 |
SomeoneSerge (back on matrix) | * mkShell.override { stdenv = cudaPackages.backendStdenv; } | 20:53:46 |
SomeoneSerge (back on matrix) | * mkShell.override { stdenv = cudaPackages.backendStdenv; } { packages = [ cmake etc etc ]; } | 20:54:01 |
SomeoneSerge (back on matrix) | nix-shell -p implicitly uses mkShell, which is where gcc13 comes from in your which gcc | 20:54:28 |
SomeoneSerge (back on matrix) | ❯ nix-shell -p gcc12 --run 'echo $PATH' | rg gcc
...:/nix/store/mpm3i0sbqc9svfch6a17179fs64dz2kv-gcc-wrapper-13.3.0/bin:...:/nix/store/7hqg1pzya6xjp6vri2fsfjnf4w622p7w-gcc-wrapper-12.3.0/bin:...
| 20:55:31 |
SomeoneSerge (back on matrix) | Compare this to nix3-shell:
❯ nix shell nixpkgs#gcc12 --command which gcc
/nix/store/7hqg1pzya6xjp6vri2fsfjnf4w622p7w-gcc-wrapper-12.3.0/bin/gcc
| 20:56:16 |
SomeoneSerge (back on matrix) | And the usual reminder that LD_LIBRARY_PATH="${pkgs.linuxPackages.nvidia_x11}/lib" is wrong and breaks things | 20:56:49 |
SomeoneSerge (back on matrix) | * And here's the usual reminder that LD_LIBRARY_PATH="${pkgs.linuxPackages.nvidia_x11}/lib" is wrong and breaks things | 20:56:56 |
SomeoneSerge (back on matrix) | CC ornx | 20:57:24 |
ornx | Ah, so I *was* holding it wrong... | 21:28:01 |
| 8 Jul 2024 |
| kaya 𖤐 changed their profile picture. | 00:51:09 |
| 9 Jul 2024 |
ornx | i'm having a lot of trouble getting things working still - i'm using devShell.override with cudaPackages.backendStdenv and cuda programs compile and run with nvcc, but i'm getting a cudaErrorInsufficientDriver error even though i'm building my flake against the same revision of nixpkgs that the system was built against | 00:35:00 |
ornx | actually nvidia-smi in the system env closure has also stopped working so probably there is something wrong with my system flake lol | 00:38:13 |
ornx | okay yeah even after rebooting and fixing that i get an error when i try to actually run programs i compiled with nvcc | 00:42:55 |
ornx | is there some kind of mismatch between what driver version cudaPackages.cudatoolkit is expecting and what the system flake is using, even though they're on the same revision and config.boot.kernelPackages.nvidiaPackages is unset? | 00:43:56 |
SomeoneSerge (back on matrix) | In reply to @ornx:littledevil.club i'm having a lot of trouble getting things working still - i'm using devShell.override with cudaPackages.backendStdenv and cuda programs compile and run with nvcc, but i'm getting a cudaErrorInsufficientDriver error even though i'm building my flake against the same revision of nixpkgs that the system was built against Ohhh you're probably stil running into the stub driver issue | 00:44:33 |
SomeoneSerge (back on matrix) | * Ohhh you're probably stil running into the stub driver issue (It should be gone once the getOutput PR reaches unstable) | 00:45:20 |