!eWOErHSaiddIbsUNsJ:nixos.org

NixOS CUDA

290 Members
CUDA packages maintenance and support in nixpkgs | https://github.com/orgs/NixOS/projects/27/ | https://nixos.org/manual/nixpkgs/unstable/#cuda57 Servers

Load older messages

SenderMessageTime
24 Jun 2024
@glepage:matrix.orgGaétan Lepage Link: https://github.com/GaetanLepage/nix-config/blob/master/home/modules/tui/dev/nix/review.sh 13:31:22
@glepage:matrix.orgGaétan Lepage https://github.com/GaetanLepage/nix-config/blob/91a5a77f1bcfb779dbe4daa86702de8c5483c19e/home/modules/tui/dev/nix/default.nix#L16-L23 13:31:39
@connorbaker:matrix.orgconnor (he/him)

Every time I try to contribute I get distracted by something in my periphery and go down a different rabbit hole :/

I definitely want to improve the markdown output (to better support multi-line nixpkgs configs) and find a way to do evaluation faster.

13:43:43
@glepage:matrix.orgGaétan Lepage Yes, on one of my PRs you have put some very fancy markdown outputs.
Also, I noticed that natsukium was providing a link to each failing build log, that's very cool 		14:39:09
@ss:someonex.netSomeoneSerge (back on matrix)
In reply to @hexa:lossy.network
what are you quoting?
I can't remember exactly, but there was a moment when joked along the lines of "that's what hydra is for" in relation to some mildly nasty test suite 🙃 		15:16:20
26 Jun 2024
@hexa:lossy.networkhexa SomeoneSerge (UTC+3): how would you describe the current situation of the cuda maintainers cache? 01:05:54
@hexa:lossy.networkhexa * SomeoneSerge (UTC+3): how would you describe the current state of the cuda maintainers cache? 01:06:02
@hexa:lossy.networkhexaI think every rebuild in the last few months for my homeserver took ~3-5 hours 😄 01:12:33
@ss:someonex.netSomeoneSerge (back on matrix)
In reply to @hexa:lossy.network
SomeoneSerge (UTC+3): how would you describe the current state of the cuda maintainers cache?
I'd describe it as "sad". I ditched all but the "default" job because hercules effects (used to update the lock file) were misbehaving. I had patched the hole with a github action and switched to other business 		01:17:58
@ss:someonex.netSomeoneSerge (back on matrix)I'm slowly getting my sh-t together, let's chat again about hydra later this week01:44:12
@glepage:matrix.orgGaétan Lepage Who provides cuda_runtime.h ? 13:22:13
@hexa:lossy.networkhexa 
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7981/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_aligned_k64.cu.ou.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7982/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536.cu.o.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7983/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536_dropout.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) client_loop: send disconnect: Broken pipe
stderr) error: Nix daemon disconnected unexpectedly (maybe it crashed?)
stderr) error: builder for '/nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv' failed with exit code 1;
stderr)        last 1 log lines:
stderr)        > client_loop: send disconnect: Broken pipe
stderr)        For full logs, run 'nix log /nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv'.
13:25:27
@hexa:lossy.networkhexa😭13:25:31
@glepage:matrix.orgGaétan Lepage
In reply to @hexa:lossy.network
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7981/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_aligned_k64.cu.ou.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7982/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536.cu.o.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7983/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536_dropout.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) client_loop: send disconnect: Broken pipe
stderr) error: Nix daemon disconnected unexpectedly (maybe it crashed?)
stderr) error: builder for '/nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv' failed with exit code 1;
stderr)        last 1 log lines:
stderr)        > client_loop: send disconnect: Broken pipe
stderr)        For full logs, run 'nix log /nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv'.
8253 ? That's torchWithCuda. (Why the hell do I know that by heart 🫠 ?) 		13:26:36
@hexa:lossy.networkhexabecause you are as emotionally damaged by cudaSupport as me13:27:07
@hexa:lossy.networkhexathe only thing worse is being gaslit by bazel build jobs13:27:28
@glepage:matrix.orgGaétan Lepage
In reply to @hexa:lossy.network
the only thing worse is being gaslit by bazel build jobs
I suffer from this too 		13:27:56
@ss:someonex.netSomeoneSerge (back on matrix) 
❯ ag cuda_runtime.h
...

pkgs/development/python-modules/torch/default.nix
453:        cuda_cudart.dev # cuda_runtime.h and libraries
13:32:00
@hexa:lossy.networkhexathe first 7800 build steps of torchWithCuda are essentially free14:17:54
@hexa:lossy.networkhexaI'm not sure what happens between 7800 and 825314:18:01
@hexa:lossy.networkhexait feels like I'm doing proof of work in that range14:18:12
@glepage:matrix.orgGaétan Lepagecuda kernels :/14:18:55
@glepage:matrix.orgGaétan LepageThey are excuciatingly slow14:19:17
@ss:someonex.netSomeoneSerge (back on matrix)And it's good to limit parallelism there...14:19:31
@hexa:lossy.networkhexaimage.png
Download image.png		14:20:28
@hexa:lossy.networkhexado we? 😄 14:20:29
@hexa:lossy.networkhexaI mean, its all userspace load, so it's not too bad14:20:44
@hexa:lossy.networkhexasome 5% system load14:21:06
@hexa:lossy.networkhexa 
error: builder for '/nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv' failed with exit code 1;
       last 1 log lines:
       > client_loop: send disconnect: Broken pipe
       For full logs, run 'nix log /nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv'.
15:40:24
@hexa:lossy.networkhexaremote building faiss fails repeatedly 😕15:40:42

Show newer messages

Back to Room ListRoom Version: 9