| 24 Jun 2024 |
hexa | *
"Wave propagation modules for PyTorch"
| 10:11:12 |
hexa |
"Open-source software for simulating the dynamics of closed and open quantum systems"
| 10:11:27 |
SomeoneSerge (back on matrix) | "Not just an infra problem" | 10:14:54 |
hexa | what are you quoting? | 10:20:20 |
connor (he/him) | In reply to @glepage:matrix.org Thanks for taking the time connor (he/him) (UTC-5).
I actually wanted you to try my jaxlib branch.
The jax branch is for the jax update and it is basically stuck becaus of some important changes in the upstream packaging/build process I tried but didn’t see a jaxlib branch :/ | 12:51:28 |
Gaétan Lepage | Oh indeed, I deleted it after the PR was merged. | 13:09:03 |
connor (he/him) | Sorry I didn’t get to it sooner, my bad :( | 13:19:45 |
connor (he/him) | Gaétan Lepage: you mentioned you had a wrapper (script?) around Nixpkgs-review which ran it with different configurations. Any chance you can share it? I’ve been wanting to have some way to have Nixpkgs-review generate an expression for everything it would build, parameterized by multiple Nixpkgs configs. | 13:21:40 |
Gaétan Lepage | In reply to @connorbaker:matrix.org Sorry I didn’t get to it sooner, my bad :( No worry ! | 13:29:52 |
Gaétan Lepage | In reply to @connorbaker:matrix.org Gaétan Lepage: you mentioned you had a wrapper (script?) around Nixpkgs-review which ran it with different configurations. Any chance you can share it? I’ve been wanting to have some way to have Nixpkgs-review generate an expression for everything it would build, parameterized by multiple Nixpkgs configs. Yes, I have a script which start nixpkgs-review for several architectures at once.
My plan long term is to contribute to upstream for being able to run the tool for several architectures at once and have a single message posted on the PR | 13:31:20 |
Gaétan Lepage | Link: https://github.com/GaetanLepage/nix-config/blob/master/home/modules/tui/dev/nix/review.sh | 13:31:22 |
Gaétan Lepage | https://github.com/GaetanLepage/nix-config/blob/91a5a77f1bcfb779dbe4daa86702de8c5483c19e/home/modules/tui/dev/nix/default.nix#L16-L23 | 13:31:39 |
connor (he/him) | Every time I try to contribute I get distracted by something in my periphery and go down a different rabbit hole :/
I definitely want to improve the markdown output (to better support multi-line nixpkgs configs) and find a way to do evaluation faster. | 13:43:43 |
Gaétan Lepage | Yes, on one of my PRs you have put some very fancy markdown outputs.
Also, I noticed that natsukium was providing a link to each failing build log, that's very cool | 14:39:09 |
SomeoneSerge (back on matrix) | In reply to @hexa:lossy.network what are you quoting? I can't remember exactly, but there was a moment when joked along the lines of "that's what hydra is for" in relation to some mildly nasty test suite 🙃 | 15:16:20 |
| 26 Jun 2024 |
hexa | SomeoneSerge (UTC+3): how would you describe the current situation of the cuda maintainers cache? | 01:05:54 |
hexa | * SomeoneSerge (UTC+3): how would you describe the current state of the cuda maintainers cache? | 01:06:02 |
hexa | I think every rebuild in the last few months for my homeserver took ~3-5 hours 😄 | 01:12:33 |
SomeoneSerge (back on matrix) | In reply to @hexa:lossy.network SomeoneSerge (UTC+3): how would you describe the current state of the cuda maintainers cache? I'd describe it as "sad". I ditched all but the "default" job because hercules effects (used to update the lock file) were misbehaving. I had patched the hole with a github action and switched to other business | 01:17:58 |
SomeoneSerge (back on matrix) | I'm slowly getting my sh-t together, let's chat again about hydra later this week | 01:44:12 |
Gaétan Lepage | Who provides cuda_runtime.h ? | 13:22:13 |
hexa | stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7981/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_aligned_k64.cu.ou.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7982/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536.cu.o.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7983/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536_dropout.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) client_loop: send disconnect: Broken pipe
stderr) error: Nix daemon disconnected unexpectedly (maybe it crashed?)
stderr) error: builder for '/nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv' failed with exit code 1;
stderr) last 1 log lines:
stderr) > client_loop: send disconnect: Broken pipe
stderr) For full logs, run 'nix log /nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv'.
| 13:25:27 |
hexa | 😭 | 13:25:31 |
Gaétan Lepage | In reply to @hexa:lossy.network
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7981/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_aligned_k64.cu.ou.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7982/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536.cu.o.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) [7983/8253] Building CUDA object caffe2/CMakeFiles/torch_cuda.dir/__/aten/src/ATen/native/transformers/cuda/mem_eff_attention/kernels/cutlassB_f16_notaligned_k65536_dropout.cu.o
stderr) nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
stderr) client_loop: send disconnect: Broken pipe
stderr) error: Nix daemon disconnected unexpectedly (maybe it crashed?)
stderr) error: builder for '/nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv' failed with exit code 1;
stderr) last 1 log lines:
stderr) > client_loop: send disconnect: Broken pipe
stderr) For full logs, run 'nix log /nix/store/5dx3yrij3jn4fybsmxvl6dk6d4hl7hzg-faiss-1.7.4.drv'.
8253 ? That's torchWithCuda. (Why the hell do I know that by heart 🫠 ?) | 13:26:36 |
hexa | because you are as emotionally damaged by cudaSupport as me | 13:27:07 |
hexa | the only thing worse is being gaslit by bazel build jobs | 13:27:28 |
Gaétan Lepage | In reply to @hexa:lossy.network the only thing worse is being gaslit by bazel build jobs I suffer from this too | 13:27:56 |
SomeoneSerge (back on matrix) | ❯ ag cuda_runtime.h
...
pkgs/development/python-modules/torch/default.nix
453: cuda_cudart.dev # cuda_runtime.h and libraries
| 13:32:00 |
hexa | the first 7800 build steps of torchWithCuda are essentially free | 14:17:54 |
hexa | I'm not sure what happens between 7800 and 8253 | 14:18:01 |