| 28 Dec 2024 |
 connor (burnt/out) (UTC-8) | ugh thinking about software making me sad
Samuel Ainsworth did you ever find some sort of serenity with CUDA and Nixpkgs? | 00:43:26 |
| connor (burnt/out) (UTC-8) | I'm having thoughts about https://github.com/connorbaker/cuda-packages.
In particular, does it make sense to include CUDA stuff in Nixpkgs proper when we can't take advantage of anything but eval checks?
Would nix-community be a better home?
Just having a growing sense of dread about updating and trying to maintain fast-moving libraries in an environment where stuff can (or does) break constantly and there's no notification of such breakage (except maybe by the community Hydra instance?).
There's also the understanding that in Nixpkgs, everything work together simultaneously.
As an example, I'd hate to try to upgrade OpenCV (or PyTorch) so it works with newer versions of CUDA, only to find out it causes some gnarly Darwin/ROCm/non-CUDA issue.
Thinking out-of-tree designs would afford us the ability to break stuff, though that comes with a number of drawbacks (duplicating nix expressions for packages and having slight variations, merging in upstream changes, etc.).
Maybe this is just fatigue talking; I think a number of these complaints were raised in a discourse post Sam made a few years ago. | 00:54:19 |
| connor (burnt/out) (UTC-8) | I mean, I certainly want to upstream the library functions and additional setup hooks/logging functionality I wrote because they're (in my opinion) widely useful.
Just... the CUDA stuff. | 00:55:26 |
| connor (burnt/out) (UTC-8) | * I'm having thoughts about https://github.com/connorbaker/cuda-packages. In particular, does it make sense to include CUDA stuff in Nixpkgs proper when we can't take advantage of anything but eval checks? Would nix-community be a better home? Just having a growing sense of dread about updating and trying to maintain fast-moving libraries in an environment where stuff can (or does) break constantly and there's no notification of such breakage (except maybe by the community Hydra instance?). There's also the understanding that in Nixpkgs, everything work together simultaneously. As an example, I'd hate to try to upgrade OpenCV (or PyTorch) so it works with newer versions of CUDA, only to find out it causes some gnarly Darwin/ROCm/non-CUDA issue. Thinking out-of-tree designs would afford us the ability to break stuff, though that comes with a number of drawbacks (duplicating nix expressions for packages and having slight variations, merging in upstream changes, etc.). Maybe this is just fatigue talking; I think a number of these complaints were raised in a discourse post Samuel made a few years ago. | 12:33:14 |
 lromor | Is anyone at chaos congress? | 16:16:44 |
 @trofi:matrix.org | Good idea! Done as https://github.com/NixOS/nix/issues/12106#issuecomment-2564375843 | 16:36:40 |
|  matthewcroughan changed their display name from matthewcroughan to matthewcroughan (DECT: 56490). | 18:41:55 |
| 29 Dec 2024 |
| lromor set a profile picture. | 16:13:20 |
| connor (burnt/out) (UTC-8) | Just tried to build PyTorch and I completely forgot it vendors its dependencies, was stunned to see it building ONNX | 21:49:20 |
 SomeoneSerge (back on matrix) | I wish... matthewcroughan (DECT: 56490) maybe? | 21:50:20 |
| SomeoneSerge (back on matrix) | Yeah | 21:50:35 |
| connor (burnt/out) (UTC-8) | I remember I had tried to work on using system-provided dependencies (I guess more than a year ago now) and gave up because it would have required a bunch CMake rewriting.
And every time upstream changed something, BOOM! Another merge conflict or more rewriting required.
But I suppose it’s that way with lots of projects. | 21:55:11 |
| connor (burnt/out) (UTC-8) | Serge, how do you stay upbeat about packaging stuff? | 21:55:58 |
| SomeoneSerge (back on matrix) | Yes, which is why this is really is about working with the upstream and getting the changes through on their side, not on nixpkgs side | 21:56:38 |
| SomeoneSerge (back on matrix) | I clearly don't... | 21:57:31 |
 Gaétan Lepage | In reply to @connorbaker:matrix.org
I remember I had tried to work on using system-provided dependencies (I guess more than a year ago now) and gave up because it would have required a bunch CMake rewriting.
And every time upstream changed something, BOOM! Another merge conflict or more rewriting required.
But I suppose it’s that way with lots of projects. At least we still build pytorch from source... looking at you protobuf-python, tensorflow and, since today, jax | 21:57:59 |
| Gaétan Lepage | 🥲 | 21:58:05 |
| Gaétan Lepage | At least they take less resources to build 🤡 | 21:58:59 |
| Gaétan Lepage | * At least they take less resources to "build" 🤡 | 21:59:04 |
| connor (burnt/out) (UTC-8) | In reply to @ss:someonex.net
Yes, which is why this is really is about working with the upstream and getting the changes through on their side, not on nixpkgs side Thoughts on what to do when upstream makes it clear they don’t care or don’t want to implement changes that make it easier (or feasible) to build with Nix? | 21:59:14 |
| SomeoneSerge (back on matrix) | In case of pytorch, I think they are willing to accept stuff | 21:59:41 |
| SomeoneSerge (back on matrix) | They just won't do it themselves | 21:59:53 |
| Gaétan Lepage | Also, I'm afraid we are severly under-staffed :/ | 22:00:12 |
| connor (burnt/out) (UTC-8) | I’ve had good experiences with them too; I meant more like NVIDIA and the ONNX ecosystem | 22:00:14 |
| Gaétan Lepage | Since the latest staging-next merge, everything is kind of broken... | 22:00:38 |
| Gaétan Lepage | Hopefully, we merged the triton-llvm fix and the jax/jaxlib switch to bin. | 22:00:54 |
| SomeoneSerge (back on matrix) | Yeah right... like, pray that they lose the market? | 22:00:55 |
