| 18 Mar 2023 |
 mjlbach | Ah ok, so in general we should use the default's and not override them if we expect to hit the cachix cahce? | 00:55:06 |
 SomeoneSerge (back on matrix) | I see | 00:55:06 |
| mjlbach | * Ah ok, so in general we should use the default's and not override them if we expect to hit the cachix cache? | 00:55:09 |
| SomeoneSerge (back on matrix) | Btw, CUDA_PATH etc usually aren't needed | 00:55:14 |
| mjlbach | I'm planning on compiling some stuff out of tree haha | 00:55:27 |
| SomeoneSerge (back on matrix) | I mean, depends on how exotic is the stuff you're trying to build | 00:55:28 |
| mjlbach | very | 00:55:32 |
| mjlbach | 🏖️ | 00:55:38 |
| mjlbach | * 🌴🏖️🍍 | 00:55:51 |
| SomeoneSerge (back on matrix) | But if it's anything reasonable (cmake, lmao, am I really saying this) then just having pkg-config and cmake in shell's packages suffices | 00:56:04 |
| mjlbach | Oh really? It will find cuda libraries? | 00:56:22 |
| SomeoneSerge (back on matrix) | Plus the relevant bits of cudaPackages ofc | 00:56:23 |
| SomeoneSerge (back on matrix) | Usually, yes | 00:56:31 |
| mjlbach | Would be good to write all this down | 00:56:34 |
| mjlbach | * Would be good to write all this down somewhere public | 00:56:38 |
| SomeoneSerge (back on matrix) | E.g. https://github.com/SomeoneSerge/nix-torchlib-shell | 00:56:39 |
| mjlbach | Ah this is great, thanks | 00:56:57 |
| SomeoneSerge (back on matrix) | In reply to @atrius:matrix.org
Would be good to write all this down somewhere public Yea, this definitely needs more work | 00:57:18 |
| mjlbach | Have you tried mixing package versions? Often I can't use the latest nixpkgs versions of various things (e.g TIMM) and used to use poetry | 00:57:25 |
| mjlbach | In reply to @ss:someonex.net
Yea, this definitely needs more work I'm happy to collaborate on this! | 00:57:33 |
| mjlbach | I used to help out with nix tensorflow back in the day haha | 00:57:58 |
| mjlbach | It seems like the ML situation has gotten worse until very recently :/ | 00:58:18 |
| SomeoneSerge (back on matrix) | In reply to @atrius:matrix.org
Have you tried mixing package versions? Often I can't use the latest nixpkgs versions of various things (e.g TIMM) and used to use poetry No, I haven't tried much. I would usually have one revision of nixpkgs per project, and I wouldn't even fix it with flakes lately, instead I'd have a simple shell.nix use whatever is pinned system-wide, and I'd use direnv | 00:59:42 |
| mjlbach | Ah I see, I'll try to figure it out | 01:01:14 |
| mjlbach | have you thought where the info would be? Is there going to be a dedicated cuda/ml nix website like nix.dev, just cleaning up the old wiki articles? | 01:01:46 |
| SomeoneSerge (back on matrix) | I think https://nixos.wiki/wiki/CUDA is good for info. I think it would already help a great deal to have that up to date | 01:04:29 |
| SomeoneSerge (back on matrix) | Longer-term, would be nice to have a public dashboard with stats: what's cached, what's broken, when was the last time something ran | 01:05:17 |
| SomeoneSerge (back on matrix) | * Longer-term, would be nice to have a public dashboard with stats: what's cached, what's broken, since when it's broken, when was the last time something ran | 01:05:24 |
| SomeoneSerge (back on matrix) | Samuel Ainsworth: I restarted daily builds for the default set of cudaCapabilities, but only for a smaller set of fat packages. It does include pytorch, jax, tf, and then some non-sense I don't remember. Again, I feel like I shouldn't have changed it so abruptly, and we need to handle nixpkgs' defaults first | 01:08:01 |
| SomeoneSerge (back on matrix) | And maybe it's time we start thinking again about finding dedicated build nodes. I'll maybe check the hercules-in-docker situation too during the weekend, I think it should "just work" | 01:11:08 |
