| 31 Aug 2022 |
 tpw_rules | the tl;dr is that almost every generation of card requires binaries that target it specifically. binaries can't be used on cards that don't match. there are pre-binary forms which can be JITed into binaries in most circumstances, but performance can suffer. distributors would compile all binaries possible, but for the user who wants it to work on just their card, not doing that can save literal hours of compilation time | 00:46:47 |
| tpw_rules | * the tl;dr is that almost every generation of card requires binaries that target it specifically. binaries can't be used on cards that don't match. there are pre-binary forms which can be JITed into binaries in most circumstances, but performance can suffer. distributors usually compile all binaries possible, but for the user who wants it to work on just their card and has to compile from source, which is everybody for nixpkgs cuda stuff, not doing that can save literal hours of compilation time | 00:47:09 |
| tpw_rules | "all binaries possible" also depends on the cuda library version and specific package capabilities | 00:47:46 |
 Samuel Ainsworth | IIUC the tradeoff here is between user compile times and the size of cached builds, ie. not every user needs every arch but we support more than one so that users don't have to rebuild locally | 00:47:57 |
| tpw_rules | i wonder what percentage of nixpkgs cuda users use cached builds. i think it's very low | 00:48:29 |
| Samuel Ainsworth | there's no established guidelines for this atm, packages set their own cuda arch's independently | 00:48:39 |
| tpw_rules | maybe i am wrong | 00:48:41 |
| Samuel Ainsworth | I would actually assume it's quite high. tensorflowWithCuda is something like 48 CPU-hours to build | 00:49:02 |
| Samuel Ainsworth | or maybe 24... I don't remember exactly | 00:49:09 |
| Samuel Ainsworth | but it's a big boy | 00:49:12 |
| tpw_rules | yea, that's why i have a 48 core server as a remote builder and don't update my nixpkgs set except every 6 months :) | 00:49:41 |
| Samuel Ainsworth | hehe lucky you | 00:49:53 |
| Samuel Ainsworth | this was exactly why we built out the cachix cache | 00:50:16 |
| Samuel Ainsworth | https://app.cachix.org/cache/cuda-maintainers#pull | 00:50:34 |
| Samuel Ainsworth | and full wiki for context: https://nixos.wiki/wiki/CUDA | 00:50:51 |
| tpw_rules | but in any case, being able to know what you'll get and how to ask for it would be good improvements to make, even if we leave the default at "all possible" is a good thing | 00:50:52 |
| Samuel Ainsworth | yeah, that's fair | 00:51:08 |
| Samuel Ainsworth | would be nice to get all packages aligned on how to do this in a consistent manner | 00:51:30 |
| tpw_rules | yeah i've used that before, but the fact that it GCs old stuff spooks me a little. like i said i don't update much and not just because of build times, so being forced to upgrade or have to suffer the compile anyway in the future i don't like | 00:51:47 |
| Samuel Ainsworth | I haven't encountered missing things due to GC myself, but I can understand the concern | 00:52:23 |
| tpw_rules | isn't that the deal with cachix? it will only keep around the latest builds? i figured with how hongry these derivations are that time would come quickly | 00:52:46 |
| tpw_rules | but if it's been going for many months now and it hasn't had to GC anything i will reconsider my stance | 00:53:19 |
| Samuel Ainsworth | Domen Kožar has graciously offered the group some nice headroom. o/w ur right we would burn through the cache like every day | 00:53:52 |
| Samuel Ainsworth | def show cachix some love if you can! | 00:54:32 |
| tpw_rules | i've said it before but it's still true, thanks very much for leading this. i know the road is bumpy but the value is immense | 00:55:00 |
| Samuel Ainsworth | Thank you so much tpw_rules ! I really appreciate that. It can be grueling at times, so I'm really happy to hear your words of encouragement | 00:56:05 |
| Samuel Ainsworth | glad that our work is valuable to you as well! | 00:56:21 |
 hexa | sank a few hours into setting up stable-diffusion | 02:20:37 |
| hexa | no dice with rocm on my 5700 XT (NAVI 10) | 02:20:48 |
| hexa | I hate gpus | 02:20:56 |
