| 1 Mar 2023 |
 SomeoneSerge (matrix works sometimes) | I thought it would be a good idea to run nixpkgs-review with cudaSupport = true, but that just opened a hellgate: https://gist.github.com/SomeoneSerge/6cc00b41964e43f725fc12046778532d#file-218265-log-L23 | 22:38:52 |
| SomeoneSerge (matrix works sometimes) | adapta-gtk-theme, ..., chromium, ..., wine64, ... | 22:39:38 |
| SomeoneSerge (matrix works sometimes) | Mostly outPaths react to cudaSupport through gst-plugins-bad, which depend on opencv | 22:40:23 |
| SomeoneSerge (matrix works sometimes) | 😢 | 22:40:55 |
 Kevin Mittman (UTC-7) | is that the nix equivalent of portage's emerge world after changing USE flags? | 22:51:03 |
| SomeoneSerge (matrix works sometimes) | In reply to @justbrowsing:matrix.org
is that the nix equivalent of portage's emerge world after changing USE flags? I'm not familiar with Gentoo, but I think I can guess the answer | 23:44:51 |
| SomeoneSerge (matrix works sometimes) | Tbh, I'd be happy enough to build gst bad against cpu-only opencv, and then just fork the output and patchelf it to use opencv-cuda. Ad hoc as it is, it'd probably discard 90% of false-positives. Just need a good tool for "forking"... | 23:51:45 |
| 2 Mar 2023 |
 connor (he/him) | Ugh yeah changes which touch OpenCV are such a pain. I flipped when I saw my PR needed me to rebuild 700+ packages | 00:06:44 |
| SomeoneSerge (matrix works sometimes) | In reply to @FRidh:matrix.org
Yes, I think this was based on https://github.com/NixOS/rfcs/pull/3 From a quick look, couldn't tell if this was implemented after all. Do security updates result in massive rebuilds today, or are they shipped using some of these tricks? | 14:53:02 |
| SomeoneSerge (matrix works sometimes) | In reply to @connorbaker:matrix.org
With respect to specifying capabilities -- some packages (glares at magma) don't support every capability: https://github.com/NixOS/nixpkgs/pull/217410/files#diff-1e7812b78446dca0e64c4bb933e9255fca6f6539ec1ecd610edf1285a3fcbc56R55
Like, the hell? Skipping 8.6, 8.7, and 8.9? Packages like that make me think we need some way for the coda configuration and derivation to interact to agree on a list of architectures to build for. Imaging setting cudaCapabilities = [ "8.6" ]; and getting failures because, while some packages support Ampere, they don't support that capability by name. That'd be annoying right?
Or maybe that's desirable? Maybe it would be more annoying if the package used the greatest common factor (say, 8.0 when 8.6 was requested) and no errors were thrown? Is that misleading the user?
One more thing I'm afraid of: if we go with defaulting to very few capabilities (TM), I imagine the UX of a new user is basically: spend close to an hour on download, find out the stuff you downloaded doesn't work with your graphics | 18:40:23 |
| SomeoneSerge (matrix works sometimes) | Someone on Discourse mention the other day that unpacking the cudatoolkit's runfile took them about 40 minutes (we should get rid of it already) | 18:40:56 |
| SomeoneSerge (matrix works sometimes) | * Someone on Discourse mentioned the other day that unpacking the cudatoolkit's runfile took them about 40 minutes (we should get rid of it already) | 18:41:03 |
| SomeoneSerge (matrix works sometimes) | Download size is another reason for single-capability default, although it's laughable when we use the run-file | 18:45:13 |
| SomeoneSerge (matrix works sometimes) | Still. Tensorflow built for 8.6 only is 554.24MiB versus 918.59MiB for "all supported caps" | 18:46:55 |
| SomeoneSerge (matrix works sometimes) | (laughable because this doesn't account for runtime dependencies, which are 9GiB in total) | 18:47:35 |
| SomeoneSerge (matrix works sometimes) | * (laughable because this doesn't account for runtime dependencies, which are ~~9GiB in total~~ 9GiB for 8.6-only and 11.15 for [3.5, ..., 8.6]) | 18:48:12 |
| SomeoneSerge (matrix works sometimes) | * (laughable because this doesn't account for runtime dependencies, which are ~~9GiB in total~~ 9GiB for 8.6-only and 11.15GiB for [3.5, ..., 8.6]) | 18:48:23 |
| SomeoneSerge (matrix works sometimes) | Oh my god, pytorch has migrated to FindCUDAToolkit.cmake!!!!!!!!!!! | 19:09:28 |
| SomeoneSerge (matrix works sometimes) | Fun fact: we build stuff for sm_35, but cudnn only ships with 5.0+
❯ cuobjdump result/lib/libcudnn_cnn_infer.so.8.8.0 | rg 'arch =' | sort | uniq
arch = sm_50
arch = sm_60
arch = sm_61
arch = sm_70
arch = sm_75
arch = sm_80
arch = sm_86
arch = sm_90
| 19:23:11 |
| connor (he/him) | In reply to @ss:someonex.net
Fun fact: we build stuff for sm_35, but cudnn only ships with 5.0+
❯ cuobjdump result/lib/libcudnn_cnn_infer.so.8.8.0 | rg 'arch =' | sort | uniq
arch = sm_50
arch = sm_60
arch = sm_61
arch = sm_70
arch = sm_75
arch = sm_80
arch = sm_86
arch = sm_90
That’s because it supports CUDA 12 I guess, which drops pre 5.0 support | 20:09:44 |
| SomeoneSerge (matrix works sometimes) | Ah, I see | 20:39:03 |
| SomeoneSerge (matrix works sometimes) | connor (he/him): when you build locally, which capabilities do you build against? | 22:14:11 |
| connor (he/him) | Aw we can only vote once? | 22:22:24 |
| SomeoneSerge (matrix works sometimes) | trying to figure out the interface | 22:22:34 |
| SomeoneSerge (matrix works sometimes) | I hoped it would be mulitple options | 22:22:40 |
| connor (he/him) | In reply to @ss:someonex.net
connor (he/him): when you build locally, which capabilities do you build against? I use 8.0 because some packages -- glares at magma -- don't support 8.6/8.9 | 22:23:16 |
| SomeoneSerge (matrix works sometimes) | In reply to @connorbaker:matrix.org
I use 8.0 because some packages -- glares at magma -- don't support 8.6/8.9 You mentioned and I still haven't taken a look. How the hell does it not support 8.6? | 22:25:32 |
| connor (he/him) | Oh you'll like this -- they have a hardcoded list of supported capabilities: https://bitbucket.org/icl/magma/src/f4ec79e2c13a2347eff8a77a3be6f83bc2daec20/CMakeLists.txt#lines-279 | 22:27:31 |
| connor (he/him) | That section of the CMakeLists.txt has them doing matches based on the capability. Earlier they do matches on the architecture name (like Ampere, etc.). But even if you specify Ampere you still only get sm_80. | 22:28:31 |
| connor (he/him) | A little bit farther down they do the same thing for AMD GPUs -- first by architecture name, then by a hardcoded list of names: https://bitbucket.org/icl/magma/src/f4ec79e2c13a2347eff8a77a3be6f83bc2daec20/CMakeLists.txt#lines-386. | 22:29:22 |
