| 2 Mar 2023 |
 SomeoneSerge (matrix works sometimes) | Oh my god, pytorch has migrated to FindCUDAToolkit.cmake!!!!!!!!!!! | 19:09:28 |
| SomeoneSerge (matrix works sometimes) | Fun fact: we build stuff for sm_35, but cudnn only ships with 5.0+
❯ cuobjdump result/lib/libcudnn_cnn_infer.so.8.8.0 | rg 'arch =' | sort | uniq
arch = sm_50
arch = sm_60
arch = sm_61
arch = sm_70
arch = sm_75
arch = sm_80
arch = sm_86
arch = sm_90
| 19:23:11 |
 connor (he/him) | In reply to @ss:someonex.net
Fun fact: we build stuff for sm_35, but cudnn only ships with 5.0+
❯ cuobjdump result/lib/libcudnn_cnn_infer.so.8.8.0 | rg 'arch =' | sort | uniq
arch = sm_50
arch = sm_60
arch = sm_61
arch = sm_70
arch = sm_75
arch = sm_80
arch = sm_86
arch = sm_90
That’s because it supports CUDA 12 I guess, which drops pre 5.0 support | 20:09:44 |
| SomeoneSerge (matrix works sometimes) | Ah, I see | 20:39:03 |
| SomeoneSerge (matrix works sometimes) | connor (he/him): when you build locally, which capabilities do you build against? | 22:14:11 |
| connor (he/him) | Aw we can only vote once? | 22:22:24 |
| SomeoneSerge (matrix works sometimes) | trying to figure out the interface | 22:22:34 |
| SomeoneSerge (matrix works sometimes) | I hoped it would be mulitple options | 22:22:40 |
| connor (he/him) | In reply to @ss:someonex.net
connor (he/him): when you build locally, which capabilities do you build against? I use 8.0 because some packages -- glares at magma -- don't support 8.6/8.9 | 22:23:16 |
| SomeoneSerge (matrix works sometimes) | In reply to @connorbaker:matrix.org
I use 8.0 because some packages -- glares at magma -- don't support 8.6/8.9 You mentioned and I still haven't taken a look. How the hell does it not support 8.6? | 22:25:32 |
| connor (he/him) | Oh you'll like this -- they have a hardcoded list of supported capabilities: https://bitbucket.org/icl/magma/src/f4ec79e2c13a2347eff8a77a3be6f83bc2daec20/CMakeLists.txt#lines-279 | 22:27:31 |
| connor (he/him) | That section of the CMakeLists.txt has them doing matches based on the capability. Earlier they do matches on the architecture name (like Ampere, etc.). But even if you specify Ampere you still only get sm_80. | 22:28:31 |
| connor (he/him) | A little bit farther down they do the same thing for AMD GPUs -- first by architecture name, then by a hardcoded list of names: https://bitbucket.org/icl/magma/src/f4ec79e2c13a2347eff8a77a3be6f83bc2daec20/CMakeLists.txt#lines-386. | 22:29:22 |
| SomeoneSerge (matrix works sometimes) | This looks suspiciously similar to opencv's script, as if they just changed the names and move a few lines around | 22:30:06 |
| connor (he/him) | Now I don't have experience with CMake or compiling/linking GPU-accelerated code anywhere near as much as they do, but... WHY does it make sense to do stuff like that if they're not going to do anything with those values? Why make the CMakeLists.txt the source of truth for what's supported instead of letting the compiler (NVCC) do that when you pass it those flags? | 22:31:12 |
| SomeoneSerge (matrix works sometimes) | But it looks like if we pass sm_86 explicitly, they would just keep it there? | 22:31:40 |
| SomeoneSerge (matrix works sometimes) | Ah, I see | 22:32:10 |
| SomeoneSerge (matrix works sometimes) | We need to modify CUDA_ARCHITECTURES directly | 22:32:18 |
| SomeoneSerge (matrix works sometimes) | OTherwise they fail to convert sm_86 to 86 | 22:32:34 |
| connor (he/him) | Yeah, and MIN_ARCH isn't set (because none of those cases match) and we get an error | 22:32:53 |
| SomeoneSerge (matrix works sometimes) | Alright. This looks clumsy, but it's nothing we can't fix | 22:33:16 |
| connor (he/him) | In reply to @ss:someonex.net
This looks suspiciously similar to opencv's script, as if they just changed the names and move a few lines around iirc there was a fair amount of copy-pasting of CMake (specifically CUDA-related stuff) configurations prior to first-party support, so it's possible they all share a common ancestor or a research advisor just pointed a grad student at something that worked and told them to do what they did lol | 22:34:11 |
| connor (he/him) | In reply to @ss:someonex.net
Alright. This looks clumsy, but it's nothing we can't fix What do you envision, implementation-wise, as a fix? Do you see a .patch we apply, or something else?
Is it worth submitting a PR upstream to see if they can use built-in CMake functionality?
| 22:38:49 |
| SomeoneSerge (matrix works sometimes) | Well, if they just switched to FindCUDAToolkit, that would've been perfect | 22:39:37 |
| SomeoneSerge (matrix works sometimes) | I'm wondering now if cmake would override CUDA_ARCHITECTURES if we just gave them on the command line | 22:39:56 |
| SomeoneSerge (matrix works sometimes) | (idk, does one need a CACHE flag in set for that, or does it just work) | 22:40:21 |
| SomeoneSerge (matrix works sometimes) | In reply to @ss:someonex.net
I'm wondering now if cmake would override CUDA_ARCHITECTURES if we just gave them on the command line Wow, I think it just worked | 22:49:45 |
| connor (he/him) | In reply to @ss:someonex.net
Wow, I think it just worked Do you mean instead of passing GPU_TARGET just providing CUDA_ARCHITECTURES? | 22:54:38 |
| SomeoneSerge (matrix works sometimes) | Yes, and MIN_ARCH. I'm waiting for the build to finish to check the binaries, I might have misunderstood their logs | 22:55:28 |
| connor (he/him) | I'll check with MIN_ARCH too -- when I got rid of GPU_TARGET it switched to building the default set of GPU targets and I saw this in the CMake logs:
magma> -- Found CUDA
magma> -- CUDA_CUDART_LIBRARY: CUDA::cudart
magma> -- compile for CUDA arch 3.0 (Kepler)
magma> -- compile for CUDA arch 3.5 (Kepler)
magma> -- compile for CUDA arch 3.7 (Kepler)
magma> -- compile for CUDA arch 5.0 (Maxwell)
magma> -- compile for CUDA arch 6.0 (Pascal)
magma> -- Define -DMAGMA_HAVE_CUDA -DMAGMA_CUDA_ARCH_MIN=300
| 22:56:23 |
